Ask A Question


You’re not receiving notifications from this thread.

Stuck writing an interpreter in Ruby. Not able to consume whitespace properly

MontieFernandes asked in General

So I am following the "Writing an interpreter book" and implementing it in Ruby instead of Go. I am able to scan tokens like ; =, +, etc but it seems to behave differently when I have identifiers like let, 10, etc in my input string. Tried to hunt this bug this whole week but in vain so I thought a fresh pair of eyes might be able to catch it.

Here is an overview.

The codebase is very small and most of the logic resides in lib/lexer/lexer.rb

The class Lexer maintains the following state a cursor for the current character in the input string, a cursor for the next character and the current character in the input string

Lexer has the following methods

read_char which sets the data members to appropriate values

read_indentifier which is used to extract all the characters belonging to strings that are not reserved keywords but identifiers and call read_char before returning

read_number same as read_identifier but for numbers

consume_whitespace to skip over spaces, newlines, etc

next_token used to match the current character with the appropriate case and return its Token object defined in lib/token/token.rb and call read_char to increment the cursors before returning

This is the input string I use in my lexer_test.rb

Join the discussion
Create an account Log in

Want to stay up-to-date with Ruby on Rails?

Join 82,464+ developers who get early access to new tutorials, screencasts, articles, and more.

    We care about the protection of your data. Read our Privacy Policy.