Skip to main content
Ask A Question
You’re not receiving notifications from this thread.

Stuck writing an interpreter in Ruby. Not able to consume whitespace properly

General • Asked by MontieFernandes

So I am following the "Writing an interpreter book" and implementing it in Ruby instead of Go. I am able to scan tokens like ; =, +, etc but it seems to behave differently when I have identifiers like let, 10, etc in my input string. Tried to hunt this bug this whole week but in vain so I thought a fresh pair of eyes might be able to catch it.

Here is an overview.

The codebase is very small and most of the logic resides in lib/lexer/lexer.rb

The class Lexer maintains the following state a cursor for the current character in the input string, a cursor for the next character and the current character in the input string

Lexer has the following methods

read_char which sets the data members to appropriate values

read_indentifier which is used to extract all the characters belonging to strings that are not reserved keywords but identifiers and call read_char before returning

read_number same as read_identifier but for numbers

consume_whitespace to skip over spaces, newlines, etc

next_token used to match the current character with the appropriate case and return its Token object defined in lib/token/token.rb and call read_char to increment the cursors before returning

This is the input string I use in my lexer_test.rb

Login or Create An Account to join the conversation.

Subscribe to the newsletter

Join 31,152+ developers who get early access to new screencasts, articles, guides, updates, and more.

    By clicking this button, you agree to the GoRails Terms of Service and Privacy Policy.

    More of a social being? We're also on Twitter and YouTube.