Ask A Question


You’re not receiving notifications from this thread.

Stuck writing an interpreter in Ruby. Not able to consume whitespace properly

MontieFernandes asked in General

So I am following the "Writing an interpreter book" and implementing it in Ruby instead of Go. I am able to scan tokens like ; =, +, etc but it seems to behave differently when I have identifiers like let, 10, etc in my input string. Tried to hunt this bug this whole week but in vain so I thought a fresh pair of eyes might be able to catch it.

Here is an overview.

The codebase is very small and most of the logic resides in lib/lexer/lexer.rb

The class Lexer maintains the following state a cursor for the current character in the input string, a cursor for the next character and the current character in the input string

Lexer has the following methods

read_char which sets the data members to appropriate values

read_indentifier which is used to extract all the characters belonging to strings that are not reserved keywords but identifiers and call read_char before returning

read_number same as read_identifier but for numbers

consume_whitespace to skip over spaces, newlines, etc

next_token used to match the current character with the appropriate case and return its Token object defined in lib/token/token.rb and call read_char to increment the cursors before returning

This is the input string I use in my lexer_test.rb

Join the discussion
Create an account Log in

Want to stay up-to-date with Ruby on Rails?

Join 79,047+ developers who get early access to new tutorials, screencasts, articles, and more.

    We care about the protection of your data. Read our Privacy Policy.

    Screencast tutorials to help you learn Ruby on Rails, Javascript, Hotwire, Turbo, Stimulus.js, PostgreSQL, MySQL, Ubuntu, and more.

    © 2023 GoRails, LLC. All rights reserved.