All threads / Stuck writing an interpreter in Ruby. Not able to consume whitespace properly

Ask A Question

Notifications

You’re not receiving notifications from this thread.

Stuck writing an interpreter in Ruby. Not able to consume whitespace properly

MontieFernandes asked in General

So I am following the "Writing an interpreter book" and implementing it in Ruby instead of Go. I am able to scan tokens like ; =, +, etc but it seems to behave differently when I have identifiers like let, 10, etc in my input string. Tried to hunt this bug this whole week but in vain so I thought a fresh pair of eyes might be able to catch it. https://myip.kim/

Here is an overview.

The codebase is very small and most of the logic resides in lib/lexer/lexer.rb https://birthdaywishes.onl/

The class Lexer maintains the following state a cursor for the current character in the input string, a cursor for the next character and the current character in the input string

Lexer has the following methods https://elecpay.in/tneb/

read_char which sets the data members to appropriate values

read_indentifier which is used to extract all the characters belonging to strings that are not reserved keywords but identifiers and call read_char before returning

read_number same as read_identifier but for numbers

consume_whitespace to skip over spaces, newlines, etc

next_token used to match the current character with the appropriate case and return its Token object defined in lib/token/token.rb and call read_char to increment the cursors before returning

This is the input string I use in my lexer_test.rb

Join the discussion

Want to stay up-to-date with Ruby on Rails?

Join 37,629+ developers who get early access to new tutorials, screencasts, articles, and more.

    We care about the protection of your data. Read our Privacy Policy.

    logo Created with Sketch.

    Ruby on Rails tutorials, guides, and screencasts for web developers learning Ruby, Rails, Javascript, Turbolinks, Stimulus.js, Vue.js, and more. Icons by Icons8

    © 2020 GoRails, LLC. All rights reserved.