So I am following the "Writing an interpreter book" and implementing it in Ruby instead of Go. I am able to scan tokens like ; =, +, etc but it seems to behave differently when I have identifiers like let, 10, etc in my input string. Tried to hunt this bug this whole week but in vain so I thought a fresh pair of eyes might be able to catch it. https://myip.kim/
Here is an overview.
The codebase is very small and most of the logic resides in lib/lexer/lexer.rb https://birthdaywishes.onl/
The class Lexer maintains the following state a cursor for the current character in the input string, a cursor for the next character and the current character in the input string
Lexer has the following methods https://elecpay.in/tneb/
read_char which sets the data members to appropriate values
read_indentifier which is used to extract all the characters belonging to strings that are not reserved keywords but identifiers and call read_char before returning
read_number same as read_identifier but for numbers
consume_whitespace to skip over spaces, newlines, etc
next_token used to match the current character with the appropriate case and return its Token object defined in lib/token/token.rb and call read_char to increment the cursors before returning
This is the input string I use in my lexer_test.rb
Join 31,152+ developers who get early access to new screencasts, articles, guides, updates, and more.