Firmware Studio Development Environment

Host Builder Tester Tokenizer Forthing

Tokenizer Interpreter

Tokenizer mode is for appending tokenized code to the target image. In the example below I launched Firmware Studio, loaded a token assignment file, and loaded the driver code for an LCD module. The resulting lcd_emit.sb file is the ROM image (tokenized code with a boilerplate) that gets loaded into the LCD module's serial EEPROM.

All this was done without source code for the system's ROM. An end user can write add-on code given only the token assignment file and some guidance as to how to use the language.

You can enter tokenizer mode by typing TI or TOKENIZING. In tokenizer mode, the left margin displays the same as in host mode.

The tokenizer interpreter is the simplest of all the interpreters:

Search the search order for the word
Found?
IF Execute it
Was this a target word?
IF compile a token for it
ENDIF
ELSE
Try to convert the word to a number
Converted ok?
IF Compile the token for literal
ELSE Error: word is unrecognized
ENDIF
ENDIF

Token values are represented by either one or two bytes. The most-used tokens are 1-byte values between 32 and 255. Values between 1 and 31 cause a second byte to be fetched and combined with the first to form a number between 256 and 8191. If the first byte is zero, two bytes are fetched to form all remaining 16-bit values.

Token value zero and any other three-byte tokens are included for future expansion. Currently, TOF doesn't use and probably won't work with three-byte tokens.