Day 03 of 30 · Compiler foundations
Turn characters into tokens
- Time
- About 150 minutes: 40 reading, 80 lab, 20 testing, 10 notes.
- Prerequisites
- Day 01: Understand what a compiler does
Toolchain: your host C++ compiler
Today’s lab is the toy expression frontend. Any working C++ compiler builds it; no LLVM libraries are needed yet. Setup guide
Learn
- Identifiers, integer literals, punctuation, whitespace, source locations and end-of-file.
Read Reading
- R1 · Learn LLVM 17, Second Edition
Chapter 2: lexical analysis · 25 min - R3 · Crafting Interpreters
Scanning discussion; use it for concepts, not a second implementation · 15 min
Reading budgets cover selected sections, not whole chapters. Link to the book; this site does not reproduce it.
Build Build or investigate
- Start a tiny expression language in C++.
- Write a lexer for integers, names, +, -, *, parentheses, = and semicolons.
- Track line and column.
- Reject an unknown character and an overflowing integer literal cleanly.
Done when
Token output for valid input plus tests for whitespace, end-of-file and malformed input.
Recall question
Why should tokens retain a location after whitespace is discarded?
Completion checks
Progress tracking needs JavaScript. The lesson above is complete without it.
Notes
Saved only in this browser. Keep it short enough to finish in ten minutes; none of the prompts are required. All notes, export and import