Episode 5 Tinker and the Long-Awaited 3

The Life of Code 06

Who Made the Compiler?

Is a compiler a program someone wrote, too?

Liney has met every friend on the first lap, and one question is left.

This time we look at the tool that checked Liney.

Today's odd question

What will we find in the first ten lines of the compiler's entry file?

The answer is on the back of the map.

Let's turn it over with Seedling.

LineySo who made the tool that checked me?

SeedlingPeople did.The compiler that checked you is a program someone wrote, Liney.Want to see which language they used?

LineyWhoa, the back is full of writing!

SeedlingIt's the source of the compiler that checked you.Today we've copied out its first ten lines.

Lineysrc/main.tpz? It ends just like my file name!

SeedlingThe compiler's settings file names this file as its starting place.Like you, this file is written in Topaz.

TwigHang on, haven't I seen this word before?

SeedlingYou don't live in here, of course.But the jobs we explored on the first lap are written here by name.Want to find them?

TinkerWhat's that 5.21.1 in the corner?

SeedlingIt's the version of Topaz we opened.That number tells us which release this source came from.

Now it's your turn to read the back of the map.

Experiment

The Back of the Map

Liney's file is called main.tpz.

The compiler's settings point to a file called src/main.tpz as its starting place.

Both names end in main.tpz.

Let's open the compiler's file and read its first ten lines.

Nothing runs on this page. The text on the back is copied exactly from the source Topaz published.

Liney's file

main.tpz

  1. 1 + 2

6 bytes · 1 line · The file we've followed since Episode 1

The compiler's entry file

src/main.tpz

16,891 bytes · The first 10 of 560 lines · Topaz 5.21.1 public source

Topaz 5.21.1 public source

What's on the other side of the map?

Compiler source src/main.tpz, lines 1 to 10
  1. import std.json as jsonBoundary
  3. import checker { check }
  4. import emitter { emitRust, emitStep }
  5. import layout { normalize }
  6. import lowering { lowerStep, lowerValue }
  7. import parser { parse }
  8. import profile { checkProfile }
  9. import raw { lex }
  10. import resolver { resolvePreview, resolveWithFacts }

Think back to the job you explored with this friend on the first lap. Which line brings in a name for that job?

Line for Snip's pin

With Snip, we split 1 + 2 into tokens. Episode 1 called that job lexical analysis. Which line has a name that looks like the start of lexical?

With Snip, we split 1 + 2 into tokens. Episode 1 called that job lexical analysis. Which line has a name that looks like the start of lexical?

With Snip, we split 1 + 2 into tokens. Episode 1 called that job lexical analysis. Which line has a name that looks like the start of lexical?

With Snip, we split 1 + 2 into tokens. Episode 1 called that job lexical analysis. Which line has a name that looks like the start of lexical?

With Snip, we split 1 + 2 into tokens. Episode 1 called that job lexical analysis. Which line has a name that looks like the start of lexical?

With Snip, we split 1 + 2 into tokens. Episode 1 called that job lexical analysis. Which line has a name that looks like the start of lexical?

With Snip, we split 1 + 2 into tokens. Episode 1 called that job lexical analysis. Which line has a name that looks like the start of lexical?

With Snip, we split 1 + 2 into tokens. Episode 1 called that job lexical analysis. Which line has a name that looks like the start of lexical?

lex is the name. It matches the first three letters of lexical. Line 261 of the full file passes the source in with lex(request.source), and the very next line takes out raw.tokens.

With Snip, we split 1 + 2 into tokens. Episode 1 called that job lexical analysis. Which line has a name that looks like the start of lexical?

Line for Twig's pin

With Twig, we grouped the pieces into a tree. Episode 2 called that job parsing. Which line has a name that looks like that word?

With Twig, we grouped the pieces into a tree. Episode 2 called that job parsing. Which line has a name that looks like that word?

With Twig, we grouped the pieces into a tree. Episode 2 called that job parsing. Which line has a name that looks like that word?

With Twig, we grouped the pieces into a tree. Episode 2 called that job parsing. Which line has a name that looks like that word?

With Twig, we grouped the pieces into a tree. Episode 2 called that job parsing. Which line has a name that looks like that word?

With Twig, we grouped the pieces into a tree. Episode 2 called that job parsing. Which line has a name that looks like that word?

parse is the name. It connects to the parsing in Episode 2, where we grouped the pieces into a tree.

With Twig, we grouped the pieces into a tree. Episode 2 called that job parsing. Which line has a name that looks like that word?

With Twig, we grouped the pieces into a tree. Episode 2 called that job parsing. Which line has a name that looks like that word?

With Twig, we grouped the pieces into a tree. Episode 2 called that job parsing. Which line has a name that looks like that word?

Line for Tami's pin

No pin yet

With Tami, we looked at the types of values. Which line names that job?

check is the name. The first comment in checker.tpz says this module does type checking.

With Tami, we looked at the types of values. Which line names that job?

With Tami, we looked at the types of values. Which line names that job?

With Tami, we looked at the types of values. Which line names that job?

With Tami, we looked at the types of values. Which line names that job?

You spotted check here too. Which line imports from checker, the module for the type check we saw in Episode 3?

With Tami, we looked at the types of values. Which line names that job?

Tami looks at names too. But where is the job of checking the types of values, as in Episode 3?

Line for Momo's pin

No pin yet

With Momo, we copied the tree onto operation cards. Which line names that job?

With Momo, we copied the tree onto operation cards. Which line names that job?

Line 4 imports two names from emitter. The compiler calls emitRust to generate Rust code. Which line names the lowering we saw in Episode 4?

With Momo, we copied the tree onto operation cards. Which line names that job?

lowerStep and lowerValue are the names. Both connect to the lowering in Episode 4, where we copied the tree onto cards.

With Momo, we copied the tree onto operation cards. Which line names that job?

With Momo, we copied the tree onto operation cards. Which line names that job?

With Momo, we copied the tree onto operation cards. Which line names that job?

With Momo, we copied the tree onto operation cards. Which line names that job?

Try each of the four friends' pins once and the discovery card opens.

We found familiar jobs in the compiler's entry file.

The import lines show where this file gets its names.

Farther down, you can see one of them at work in lex(request.source).

The four friends' lines

  • Snip Line 9 import raw { lex }

    lex is the name. It matches the first three letters of lexical. Line 261 of the full file passes the source in with lex(request.source), and the very next line takes out raw.tokens.

  • Twig Line 7 import parser { parse }

    parse is the name. It connects to the parsing in Episode 2, where we grouped the pieces into a tree.

  • Tami Line 3 import checker { check }

    check is the name. The first comment in checker.tpz says this module does type checking.

  • Momo Line 6 import lowering { lowerStep, lowerValue }

    lowerStep and lowerValue are the names. Both connect to the lowering in Episode 4, where we copied the tree onto cards.

More imports to explore
  • Line 1 brings in the std.json module of the standard library under the name jsonBoundary.
  • Line 2 is a blank line in the original.
  • Line 4 imports two names from emitter. The compiler calls emitRust to generate Rust code. Which line names the lowering we saw in Episode 4?
  • Line 5 brings in normalize from layout. Line 262 uses it on raw.tokens. See how normalize handles line breaks in Episode 7
  • Line 8 brings in checkProfile from profile. Which line imports from checker, the module for the type check we saw in Episode 3?
  • Line 10 brings in resolvePreview and resolveWithFacts from resolver. To resolve a name is to find what it points to.

Your computer runs the installed topaz file. The text we're reading comes from the source Topaz published.

Where does this record come from?

The settings file topaz.toml says entry = "src/main.tpz".

Only the first 10 of 560 lines are copied here. The whole file is in the official source archive.

Public source archive
topaz-source-snapshot-v5.21.1.tar.gz
Archive bytes
305,303
Archive sha256
d3a0e5365b8a39605f095a7c07b43556848a95042e7efd346ab84c95e583cf3d
File
compiler/crates/topaz_self_frontend/topaz/src/main.tpz
File bytes
16,891
File sha256
ef16cb39e9956e7e3aaed8023f9a45388c49a17bd9336a6e6f29dfa5aeb63bd3
Downloaded on
2026-11-02

The copied source comes under the license and notice of the same release. Read LICENSE Read NOTICE

The map and the pins are part of the story. The text and numbers on the back come from the published originals.

The real story

A compiler is a program someone wrote, too

The writing on the back of the map is the compiler's source code. Like Liney, it's a program someone wrote out in text.

The Topaz documentation says The default compiler is written in Topaz. Toolchain Status

A self-hosting compiler is written in the language it compiles and can compile its own source.

The compiler's folder holds 50 .tpz files. Its entry file, main.tpz, is 560 lines long. Liney's main.tpz is one line.

Inside the compiler, let gives a value a name too. Line 261 has let raw = lex(request.source), and the very next line takes out that raw.tokens.

We used the default compiler in Topaz 5.21.1 for the token, tree, type and operation records in Episodes 1 to 4. Today we opened that compiler's published source.

The copied source comes under the license and notice of the same release. Read LICENSE Read NOTICE

Do the friends live in there?

No. The friends are how the story pictures things. The compiler splits its work across files and names, and some of those names look like the jobs we explored on the first lap.

The import lines from line 3 to line 10 are in alphabetical order of their module names, not the order the work happens in. How the work is split, and in what order, can differ from compiler to compiler.

Is every compiler written in its own language?

No. Some compilers are written in the language they compile, and some are written in another language.

Topaz 5.21.1 records and public source

Look closer

Two compilers?

Written in the record Ask Topaz 5.21.1 for its status and it names a default compiler and a reference compiler.

Default compiler: self
Reference compiler: rust

Written in the record It lists self as the default compiler and rust as the reference compiler.

The Topaz documentation shows how to use the reference compiler to compare results for the same input.

Written in the record We checked 1 + 2 three times. The note tray names the compiler selected each time.

Written in the record A check that chose nothing says current-default, and a check that named a compiler says explicit.

Compared by this site The two lines in the screen window matched byte for byte across all three checks.

Setting the two compilers side by side to see whether they do the same work is for a later lap.

Original files topaz compiler statustopaz compiler status --json

Read the provenance record

The 1 + 2 observation behind Momo's cards in Episode 4 also has a provenance file, provenance.json.

Written in the record The Episode 4 record says two successive stages produced identical Rust bytes.

Written in the record It lists the fixed point for the intermediate records as not-established.

Written in the record For the executable-byte comparison, it says not-run.

Written in the record The record lists rustc 1.96.0 under the tools used to link this compiler.

Compared by this site The nine fields we compared had the same values in 5.21.0 and 5.21.1. The two provenance files differed at the byte level.

A bundle called the seed

Topaz 5.21.1's compiler seed is published as one compressed bundle and a list of what's in it.

Compared by this site The seed contains the compiler source written in Topaz, the Rust generated from it, runtime code and build scripts.

Compared by this site Three identifiers in the seed list match the installed compiler's provenance record.

Compared by this site We also took the Rust file out of the bundle and found that its hash matched the one listed.

Seed bundle
topaz-bootstrap-seed-v5.21.1.tar.gz
Bundle bytes
1,926,265
Bundle sha256
cf6a09b66fa5197472808fc01388e799b394686fb705b8e977cc39962583e974

The three identifiers we matched

  • Installed compiler's provenancecompiler.sourceSetIdsha256:17d598ebcb0dea65195aba8c0b3ab20f94003a7f343714f7b1df25d790cdacd5
    Seed listsourceReuse.compilerSourceSetIdsha256:17d598ebcb0dea65195aba8c0b3ab20f94003a7f343714f7b1df25d790cdacd5
  • Installed compiler's provenancecompiler.runtimeSourceSetIdsha256:85ed8d3b0571c34bc40c3ffd1be8bcda2a539d6c82c92b417c834507200d9c60
    Seed listsourceReuse.runtimeSourceSetIdsha256:85ed8d3b0571c34bc40c3ffd1be8bcda2a539d6c82c92b417c834507200d9c60
  • Installed compiler's provenancecompiler.generatedRustSha256sha256:5f32cee07a1adea1c67ba8ac8fbd4f3ee4051f019a7856fcdb228740a1d4b8b1
    Seed listsourceReuse.generatedRustSha256sha256:5f32cee07a1adea1c67ba8ac8fbd4f3ee4051f019a7856fcdb228740a1d4b8b1

Hash we took again from the file in the bundle generated/compiler.rs 5f32cee07a1adea1c67ba8ac8fbd4f3ee4051f019a7856fcdb228740a1d4b8b1

That's as far as this site went.

Compared by this site We matched the compiler's 50 .tpz files by path across the seed lists for 5.21.0 and 5.21.1. Nine files had different hashes. The entry file we opened had the same hash in both.

Files with different hashes

  • checker/call_complete.tpz
  • checker/call_facts.tpz
  • checker/expression_control.tpz
  • checker/expression_core.tpz
  • checker/members_capabilities.tpz
  • checker/statements.tpz
  • checker/type_algebra.tpz
  • emitter/emit.tpz
  • resolver/scopes_walk.tpz

Written in the record The publisher records whether it built the seed bundle in built and whether it regenerated the Rust in regenerated.

built
true
regenerated
true

Written in the record The seed's guide says The compiler written in Topaz performs parsing, checking, lowering and Rust generation.

A question for the next lap

Will the regenerated Rust match the Rust in the bundle byte for byte?

The copied source comes under the license and notice of the same release. Read LICENSE Read NOTICE

In one line

What we found out today

A compiler is a program someone wrote in some language, too.

The default Topaz compiler that checked Liney is written in Topaz.

How did Seedling know to open src/main.tpz first?

Show the answer

The settings file, topaz.toml, names src/main.tpz as the entry file.

We followed it and found compiler code written in Topaz.

Next time you meet a compiler, you might picture the source on its back.

Field GuideRevisit these wordsCompilerSelf-Hosting Compiler

Looking back

First lap · 01–06

  1. 01

    With Snip, we split 1 + 2 into tokens.

    Read Episode 1 again
  2. 02

    With Twig, we found where the two numbers fit in the addition tree.

    Read Episode 2 again
  3. 03

    With Tami, we read why the type check stopped at 1 + "apple".

    Read Episode 3 again
  4. 04

    On Momo's cards, we found the two numbers in 1 + 2 again.

    Read Episode 4 again
  5. 05

    With Tinker, we read the run record for a program with lines that print. The screen window showed start, then 3.

    Read Episode 5 again
  6. 06

    With Seedling, we turned the map over and opened the source of the compiler that checked our code.

On this lap we explored 1 + 2 and a few other examples in Topaz 5.21.1.

The next lap starts at Snip's stop with a new question.

Next story

Episode 7Where the Spaces and Line Breaks Go

SnipI'm not done yet. Let's talk about spaces.

Back at the first stop on the path, Snip picks up the scissors again.

When do spaces and line breaks mean something?

Next time, let's head back to the first stop.