The Life of Code 01

Liney'sFirst Dayat Work

Where does 1 + 2 go?

Someone just wrote one line of code.

But what will happen to it inside the computer?

Let's tag along on Liney's first day at work.

Today's odd questionHow many pieces will 1 + 2 become?

Snip is about to cut this code into little pieces and pass them to the next friend.

So how many pieces will there be?

So that's your guess.

Let's see how Snip really cuts it.

LineyHuh? I was just born.And I'm off to work already?

SnipHold on. You can't just walk in.

LineyWhy not? I'm just one little line.

SnipOne line or not, I still need to see what you have inside.

SnipA number, a space, a plus, a space, a number.

LineyAre you taking the spaces along too?

SnipWe'll see. Watch closely which pieces are left.

LineyHuh? I'm five characters long, but only three pieces?

SnipRight. The number of characters and the number of pieces aren't always the same.

LineyTurns out I'm more complicated than I thought.

TwigGot the pieces, thanks.But how am I supposed to put these together?

LineyThere's still more to do?

SnipMy job ends here. Now it's Twig's turn.

Liney's first day at work isn't over yet.

A little experiment

Snip's Scissors

If the spaces change, does it still split the same way?

This time, you pick the expression.

Let's see which pieces Snip cuts out.

Expressions

Split exactly as Topaz compiler 5.21.1 did

Characters

  1. 1
  3. +
  5. 2

Pieces

  1. 1number
  2. +plus sign
  3. 2number

5 characters, 3 pieces

Split exactly as Topaz compiler 5.21.1 did

Characters

  1. 1
  2. +
  3. 2

Pieces

  1. 1number
  2. +plus sign
  3. 2number

3 characters, 3 pieces

Split exactly as Topaz compiler 5.21.1 did

Characters

  1. 1
  5. +
  9. 2

Pieces

  1. 1number
  2. +plus sign
  3. 2number

9 characters, 3 pieces

Split exactly as Topaz compiler 5.21.1 did

Characters

  1. 1
  2. 2
  4. +
  6. 3

Pieces

  1. 12number
  2. +plus sign
  3. 3number

6 characters, 3 pieces

Split exactly as Topaz compiler 5.21.1 did

Characters

  1. 1
  3. +
  5. 2
  7. +
  9. 3

Pieces

  1. 1number
  2. +plus sign
  3. 2number
  4. +plus sign
  5. 3number

9 characters, 5 pieces

First discovery

Compare 1 + 2 with 1+2.

Check

Split exactly as Topaz compiler 5.21.1 did

1 + 25 characters, 3 pieces

  1. 1
  3. +
  5. 2
  1. 1number
  2. +plus sign
  3. 2number

1+23 characters, 3 pieces

  1. 1
  2. +
  3. 2
  1. 1number
  2. +plus sign
  3. 2number

1 + 29 characters, 3 pieces

  1. 1
  5. +
  9. 2
  1. 1number
  2. +plus sign
  3. 2number

The number of spaces is different, but the number and plus sign pieces that go on to the next step are the same.

Second discovery

Now pick 12 + 3.

Check

See how 12 didn't split into 1 and 2?

Two digits can make a single piece.

The real inside story

What those pieces are called

Pieces like the 1, + and 2 that Snip cut out are called tokens.

When a compiler scans the characters of code and picks out pieces like these, it's doing lexical analysis.

A token can carry facts like what kind it is, which characters it came from and where it sat in the source.

Here is the real record

Split exactly as Topaz compiler 5.21.1 did

PieceKindPlace in the source
1integernumber
+plusplus sign
2integernumber

The compiler also notes which cells each piece sat in. That way, if something goes wrong, it can point to the exact spot.

Are spaces really useless?

Not quite.

In the sum you just saw, adding or removing spaces still gave the same number and plus sign pieces.

But in some code, a single line break can change the meaning.

You'll run into that story again on Liney's journey.

And nobody has even worked out 1 + 2 yet.

Working out how to put the pieces together is the next job, and it's Twig's.

In one line

What we found out today

A compiler scans the characters of code and makes pieces called tokens.

So how many pieces is this?

10 + 20
Show the answer

Three pieces.

10+20

10 and 20 have two characters each, but each one is just one number token.

Now you know a little about Snip's job, right?

Field GuideRevisit these wordsCompilerToken

Next story

Episode 2Twig's Assembly Time

Snip cut the pieces just right.

But then Twig found some strange code.

1 + * 2

TwigWait. How am I supposed to put this together?

Two symbols sit side by side in this one line. Can Twig put it together?