Week 1 — From ChatGPT to Transformer
Sign in
The Visible LLM · A 6-week course · Lesson 1 of 6
Week 1 — From ChatGPT to Transformer
What's inside the machine that talks?
Today: 3 hours · 35 slides · Your first API calls

Adventure 8

Where We're Going — The Whole Course in One Slide
Week 1 · TODAY
What is an LLM? Big picture — next-word prediction, training vs inference, your first API call
Week 2
Inside the model — tokens, embeddings, attention. How words become math.
Week 3
Generation & prompt engineering — making the model do exactly what you want
Week 4
Agents — when LLMs can use tools. Claude Code as a live case study.
Week 5
Agent frameworks & MCP — how agents talk to the real world
Week 6
Capstone — you build something real

Adventure 8

Today's Destination — 3 Hours From Now
Here's exactly what you'll be able to do by the time we're done. Specific, testable things — not vague goals.
✓ EXPLAIN
What an LLM is — in your own words, not Wikipedia-speak
✓ DISTINGUISH
Training vs inference — and why this distinction explains the entire AI industry
✓ RECOGNIZE
The three stages every LLM uses: Understand → Think → Generate
✓ READ
A 5-line API call and explain what every line does (read it, not write it)
✓ MADE + WROTE
Your first 10+ API calls to Claude · 5 project ideas to bring back next week

Adventure 8

The Deal — 3 Ground Rules
Before we start, here's the operating system for the next 6 weeks.
Rule 1 — You don't write code. You read code.
Like reading Shakespeare without writing plays. Every code block gets annotated line-by-line.
Rule 2 — Every concept gets a real-world metaphor.
Tokens → Lego blocks. Embeddings → coordinates on a map. Attention → spotlights. If something feels too abstract, ask for the metaphor.
Rule 3 — The goal isn't a grade. It's your project.
Starting next week you narrow ideas. Week 6 you start building. This is a real skill, not a school assignment.

Adventure 8

Before We Start — Let's Calibrate
These aren't rhetorical questions. We stop and talk.
Usage Check
How many times this week did you use ChatGPT, Claude, or any AI? Daily? Weekly? Never?
Best Moment
What's the BEST thing AI did for you this week? Tell me the actual thing.
Worst Moment
What's one time AI got something WRONG — or made you uncomfortable?
The Big Wonder
Have you ever wondered what's actually happening when AI generates text? Is it thinking? Looking things up? Guessing?

Adventure 8

An LLM Is a Next-Word-Prediction Machine
That's it. That's the whole thing.
  • Everything ChatGPT and Claude do — essays, code, love letters, quantum physics, the bar exam — is one operation, run thousands of times: PREDICT THE NEXT WORD
  • Once you know this, the AI industry stops being magical and starts making sense
  • The mystery doesn't disappear — it gets more interesting. Because the real question becomes: how can something so simple produce something so impressive?

Adventure 8

Watch the Model Think — In Slow Motion
Let's actually see what happens. Every word it generates goes through this.
  • Input prompt: "The weather today is really ___"
  • The model considers EVERY word in its vocabulary (~50,000–200,000 options) and assigns each a probability
  • Then it picks one. Usually the most likely. Adds it to the sentence. Starts over.
Word
Probability
nice
35%
bad
20%
hot
15%
cold
8%
... 199,996 more
tiny
Repeat until a special "stop" token is predicted.

Adventure 8

The Biggest Mystery in AI Right Now
How can such a simple mechanism produce essays that pass college courses, code that runs, math proofs that check out?
Explanation 1 — The model has read everything.
GPT-4 trained on roughly all public-internet text — Wikipedia, books, papers, GitHub, Reddit. Trillions of words. Every pattern humans ever wrote down.
Explanation 2 — "Predicting the next word" is harder than it sounds.
To finish "The CEO who just stepped down was named ___" you need to know what a CEO is, what "stepped down" means, who recently resigned, and what plausible names look like. So next-word prediction is compressed reasoning.

Punchline: Simple mechanism + enormous scale + hard task = emergent intelligence. This is the core insight of the LLM revolution.

Adventure 8

Three Common Misconceptions
Most ChatGPT-daily users believe at least one of these. Let's bust them now.
❌ "ChatGPT is searching the internet."
Wrong. No live access during a normal conversation. Only training data. That's why it doesn't know this week's news — pattern recall, not search.
❌ "ChatGPT is a database."
Wrong. No lookup table inside. Knowledge is stored as billions of numbers encoding statistical patterns — that's why it confidently states wrong things sometimes (predicting, not retrieving).
⚠️ "ChatGPT thinks like a human."
Wrong-ish. It generates text that looks like human thinking, but the internal mechanism is math on number-vectors. Whether that counts as "thinking" is a real open question serious researchers argue about.

Adventure 8

Training Data — The Source of All Knowledge
Where does the model's knowledge come from? The text it was trained on. If it isn't in this data, the model doesn't know it. Full stop.
  • Common Crawl (public web) · ~60%
  • Code (GitHub) · ~10%
  • Books · ~10%
  • Wikipedia, scientific papers, news · ~10%
  • Curated/filtered data · ~10%

Adventure 8

How Big Is "Big"?
Model size = parameters — the individual numbers inside the model. Each parameter is one tiny dial. Billions of dials, all set during training.
Model
Year
Parameters
GPT-1
2018
117M
GPT-2
2019
1.5B
GPT-3
2020
175B
GPT-4
2023
~1.7T (estimated)
Claude 3 Opus
2024
undisclosed, similar scale
More dials → finer-grained patterns the model can learn.

Adventure 8

Who Makes the Models You'll Use?
Two camps: closed-source commercial (best quality, costs money) and open-source (downloadable, run locally).
🔒 Closed-Source (Commercial)
  • OpenAI (GPT-4, GPT-5)
  • Anthropic → Claude (our main tool)
  • Google (Gemini)
  • xAI (Grok)
🔓 Open-Source
  • Meta (Llama 3.x)
  • Mistral (Mixtral, Codestral)
  • DeepSeek (V3, R1)
  • Alibaba (Qwen)
  • Z.ai (GLM)

Why Claude for this course: it's what we'll use across all 6 weeks. Every concept applies to all the others equally.

Adventure 8

Quick Gut Check — Mid-Class Checkpoint
In your own words — what is ChatGPT actually doing when it answers a question? Don't be perfect. Be in your own voice. Try to use the phrase "predicting the next word" naturally.
✓ You can say it clearly
Great — you've got the core concept.
✓ You almost can say it
Totally normal. Say what you can.
✓ You can't yet
Also fine. We'll come back after the break.

⏸ BREAK 1 — 10 minutes. Get water. Walk around. Don't check your phone — let what we covered sink in. Brains consolidate during rest, not more input.

Adventure 8

How Prediction Actually Happens — Three Stages
Every word the model produces goes through all three of these. For a 100-word response, this happens 100 times. This is the architecture of every LLM in 2025.
1
1 · UNDERSTAND
Convert your words into something math can work on. Tokenization + embedding. (Deep dive: Week 2)
2
2 · THINK
Process those numbers, find what's relevant, build up meaning. The Attention mechanism inside the Transformer. (Deep dive: Week 2)
3
3 · GENERATE
Output the next word as a probability distribution. Pick one. Add it. Repeat. (Deep dive: Week 3)

Adventure 8

Stage 1 — Understand: How Does "Apple" Get Inside the Machine?
A computer only does math on numbers. But "apple" is letters. So how does language get in? Two steps.
Step A — Tokenization
Break text into chunks called tokens (roughly word-fragments). "apple" → 1 token. "unbelievable" → 3 tokens (un + believ + able). "你好" → 2 tokens. Each token gets an ID number from the model's vocabulary.
Step B — Embedding
Convert each token ID into a list of hundreds/thousands of numbers — an embedding. This list is the token's "meaning" as a coordinate in a giant abstract space. Like GPS coordinates, but for meaning.
"King" and "Queen" end up close. "Apple" and "Banana" close. "Apple" and "Tuesday" far apart. Deep dive next week — today just know: Stage 1 turns words into numbers.

Adventure 8

Stage 2 — Think: The Model's "Thinking" Is Called Attention
Once your sentence is a list of number-vectors, the model has to figure out what it means.
The spotlight metaphor
When YOU read "Mike gave the book to his sister because SHE wanted to read it," your eyes flick back to "sister" when you hit "she." You do this automatically.
Every word × every word — simultaneously
The model does the same — but for every word, looking at every other word, simultaneously, in parallel. Every word asks every other word: "how relevant are you to me?"
Why "Transformer"
This attention mechanism, stacked 96+ layers deep, is the architecture. Named after the 2017 paper "Attention Is All You Need." Before Transformers, models only saw nearby words. Transformers see everything. Week 2 we go deep.

Adventure 8

Stage 3 — Generate: How the Model Picks the Next Word
After "thinking," the model produces a probability distribution over its entire vocabulary. But it doesn't always pick the most likely word — there are settings that control this.
Temperature
How "creative" vs "boring." T=0 → always the most likely word (predictable). T=1.5+ → wild, sometimes broken. Default ≈ 0.7.
Top-K
Only consider the top K most likely words. K=50 → ignore the other 49,950.
Top-P (Nucleus)
Only consider words within the top P% of probability mass. Adaptive — sometimes 3 tokens, sometimes 200.
Coming in Week 3: prompt engineering — crafting input to get the output you want. Highest-ROI skill in this whole course.

Adventure 8

Why Has Every Other Approach Failed?
Transformers didn't appear out of nowhere. 70 years of failed attempts made them possible.
1
1950s–90s — Rule-Based Systems
"If X, then Y." Worked for chess. Broke for language (too complex for rules).
2
1990s–2000s — Statistical Machine Translation
Probability over word sequences. Couldn't handle long-range dependencies. "He went to the bank" — which bank?
3
2010–2017 — RNNs
First real "language understanding." OK on short sentences. Memory issues over long passages.
4
2017 — Transformers ("Attention Is All You Need")
Parallel + global attention. Every word sees every other word at once. Changed everything.
5
2020+ — Scaling Laws
Same Transformer, MUCH bigger. GPT-3 showed scaling alone unlocks new capabilities. Today everyone uses Transformers.

Adventure 8

Where Each Stage Gets Covered in the Course
You don't need to fully grasp each stage today — just know they exist and what they're called.
Today
The only day you see all three stages at once
Weeks 2–3
Every subsequent week zooms in on one piece
By Week 3
You've seen every stage in detail
By Week 6
You're using all of them to build something real

Adventure 8

Can You Say It Yet? — Knowledge Check 2
Try to fill this in out loud. Don't read it — say it. Even imperfectly. The act of saying it is how it sticks.
"An LLM is a __________ machine. To produce each word, it goes through three stages: First, it __________ the input by turning it into __________ . Second, it __________ using a mechanism called __________ . Third, it __________ a probability over the next word."

Answer key: next-word-prediction / understands / numbers (embeddings) / thinks / Attention / generates
If you can fill it in (even imperfectly): you've got the foundation. Everything from here just fills in details of each blank.

Adventure 8

"AI Model" Actually Means TWO Things
Most people don't realize this distinction. Once you do, the entire AI industry makes sense — costs, companies, business models, all of it.
THING 1 — The Model Itself
A giant file of numbers (the weights). The brain. A static artifact, like a written book. Sits on a server, doesn't change until next training run. Producing this is called TRAINING.
THING 2 — Using the Model
Sending a question, getting an answer. Running the model on new input. The model doesn't change — it just processes your specific prompt. This is called INFERENCE.

The cost gap is INSANE — roughly a billion-to-one ratio. Training: hundreds of millions of dollars, months. Inference: fractions of a cent, seconds.

Adventure 8

Training — Building the Brain
What actually happens during training? The full process, with real numbers.
01
Initialize Model
Start with all parameters set to random numbers. The model knows nothing.
02
Predict Next Word
Show it a sentence with the last word hidden. Make it guess.
03
Adjust Parameters
Nudge each parameter slightly toward the right answer.
04
Scale & Repeat
Do this trillions of times on trillions of words.

Real numbers: GPT-4-scale = ~$100M+ run, 3–6 months, 10,000+ GPUs running 24/7. Output: a ~1 TB file of weights.

Adventure 8

Inference — Asking the Brain a Question
What happens when you send a message to Claude right now? Notice how different it is from training.
01
Tokenization
Your text gets tokenized (Stage 1) — broken into tokens, converted to number-vectors
02
Load Weights
Model loads weights into GPU memory — already-trained, waiting on Anthropic's servers
03
Stages 2 & 3 Run
Attention + generation, repeated for every word — one token at a time
04
Tokens Stream Back
Words appear one by one — that's the model generating in real time

Real numbers: 1–8 GPUs, a few seconds, fractions of a cent per call.

Adventure 8

The Two Layers of the AI Industry
Once you understand training vs inference, the whole industry snaps into focus. Two layers. Completely different businesses.
Layer 1 — Foundation Model Labs
The 5–10 companies that train models.
  • OpenAI, Anthropic, Google
  • Meta, Mistral, DeepSeek
  • Need hundreds of millions of dollars
  • Supercomputer access + world-class researchers
Layer 2 — Everyone Else
Startups, enterprises, your dad's businesses, you.
  • Use models via API
  • Need an API key and a good idea
  • No supercomputer required
  • This is where everything you build lives

Everything you'll build in this course lives in Layer 2. You don't need hundreds of millions of dollars. You just need an API key. That's the whole point.

Adventure 8

What Does Inference Actually Cost?
Real Anthropic API pricing. This is what you actually pay.
$3
Input per 1M tokens
Claude Sonnet 4.5 input pricing
$15
Output per 1M tokens
Claude Sonnet 4.5 output pricing
$0.02
Typical Q&A call
1,000-word question + 1,000-word answer ≈ 2,500 tokens
  • 1 million tokens ≈ 750,000 words
  • In practice: most experiments today will cost you less than a penny
  • A serious side-project might cost $5–50/month

Adventure 8

The Training / Inference Cheat Sheet
If you remember nothing else from this section, remember this table.

TRAINING
INFERENCE
WHO
Foundation model labs (5–10 companies)
Everyone — you, startups, enterprises
WHEN
Once, takes 3–6 months
Every single API call
COST
$100M–$1B per run
$0.0001–$0.10 per call
HARDWARE
10,000+ GPUs for months
1–8 GPUs for seconds
DATA
Trillions of words
Just your one prompt
OUTPUT
Model weights file (~1 TB)
The response to one query

Adventure 8

We're Done With Theory — Let's Actually Do It
For the rest of class you'll be doing what every AI engineer does daily: making API calls. Transition from understanding to doing.
01
Open Anthropic Console
Go to console.anthropic.com — the control panel for everything Claude
02
Get an API Key
I'll give you one. This is your credential, your phone line to Claude.
03
Read a 5-Line Piece of Code
Every line annotated. Read it, not write it. Like reading a recipe.
04
Run It Once
Your first real API call. Anthropic's GPUs will run for you. Fractions of a cent spent.
05
Modify It
Change temperature, prompt, model. See how each change affects the output.

Reminder: You don't write code. You read it. Like reading a recipe.

Adventure 8

Your First API Call — In Python
The absolute minimum code to talk to Claude. Every Claude-powered product in the world is built on this foundation. Read it like English, left to right.
# 1: Import the Anthropic library

from anthropic import Anthropic



# 2: Create a "client" — your phone line to Claude

client = Anthropic()



# 3-7: Send a message and get a response

response = client.messages.create(

    model="claude-sonnet-4-5",       # Which Claude model

    max_tokens=1024,                 # Cap on response length

    messages=[

        {"role": "user", "content": "Hello! What's the capital of France?"}

    ]

)



# 8: Print the answer

print(response.content[0].text)
This is it. Every AI startup, every Claude integration, every agent starts here. Complexity comes from what you put in messages. The structure never changes.

Adventure 8

Read the Code Like English
Walk through every line. You should be able to explain each one to someone else.
from anthropic import Anthropic
"From this library, get this tool." Like opening a phone app before a call.
client = Anthropic()
Creates a connection. () means "make a new one." Auto-reads your API key from the environment.
model="claude-sonnet-4-5"
Which version. Sonnet = fast+smart (default). Haiku = faster+simpler (cheap tasks). Opus = smartest+slowest (hard tasks).
max_tokens=1024
"Don't write more than ~750 words." Safety cap on response length. Tokens cost money.
messages=[{"role": "user", "content": "..."}]
The conversation. role: user = us speaking. role: assistant = Claude's response. This list can grow into a multi-turn chat.
print(response.content[0].text)
Get the answer back. Wrapped in an object — we dig in to get the text.

Adventure 8

The Three Dials You'll Touch Most Often
There are many parameters, but these three handle 90% of all real use cases. Learn these and you can build almost anything.
1 · MODEL
Which Claude to use. Match to task complexity.
  • Haiku 4.5 — cheap / fast / simple tasks
  • Sonnet 4.5 — default, start here
  • Opus 4.7 — smartest / most expensive / hard problems
2 · MAX_TOKENS
Response length cap. Tokens cost money.
  • 100–500: short answer
  • 1,000–2,000: normal conversation
  • 4,000–8,000: long essay / code
3 · TEMPERATURE
Creative vs predictable.
  • 0.0 — deterministic (code / math / facts)
  • 0.7 — default, natural-feeling
  • 1.0+ — creative, brainstorming
  • 1.5+ — wild, sometimes broken

Adventure 8

How Conversations Are Structured — Roles
Every conversation is a list of "messages." Each message has a ROLE. Understanding roles is the key to building any real AI product.
USER
That's you. What you're asking or saying.
ASSISTANT
That's Claude. The responses it gives.
SYSTEM
Introduced fully in Week 3 — persistent rules of the game the model gets before the conversation starts.

A real conversation = a growing list of alternating USER and ASSISTANT messages. Each call you send the whole history — the model has no memory between calls.

Adventure 8

Your Turn — Let's Run It Live
Step-by-step. We're going to the Anthropic Console Workbench — no Python needed, just a UI wrapper around the same API.
01
Enter a prompt
Type your message in the user box
02
Set your parameters
Adjust temperature & model in the right panel
03
Hit Run
Send it — Anthropic's GPUs do the work
04
Read the response
What did it say? Does it match your expectation?
05
Change ONE thing
Run again. Compare the outputs side by side.

Adventure 8

Become a Scientist for 10 Minutes
Your first real scientific experiment with AI. Same input, vary one parameter, observe outputs. This is exactly how AI researchers test models.
01
Pick ONE interesting prompt
Examples: "Write a one-paragraph horror story about a refrigerator." / "Suggest 5 names for a 3D printing service." / "What's the meaning of life?"
02
Run at temperature = 0.0 × 5
How similar are the 5 outputs?
03
Run at temperature = 0.7 × 5
How varied are they now?
04
Run at temperature = 1.5 × 5
When does it break?

Observe: At T=0.0, how similar are the 5 outputs? At T=0.7, how varied? At T=1.5, when does it break?

Adventure 8

The 3-Hour Summary
Take a moment. Look at what you covered today. Most adults who use ChatGPT every day cannot say any of this. After 3 hours, you can.
✅ Roadmap
6-week course → long-term project. You know where we're going.
✅ What an LLM is
A next-word-prediction machine. Simple mechanism, enormous scale, surprising capability.
✅ Three Stages
Understand → Think → Generate. Every word, every response, every model.
✅ Training vs Inference
Build the brain ($1B, once) vs use it ($0.02, every call). The whole industry rests on this distinction.
✅ First API call
You read it, ran it, and modified it. You're now technically an AI developer.

Adventure 8

This Week's Mission + Week 2 Preview
📋 Homework — Bring me 5 PROJECT IDEAS
1 · REAL
Solves a problem you, friends, or family actually have. Not hypothetical.
2 · SPECIFIC
Not "AI for school." Instead: "an AI that helps me organize my Spanish vocab the night before a test."
3 · BORING IS OK
Best products solve unglamorous real problems. Don't try to be impressive — try to be useful.
Format: 2-3 sentences each, in a Notion or Google doc. Bring it to Week 2.
Next week — Inside the Machine: We crack open the box. Tokens become number-vectors. Attention becomes spotlights. You'll see exactly how "predicting the next word" turns into intelligence.

Adventure 8