Tutorials / Disciplined AI-Assisted Development · Lesson 1 of 14

On this page

How Language Models Work

You type a question into an AI chat, press send, and a fluent answer appears. It can feel like magic, and it is hard to control something you cannot picture. This lesson opens the box just enough to see how it works, and then uses that picture to answer practical questions: why do some jobs go well and others badly, what does it cost, which model should you pick, and which technology should you choose?

Our goal: understand, in plain words, how a language model produces an answer, so you can make good choices when you use one. Each idea here changes a decision you will make:

what to expect from it, and where it is weak
which model to pick, and how much reasoning to ask for
how to keep the cost down
which technology to choose

This describes models as they are at the time of writing, including reasoning models. Names and settings change, so the ideas matter more than the labels.

Here is the route:

Next-word prediction: the one trick behind every answer.
Why it can predict: training on a huge amount of text.
Why it is powerful now: the idea called the transformer.
Why it solves harder problems: reasoning models write it down first.
Why it is good at maths and code.
Tokens, briefly.
What a chat costs.
Right-sizing: the cheapest model, and the least reasoning, that is good enough.
Getting accurate answers for software development: choosing technology the model has seen a lot of.

We leave out the maths inside the model and the details of how it is trained. Lesson 02 covers how reliable models are, and lesson 03 covers how to ask.

Next-word prediction

A language model is given the text so far and asked one question: what comes next? Inside, it is a large mathematical model that gives every possible next word a probability. It picks one, adds it to the text, and asks again with the longer text. It repeats until the answer is finished. (We say "word" for now. Strictly, the model works in small pieces called tokens, which a later section explains.)

Watch it work on a short question. The reply below is an illustration, built one word per step:

Two things follow, and both matter later. The model picks among likely words with a little chance, so asking twice can give two different answers. And everything it does, writing code, explaining an error, drafting an email, comes out of this one repeated step.

💡 Is it thinking? People disagree about what to call it, and we do not need to settle that. What matters for your work is the behaviour: it produces a likely continuation, and often that looks like careful reasoning.

Why it can predict

A model predicts well because it has seen a great deal of text. Before you ever use it, it is trained: shown books, articles, documentation, forum answers and code, with the next word hidden each time, and nudged whenever its guess is wrong. After a huge amount of practice, it has absorbed the patterns of how text goes.

Take this sentence: "Apple phones cost more, so I bought a cheap ..." The model has seen this kind of sentence many times. "Android" is the likely next word, "one" is less likely, and "banana" is almost impossible.

This also explains grammar. The training text was mostly well-written, so the model's predictions are mostly well-written too.

It works the other way as well. The model knows only what was in its text, up to a date called its knowledge cutoff. A library released last month, or your team's private code, was never in that text, so it can only guess unless you tell it.

Why it is powerful now: the transformer

People have tried to build text predictors for a long time, and the old ones were weak. The recent jump comes from an idea called the transformer. You do not need the details, only one idea: it learns which words in the text are related to each other, even when they are far apart, and uses that to predict.

Read this: "The boy dropped the glass because he was clumsy." To predict well after "because he", the model has to know that "he" means the boy, not the glass. A transformer pays more attention to "boy" than to the other words. The line thickness in the picture is only an illustration of that idea.

The model stores these relationships as patterns, the same way it stores what usually follows what. That is why it can keep track of who is who in a long text, or which variable a line of code is about. It is also why the words you give it matter so much: they are what it pays attention to.

Why it solves harder problems: reasoning models write it down

Some models solve harder problems by reasoning: they write out their working before they give the answer. These are called reasoning models, and not every model is one. This section shows why writing first helps, starting with how people do it, and ends with which models do it and how much.

Give a person a hard puzzle and ask for the answer at once, and they will probably get it wrong. Let them write things down, and they do better. The notes unfold more clues, each clue suggests the next, and they get closer to the solution. It is why detectives in films fill a board with notes.

People do this everywhere, not only in films. Detectives pin clues to a board. Mathematicians fill pages with working. Engineers sketch on whiteboards, and writers make an outline before the first paragraph. The writing is part of the thinking, not only a record of it.

Here is a made-up case, as it might look on a detective's board: a stolen bicycle in the middle, with people and clues pinned around it. No single note solves the case. But once the notes are on the board, links appear that nobody would see in their head, such as the van and Peter.

A language model works the same way. It has no hidden notepad: the text so far is all it has to work with. If it must answer straight away, it has little to build on. If it first writes out clues from the question, those clues become part of the text so far, and they help it predict the answer. That written working is the reasoning, and tools often show it as thinking. It is still the same prediction loop, with more text in front of it.

Here is a small example: "A shirt costs $20 after a 20% discount. What was the original price?"

Not every model does this. A model trained to write out its working before the answer is called a reasoning model. A model without that training answers straight away, like the person on the left of the first picture, so it is quicker and cheaper but slips more often on a problem like the shirt price. You can nudge a plain model with "think step by step", which helps a little, but a reasoning model does it by itself and does it much better.

Most new models are reasoning models, and many let you choose how much they reason. The names depend on the lab that made the model, but the idea is the same everywhere:

Setting	What the model does
Off, or none	Answers straight away: fastest and cheapest
Low	Writes a little working first
Medium	Writes a fair amount of working
High (some labs add higher levels)	Writes a lot of working first: slowest and dearest, best for hard problems

Some models decide for themselves how much to write, and some have no setting at all. The levels and their names change often, so look at the tool you use. The trade-off does not change. More reasoning means more words, so it is slower and more expensive, but it is a large part of why current models solve problems so much better than older ones. For an easy, quick task, a plain model or a low setting is usually enough.

Why it is good at maths and code

Two reasons, and they are different ideas: the answers are narrow, and they are checkable.

Narrow answers

Compare two questions. Which is easier to predict the next word for?

"What is 1 + 1? The answer is ..."
"My favourite animal is ..."

The first has one answer, so the probability piles up on a single word. The second has no right answer, so it spreads across hundreds of animals. Maths and logic are narrow in this way, and open-ended questions are wide.

Checkable answers

In maths and code, a machine can tell whether an answer is right. That helps at two different moments.

During training. Practice material can be made by machine. This is a gross simplification, but picture a short program that loops and produces a million problems with their answers (1 + 1, 1 + 2, 1 + 3 and so on). The answers are known to be right, so there is endless practice. For code, a compiler (the program that turns code into something runnable) and tests can say whether the code built and passed, and that result can be used as feedback.

When you use it. An AI coding tool can run code in a sandbox, a safe throwaway space where nothing real is at stake. The compiler and tests report a pass or an error. The model reads the error, fixes the code and tries again, in seconds.

Open-ended writing has no such checker, so nothing tells the model it was wrong. That is why models are usually far better at a sum or a function than at questions of taste or about your own business.

It is not a guarantee. A model can still get a sum or a function wrong, especially when it answers straight away, so check the result, and let the tests do the checking where you can. Lesson 04 covers tools that run your tests.

Tokens, briefly

So far we have said "word". What the model really works with are tokens: the pieces its text is cut into. Usually one word is one token. A rarer or longer word may be split into two or more.

To understand how a model works, the difference does not matter. Assume one word is one token. (In English a word is on average a little more than one token, and code usually needs more, so real counts are higher.) It matters for one thing only, which comes next: tokens are what you pay for.

What a chat costs

Think about what happens as a session goes on. Each time you send a message, the model is given the whole chat so far and predicts the reply, word by word. So two things happen, and providers charge for both:

Input tokens: everything sent to the model.
Output tokens: everything the model writes back.

Output usually costs more per token than input. The prices below are hypothetical, only to show the idea. Real prices differ by model and provider, and change.

	Hypothetical price
Input	$5 per 1 million tokens
Output	$20 per 1 million tokens

The catch is that the model has no memory between turns. The whole conversation is sent again every time, so input grows with every turn. Suppose each turn adds about 1,000 tokens to the chat. Ten turns in one long chat send 1,000 + 2,000 + ... + 10,000 = 55,000 input tokens. The same ten turns as five short chats of two turns each send 3,000 per chat, which is 15,000.

That is a rough guide, not a rule. Providers have ways to reduce the cost of re-sending, so real bills are lower. The habit still holds: use one session for one task, and start a fresh one for the next. A fresh session also starts empty, with no leftover clutter in front of the model.

The model can also only read a limited amount at once. That limit is called the context window, and it is measured in tokens. A very long chat can fill it, and then early details are lost.

Cramming several tasks into one session has two costs:

It costs more. Every turn re-sends everything, including old tasks that no longer matter.
It can be less reliable. The model has more text to sort through, so it can miss a detail, mix up two tasks or follow an old instruction that no longer applies. This can happen even while everything still fits in the window, and it varies by model.

If the next task needs something from the last one, do not drag the whole chat along. Paste in just the facts it needs.

Right-sizing: the cheapest model that is good enough

One provider usually offers several models, because ability and cost go together. Here is why.

Inside, a model is a huge set of numbers, called parameters, that hold the patterns it learned. A larger model has more of them, so it can hold more and finer patterns, and it handles hard, subtle tasks better. But every word it predicts means working through a lot of those numbers. More numbers means more computing, which costs more and takes longer. A smaller model has fewer numbers, so it is quick and cheap, but it has less to draw on.

Neither is "better". They are different trade-offs. Within one family of models, bigger usually means more capable, but a new small model can beat an older large one.

At the time of writing, Anthropic offers these, from cheapest to most expensive:

Model	What it is for	Cost
Haiku	Fast, handles simple tasks	Very cheap
Sonnet	A good balance, medium tasks	Moderate
Opus	Powerful, long and hard tasks	Expensive
Fable	The most capable	Very expensive, input and output

Other labs sell the same kind of ladder:

Lab	Small and cheap	Middle	Large and expensive
Anthropic	Haiku	Sonnet	Opus, Fable
Google	Gemini Flash-Lite	Gemini Flash	Gemini Pro
OpenAI	GPT nano	GPT mini	GPT (the flagship)

The table shows tiers only, and leaves out version numbers, which change every few months. Names and details move on, but the shape stays: every lab sells a small, a middle and a large model.

It is the same as cars. A small city car is cheap to buy and run, and perfect for the daily commute. A family van carries more, and costs more. A luxury car is the most capable and the most expensive, and for the daily commute the city car does the job.

The rule is simple: use the cheapest model that is good enough for the task. It costs less and answers faster.

There is no formula for which is which. Tasks differ, and models change. You find out by testing: try the task on the smaller model, see where it fails, and move up only when it does. Many companies go further and mix models from several providers, choosing the best fit per job.

Right-size the reasoning too

Reasoning has a price as well. The working a reasoning model writes is made of tokens, and you pay for them as output, the dearer kind. A model that reasons at length before a one-line answer can cost many times what the same model costs when it answers straight away. It is also slower, because all those words are written before the answer starts.

So the same question applies: how hard is this task? Judge that before you pick the setting.

Simple, one step, the answer is clear: renaming a variable, fixing a typo, formatting a list, writing a short comment. One quick answer with reasoning off, or on low, is enough, and it is the cheapest.
Several steps, or a few things to weigh: finding why a test fails, or changing a function that other code depends on. A medium setting is a fair start.
Hard, with many parts that depend on each other: tracking a bug across several files, or planning a change to how a program is built. This is where a high setting earns its cost, because the extra working is what lets the model get there.

The longer the working, the more output tokens you pay for. For the hard task that is worth it, and for the simple one it is waste.

Start low and move up when the answers are not good enough, the same way you move up to a larger model. A higher setting does not guarantee a right answer, and on a simple task the extra words are only a cost.

Getting accurate answers in software development

Remember how it learned: the more patterns the model has seen on a subject, the better it is at it. On a subject it has seen little, it is shaky. A bigger model helps, but even the biggest cannot use a pattern that was never in its training text, so it stays weaker there than on a popular subject.

So when you choose the technology for a project, choose the one most people use: the widely used language, framework and libraries. The internet has the most code samples and discussion about them, so the model has the most patterns to draw on.

That is not a choice made only because of AI. It was the sound choice long before language models existed:

Two cautions:

Popular is a tiebreaker, not the only reason. The technology must still fit the problem.
Newer versions have fewer patterns. A version released after the model's knowledge cutoff has few or none, so the model may write code in the older style.

When you must use something less common, or your own private code, put the patterns in front of the model yourself:

Later lessons show how to do this properly.

Recap

A language model predicts the next word from the text so far, adds it, and repeats.
Training on a huge amount of text gives it patterns, so it predicts well where text was plentiful and guesses where it was not.
The transformer lets it track which words relate to which, even far apart.
Writing clues first (reasoning, shown as thinking) gives it more text to build the answer on. Only reasoning models do this, and many let you set how much.
It is strong at maths and code because the answers are narrow, and practice data can be made by machine.
Tokens are mostly words, and you pay for input and output tokens.
A long chat re-sends everything each turn, so use one session per task.
Right-size: use the cheapest model that is good enough, and find it by testing. Do the same for reasoning: it uses output tokens, so keep it low for simple tasks and raise it for hard ones.
More patterns seen, better answers: choose widely used technology, as it was always the sound choice and now the model knows it better too.

Questions to think about

The answers are left out on purpose. Try them in your own words first.

What does a language model do, over and over, to produce an answer?
Why does choosing widely used technology get better answers from an AI?
Why does writing out clues first help it solve harder problems?
Why does a long chat cost more with every turn?
Why not always use the most powerful model, with the highest reasoning setting?

Next lesson: what makes AI useful, why "is it accurate?" is the wrong question, and how the guidance you give it decides how reliable it is for your job.