What is a large language model, and what does the word large actually refer to?
By Yash Malviya·
A large language model predicts the next token, over and over, using patterns learned from text. The word large refers to two separate numbers that are easy to confuse: the parameters it holds and the tokens it was trained on.
What is a large language model
A large language model is a system that predicts the next token in a sequence, trained on a very large amount of text, with a very large number of learned parameters.
That is the whole mechanism. Everything a model appears to do, answering, summarising, translating, writing code, comes out of repeatedly asking the same question: given everything so far, what comes next?
The core operation is narrow and the behaviour on top of it is broad. Most confusion about these systems lives in that gap.
What a token actually is
A token is not a word.
It is a chunk of text produced by the model's tokenizer, which breaks writing into pieces that occur often enough in training to be worth a slot of their own. A common short word is usually one token. A long or unusual word is several. Whitespace and punctuation are tokens too.
This is why every price list, rate limit and context window is measured in tokens rather than words. It is also why the same paragraph costs different amounts in different models: the tokenizers disagree about where the seams go.
The two things people mean by large
Here is the distinction that most coverage flattens.
Parameters are the learned numbers inside the model, set during training and fixed afterwards. Mistral Large 4 was released with one trillion of them. Parameter count is a specification, and vendors usually publish it.
Training tokens are how much text the model saw during training. This number is usually not published, and it matters at least as much. Two models with identical parameter counts trained on different amounts and qualities of data are not comparable.
“The core operation is narrow and the behaviour on top of it is broad. Most confusion about these systems lives in that gap.”
These two numbers do not move together, and a model is not better because one of them is bigger. A larger model trained on less or worse data can lose to a smaller one. Treating parameter count as a score is the single most common mistake in reading model announcements.
Training and inference are different things
Training is the expensive, one-off process that sets the parameters. The model is shown enormous quantities of text and the parameters are adjusted so that its next-token predictions get better.
Inference is the model running against your prompt. The parameters do not change. Nothing you type is learned. A model that appears to remember your conversation is being handed the conversation again as input, inside its context window, every single turn.
That distinction explains several things that otherwise look like contradictions. It is why a model can know nothing about events after its training cutoff but discuss them fluently when you paste them in. It is why your chat does not make the model smarter for the next person. And it is why a mismatch in how that context is handled can leak one user's text into another's reply.
Why transformers
Nearly every modern large language model is built on the transformer, introduced in Attention Is All You Need (Vaswani et al., 2017). Its contribution was attention: a mechanism that lets the model weigh how much each part of the input matters to each other part, in parallel rather than strictly in sequence.
The parallelism is the commercially important part. Earlier recurrent architectures processed text in order, which capped how much hardware you could usefully throw at training. Transformers removed that cap, and the scale-up of the last several years followed.
We have covered how transformers work separately, including why they beat the networks that came before.
What the mechanism does not give you
Predicting the next token well does not entail being right.
A model produces fluent, confident text whether or not the content is accurate, because fluency and accuracy are produced by the same machinery. That is the honest description of what gets called hallucination, and it is a property of the method rather than a bug introduced by a careless vendor.
It also means that benchmark numbers describe performance on a benchmark. They do not describe understanding, and they transfer to your use case only as far as your use case resembles the test.
How to read a model announcement
Three questions get you most of the way.
What is the parameter count, and is it comparable? Mixture-of-experts models publish a total count and a much smaller active count. Those are different numbers and headlines routinely use the flattering one.
What was it trained on, and who says so? If the answer is undisclosed, the parameter count is the only hard figure in the announcement.
Who ran the evaluation? A vendor grading its own model on a benchmark it chose is a marketing document. That is not a reason to dismiss it. It is a reason to wait for an independent run.
Frequently asked questions
What is a large language model in simple terms?
It is a system that predicts the next chunk of text given the text before it, trained on a very large amount of writing. Every capability it appears to have, answering questions, summarising, translating, writing code, is produced by running that same prediction repeatedly. The mechanism is narrow. The range of behaviour it produces is not, and that gap is where most of the confusion about these systems comes from.
What does the large in large language model refer to?
Two different numbers that are easy to confuse. Parameters are the learned weights inside the model, fixed once training ends, and vendors usually publish this figure. Training tokens are how much text the model was trained on, and most vendors do not disclose it. The two do not move together, and neither one on its own tells you whether a model is good. A bigger model trained on less or worse data can lose to a smaller one.
What is a token, and why not just count words?
A token is a chunk of text produced by the model's tokenizer, chosen because it appears often enough in training data to deserve its own slot. Common short words are usually one token. Longer or unusual words are split into several. Punctuation and whitespace count too. Models bill, limit and measure in tokens because tokens are what they actually process, and because different tokenizers split the same sentence differently, the same paragraph can cost different amounts across models.
Does a model learn from my conversation?
Not during the conversation. Training sets the parameters once, and inference, which is the model answering your prompt, does not change them. A model that appears to remember earlier turns is being handed the whole conversation again as input every time, inside its context window. Whether your conversation is later used to train a future version is a separate question about a vendor's data policy, not about how the model works.
What is the difference between a context window and what the model knows?
The context window is how much text the model can read in one go. What it knows is baked into its parameters during training. A model can have a very large context window and still be wrong about a subject it was never trained on, and it can be well trained on a subject and still fail because the relevant text did not fit in the window. Expanding the window does not add knowledge, and a bigger window is not automatically better.
Are all large language models built on transformers?
Nearly all the ones in commercial use are. The architecture comes from Attention Is All You Need, published by Vaswani and colleagues in 2017, and its key idea is attention: weighing how much each part of the input matters to each other part, in parallel rather than in strict sequence. The parallelism is what made large-scale training economically feasible. Alternative architectures exist and are actively researched, but the transformer is the base case.
Why do models make things up if they were trained on real text?
Because fluency and accuracy come out of the same machinery. The model is scoring which token is likely next, not checking whether a statement is true. A plausible-sounding false sentence and a plausible-sounding true one can look equally good to that process. This is why measured hallucination rates vary so much between evaluations: what counts as a hallucination depends on who is measuring and against what reference.
Is a model with more parameters always better?
No, and the assumption causes real mistakes. Parameter count is a specification, like engine displacement. It does not account for training data quality, training duration, fine-tuning or the serving setup. Mixture-of-experts models complicate it further by publishing a large total parameter count alongside a much smaller active count, which is the number actually used for any given token. Headlines tend to quote whichever is more impressive.
What is the difference between training and inference?
Training is the one-off process that sets the model's parameters by showing it enormous quantities of text and adjusting until its next-token predictions improve. It is where almost all the compute cost sits. Inference is the model running against a prompt to produce output, with parameters frozen. The practical consequence is that improving a model means retraining or fine-tuning it, not using it more.
How should I read a new model announcement?
Ask three things. First, whether the quoted parameter count is total or active, because mixture-of-experts models have both and the gap can be large. Second, what the model was trained on and whether anyone outside the company has said so. Third, who ran the benchmark. A vendor grading its own model on a benchmark it selected is a marketing document, which is not a reason to dismiss it but is a reason to wait for an independent evaluation before drawing conclusions.
Sources
What each one is, and whose it is.
- 1
Attention Is All You Need, arXiv (June 12, 2017)
PaperIndependent of the vendor - 2
What are Large Language Models?, NVIDIA
Vendor announcement - 3
Mistral Large 4 open weights are out: a 1 trillion parameter model, benchmarks the vendor graded itself, The Super Intelligence News
Press reportIndependent of the vendor