Large Language Model

What is a large language model?

A large language model (LLM) is an AI system trained to understand and generate natural language. Well-known examples include the GPT models behind ChatGPT, Google Gemini and Anthropic's Claude. The word "large" refers to the enormous amount of training text and the very large number of model parameters — often many billions.

How does an LLM work?

An LLM first breaks text into tokens — small units such as words, word fragments or character sequences. These are turned into numeric vectors (embeddings) and processed by a neural network. Almost all of today's LLMs are built on the transformer architecture, introduced in the 2017 paper "Attention Is All You Need." Its core is the self-attention mechanism, which lets the model weigh how relevant other tokens are to the meaning of each token.

When responding, an LLM generates text one token at a time: for each step it computes probabilities over all possible next tokens and selects from them. This repeats until the answer is complete.

How does an LLM learn?

Training is usually two-stage. In pre-training, the model processes vast, largely unlabelled text and learns statistical patterns of language — grammar, facts, phrasing — through self-supervised learning. This is often followed by fine-tuning on more specific data or with human feedback to make responses more helpful and safer. An LLM also processes only a limited span of text at once — the so-called context window. Very long inputs, or knowledge beyond the training cut-off, fall outside this limit.

An important caveat for studying with LLMs: the model does not "know" things in a human sense; it predicts plausible text. It can therefore produce confident-sounding but incorrect statements (so-called hallucinations). LLMs are powerful tools for summarising, explaining and quizzing, but their output should be checked on important facts. Our guide on studying with ChatGPT shows practical examples.