What Is a Language Model?

Use this module as a technical deep dive into how language models work. It explains the model beneath products such as ChatGPT, Claude, and coding agents.

The resources place language models within the wider field of AI. They explain training, inference, and next-token prediction. They compare smaller and larger models, reasoning and non-reasoning models, cost, and speed. They also show how text becomes tokens and how those tokens become the streamed response you see on screen.

Starting Points

  1. Begin with OpenAI Academy’s What is AI?. It places language models inside the wider field of AI and separates a model from a product such as ChatGPT. This is the broad frame for everything that follows.

  2. Read Mark Riedl’s A Very Gentle Introduction to Large Language Models without the Hype. It explains how language models learn patterns from text, why predicting what comes next produces so many useful abilities, and why the same process can produce convincing mistakes. Connect it to Matt Pocock’s definitions of model, training, inference, and next-token prediction. These short definitions give you clear names for the main parts of that process.

  3. Read Ethan Mollick’s Scaling: The State of Play in AI. It explains why smaller models can be faster and cheaper, and why reasoning models trade more time and cost for a better chance on difficult problems. Then use Artificial Analysis’s interactive model comparison to filter between reasoning and non-reasoning models and compare quality, cost, response time, output speed, context, and disclosed model size.

  4. Make tokens visible with Simon Willison’s Understanding GPT tokenizers, which explains why text is split differently across words, code, and languages. Then read the definition of a token, type a sentence into the OpenAI Tokenizer, and open TokenSpeed to connect those pieces of text to the response streaming onto your screen.

Deep Dive