AI Context Windows: What the Number Actually Means for Your Work

What a Token Actually Is (And Why It Isn't a Word)

Every discussion of context windows requires understanding what a token is, because the number attached to any context window claim is a token count, not a word count. A token is the unit that language models process — roughly speaking, a short fragment of text that might be a whole word, a word fragment, a punctuation mark, or a number depending on how the model's tokeniser divides its vocabulary.

As a rough approximation, one thousand tokens corresponds to somewhere between seven hundred and eight hundred words of English prose. That ratio shifts for other languages, code, and highly technical text — dense Python code tokenises differently from narrative writing, and languages with longer words tend to produce different token-to-word ratios.

The context window is the total number of tokens a model can hold in its working attention at once: everything in your prompt, your conversation history, any documents you have uploaded, and the response the model is building. When a model is advertised as having a one-million token context window, it means the system can theoretically process roughly 700,000 words in a single operation. Whether that capability translates into useful performance is a separate question.

The Three Use Cases Where Context Window Size Is Decisive

There are specific task types where the size of the context window determines whether a task is possible at all, not just whether it is performed well.

The first is whole-document analysis. Legal contracts, academic papers, financial reports, and technical specifications often run to tens of thousands of words. Asking a model to summarise, cross-reference, or extract specific clauses from a document that exceeds its context window forces one of two unpleasant workarounds: chunk the document into pieces that lose cross-section coherence, or truncate it and miss information. For practitioners doing document review work, context window size is the primary capability threshold.

The second is large codebase reasoning. A model editing a function in a 20,000-line application needs to understand the call graph, the data structures, and the patterns established elsewhere to write code that fits. Smaller context windows produce superficially functional code that conflicts with the broader architecture in ways that only surface at integration time.

The third is long conversation history. Applications that maintain extended multi-session dialogues — customer service, coaching tools, project management assistants — need to hold previous exchanges in context. Without sufficient window, the model forgets earlier constraints, contradicts previous outputs, or asks the user to re-explain things it was already told.

When a Huge Context Window Is Just Marketing

For a very large class of everyday tasks, context window size above 8K to 16K tokens makes no meaningful difference. Single-document summarisation, question answering, email drafting, code generation for isolated functions, and most conversational use cases fit comfortably inside these limits. Paying a premium for a million-token window when your typical interaction involves a 500-word prompt and a 300-word response is paying for a capability you will never use.

The interactive context window planner tool lets you calculate actual token counts for your typical inputs, which is the most direct way to establish whether a larger context window is genuinely relevant to your workflow or whether you are being upsold on a number that sounds impressive but changes nothing about your results.

The marketing dimension is real. Large context windows are easy to announce, easy to remember, and difficult for most users to evaluate critically. They have become a proxy for "capability" in a way that often does not reflect actual task performance.

Lost-in-the-Middle: Why Long Context Degrades Attention

Even setting aside the marketing question, there is a genuine technical limitation that vendors rarely emphasise: models tend to perform worse on information that falls in the middle of a very long context than on information at the beginning or end. This phenomenon, documented in published research, has been called the "lost-in-the-middle" problem.

When a model is processing a 500,000-token input, the information at positions 200,000 through 300,000 receives less reliable attention than the bookends. For tasks that require synthesising information from throughout a large document, this degradation is consequential. The ability to hold one million tokens in context and the ability to reliably reason across all of them are not the same capability.

This means the practical effective context for many tasks is meaningfully smaller than the advertised limit. Teams that need genuine whole-document reasoning should test performance at the scale they actually need rather than assuming that a large advertised window translates directly into proportionally better results.

Comparing Real-World Limits Across Top Models

Context window sizes vary substantially across the models available in 2026. The most capable frontier models offer windows in the hundreds of thousands to one million tokens, while smaller and more cost-efficient models typically offer 8K to 128K.

The practical comparison is not just about the headline number but about the combination of context window size, cost per token, and the reliability of attention across that window. A 128K-token model that handles long-context tasks with consistent quality may outperform a 1M-token model that shows significant degradation at 200K. The deep-dive article on context window mechanics benchmarks several leading models against long-context tasks at scale, giving a more accurate picture than specifications alone.

Practical Tips for Managing Context on Long Projects

For most teams, the smarter response to context window constraints is active context management rather than simply defaulting to the largest available window.

Retrieval-augmented generation (RAG) addresses the problem architecturally: rather than loading an entire document corpus into the context window, a RAG system retrieves only the most relevant passages and loads those. The model sees a focused, relevant slice rather than a sprawling whole, which often produces better answers than brute-force loading would.

Summarisation chaining works well for sequential material: the model processes a long document in segments, summarising each segment, then reasoning over the summaries. This keeps active context small while preserving access to information from the full document.

For code projects, careful context engineering — passing only the files relevant to the current task, using explicit dependency declarations, and maintaining a project summary document — achieves strong performance without requiring the largest available window. The most effective context management is thoughtful selection, not simply paying for a bigger bucket.