A context window is working memory, not a hard drive. Feel that, and you manage AI differently.

Most people talk to an AI as if it remembers everything they've ever told it. Like it has a hard drive somewhere with the whole history of your project on it, and every reply is drawn from that permanent store. It isn't, and it doesn't. What the model has is closer to working memory — the RAM in a computer, not the disk — and it's finite. Once you actually feel that in your bones, you start using these tools in a completely different way.

What the context window actually is

The "context window" is the amount of text the model can hold in view while it answers you. Anthropic's own documentation calls it exactly that: a "working memory" for the model, separate from the vast training data it learned from. Everything you've put in front of it this session — your instructions, the files you pasted, the earlier back-and-forth, and even the reply it's writing right now — sits inside that window and counts against it. When the window fills, something has to give.

McKinsey has a plain-language explainer on the same idea if you want the non-technical version to hand to a colleague. The one line to hold onto: the model can only reason over what's in the window. The moment your file scrolls out the top, it's gone from the model's mind, even though it's still there on your screen.

The cheap smoke alarm I use

Here's the trick I actually rely on. I give the model one hard rule at the start of a working session: repeat my name at the top of every single reply. It's trivial to follow while there's room. And the instant it forgets to — the moment a reply comes back without my name on it — that's my tell that the window is getting crowded and the model has started quietly dropping the earliest instructions and files to make space.

It's a smoke alarm, not a diagnosis. But it's free, and it fires before the output visibly goes wrong. Without it, the first sign of trouble is usually a confidently wrong answer — the model filling a gap it no longer remembers with something plausible.

More context isn't better context. As the window fills, the model gets less reliable, not more — and it never tells you it's straining.

Why a full window quietly hurts you

There's a counterintuitive bit here that catches people out. Stuffing more into the window doesn't make the model smarter. Past a point it makes it worse. Anthropic's docs name the phenomenon directly — as the token count grows, "accuracy and recall degrade," an effect they call context rot. Their engineering team has a longer piece on why long contexts degrade and how to engineer around it, and the takeaway is blunt: curating what's in the window matters as much as how big the window is.

This is where twenty years of doing the actual work pays off in a way that has nothing to do with prompting tricks. The leverage isn't a bigger window — it's judgement about what deserves to be in it. If you know a domain, you know which ten percent of the brief actually drives the answer, and you feed the model that. If you don't, you paste everything in, watch the output drift, and blame the tool. Same model, opposite result. I've written before about how progressive disclosure — showing the machine the summary first and the detail only when it's needed — keeps the window clean and the answers sharp.

The fix when it degrades: a handoff, not a fight

When a session gets bloated, the wrong move is to keep wrestling with the same overstuffed chat, re-explaining things it's already forgotten. The right move is to start fresh — but not from zero. I have the model write a short handoff file first: the goal, the decisions made, the open questions, the two or three facts the next session can't work without. Then I open a clean window and drop that in. The information flow survives; the rot doesn't.

That's the same discipline behind teaching my assistant to prep my week — the hard part was making it stop starting over — and behind why the real test of a system is whether the work moves when you've left the desk. A good handoff is just a context window you packed on purpose instead of by accident.

Treat the model like it has working memory, not a hard drive, and most of the frustration people have with AI evaporates. It was never lying to you. It just ran out of room and was too polite to say so.

Sources & further reading

External

Anthropic — Context windows (working-memory framing, "context rot," managing context). Anthropic Engineering — Effective context engineering for AI agents. McKinsey — What is a context window?

Related posts

Beyond the junk drawer: mastering knowledge with progressive disclosure and AI. I taught my assistant to prep my week. The hard part was making it stop starting over. The real test of a system is whether the work moves when you've left the desk.