Picture a desk. Everything the bot is working on right now has to fit on it: your instructions, the conversation so far, the files it opened, the results of its searches. That desk is the context window.
How big is the desk?
It's measured in tokens. A token is roughly a word or part of a word.
Desks have gotten large. xAI's docs list a 500,000-token context window for Grok 4.7, and 1 million tokens for some other Grok models (docs.x.ai pricing). That's a lot of pages.
But big isn't endless. And a big desk can still get cluttered.
What happens when it fills up
Long-running work fills the desk. Every message, tool result and file adds to it. When it gets full, something has to go.
Systems handle this by summarizing older material and dropping the detail. That's compaction. xAI's docs mention context compaction for long agent loops (docs.x.ai). It's useful. It's also a filter, and filters can drop the wrong thing.
So when someone says "the bot forgot," it usually means: that instruction slid off the desk.
The desk vs. the filing cabinet
| Context window (desk) | Saved files and memory (filing cabinet) | |
|---|---|---|
| What's in it | What the bot is working on now | What it should keep |
| Lasts | Until the chat ends or gets compacted | Until someone changes it |
| Good for | Today's task | Rules, decisions, records |
| ISO parallel | What's in front of the worker | Controlled documents and records |
A good bot keeps the desk for today and puts anything important in the filing cabinet. Then it re-reads the file before acting, instead of trusting what it half-remembers.
Cost rides on the desk too
Every token on the desk gets read each time the model responds. More on the desk usually means more cost and sometimes slower answers. xAI's pricing also charges a higher rate once a prompt passes 200,000 tokens on many models (docs.x.ai pricing). See Cost and model choice.
A real example
Our AN Creation project bots are told: the file is the record, not the chat. Before they act, they re-read the plan file. That way, even after a long session and a few compactions, the plan they're following is the real one, not a summary of a summary.
What this means for you
- Put standing rules in files, not in chat. Chat slides off the desk.
- Keep the desk clean. Don't paste in ten documents when one will do.
- Start fresh for a new task. A clean desk beats a crowded one.
- Ask the bot to check the file. "Re-read the plan before you start" is a good habit to build in.
The desk is where the thinking happens. The filing cabinet is where the knowledge lives. Manage both.