I Stopped Chasing Bigger Context Windows Once I Gave My AI Agent a Shell

Many people believe the key to smarter AI coding is a bigger context window. The idea is that if you can shove your entire codebase into the prompt, the model can see everything at once and produce better results. So we obsess over token counts: 200k, 1M, soon 10M. But I've started to think the context window is a red herring. The real step change in agent capability comes from something much more mundane: the ability to poke around a real Linux filesystem.

The Context Window Trap

Throwing your whole repo into a single prompt is expensive and slow. The transformer's attention mechanism doesn't scale linearly; costs increase quadratically, and the model's ability to attend to every detail degrades. You can end up with the model mixing up files, hallucinating imports, or forgetting the thing you asked it to change. Even if you could fit everything, the model's attention is a finite resource: a 1M token window doesn't mean it pays equal attention to all 1M tokens. It's also a snapshot that becomes stale the moment you hit send. The marketing heavyweights might sell you on 1M tokens, but the engineering reality is that size alone rarely solves the problem.

How a Human Developer Actually Works

When I work on a large codebase, I don't load the entire thing into my brain. I use the filesystem. I run ls to see the directory structure, grep to find a function definition, and find to locate configuration files. I open the three or four files that matter, make my changes, and run the tests. The shell is my primary interface. An AI agent should be no different. It should be able to seek out information on demand, not rely on a static dump of everything.

The Filesystem as the Agent's Workspace

A persistent shell gives the agent a real working environment. It can write files, store intermediate results, and run commands. The context window becomes a short-term working memory, not a database. The agent fetches exactly what it needs at the moment it needs it. This is far more efficient and mirrors how code is actually traversed.

That's the philosophy behind borg, the AI coding agent we built on xShellz. It lives inside a persistent Linux shell. It doesn't try to swallow your whole repo. Instead, it uses grep, ls, cat, and other standard Unix tools to understand your project incrementally. It can also write and run tests, check git diff, and keep going until the task is done.

A Concrete Example

Suppose you ask an agent: "Add a new endpoint POST /v2/users that creates a user with a role field." A context-window-first agent would need to have the entire codebase in its prompt, including the router, models, serializers, and middleware, all at once. That's a lot of tokens and a high chance of hallucination.

A shell-based agent would do something like this:

grep -r "def create_user" .
grep -r "/v1/users" src/routes/

It finds the relevant files, reads the existing user creation endpoint, sees the model and serializer, and then writes the new code. It might run cat on a few files, apply a patch, and then run the test suite. It uses the filesystem as its external memory, pulling in only what's necessary.

This approach uses fewer tokens, produces less latency, and often yields more accurate results because the agent isn't overwhelmed by irrelevant context.

Why a Remote Shell Makes All the Difference

If your AI agent runs locally, it's confined to your machine's filesystem, which might be offline or inconsistent. A remote shell on a server that's always on changes the game. The agent can work in a persistent environment, with your project files, and you can SSH in at any time to check on progress or take over.

That's exactly what xShellz provides: a remote Linux shell that stays alive 24/7. borg lives there, so you can give it a task, disconnect, and come back later to see the results. It's not just a chat interface; it's a real terminal session with a full filesystem. The agent's power comes from that filesystem interaction, not just from a massive context window.

Trade-offs and Reality

Context window size still matters. If you need to summarize a long document or handle a conversation with hundreds of turns, you need a model that can hold that state. But for coding, the ability to interact with the filesystem is a force multiplier. The best agents will combine a reasonable context window with a shell. Don't be fooled by the marketing numbers.

Next time you evaluate an AI coding tool, don't just ask "How many tokens can it handle?" Ask "Does it have a real shell? Can it ls, grep, and write files?" That's the real power move.