Stop Paying for API Tokens: How to Turn Your Local GPU into an AI Coding Agent with borg / xShellz

Q: Do I need a GPU to run borg with Ollama?

Not strictly. Many small models run on CPU through Ollama, though a GPU drastically speeds up completions. For a snappy terminal coding agent, a modern GPU with at least 6 GB VRAM is recommended for 7B parameter models. Borg does not require a GPU; it just connects to whatever Ollama backend you provide.

Q: Can I use borg with OpenRouter to access many models with one key?

Yes. Set BORG_MODEL_PROVIDER=openrouter, provide your API key, and borg will route requests to whichever model ID you specify. This lets you switch between dozens of hosted models without managing separate accounts, though you still pay per token unless using a free-tier model on OpenRouter.

Q: How do I verify that borg is actually using my local Ollama and not some remote service?

Borg prints the provider and model name at the top of each session. You can also check your Ollama logs for incoming requests. If you see processing activity in the Ollama console while borg is responding, the agent is hitting your local server. No network calls leave your machine for the model inference.

TL;DR

Copy one command to install borg, set two environment variables, and you have a private AI coding agent running on your local Ollama.

Local models mean zero per-token API costs: your GPU does all the work, and no code or prompt ever leaves your machine.

You can switch between Ollama, LM Studio, OpenAI, or OpenRouter in seconds by changing BORG_MODEL_PROVIDER and a base URL.

To use borg with your local Ollama models, install the single static binary, then set the BORG_MODEL and BORG_MODEL_PROVIDER environment variables to point at your local instance. With Ollama running, borg becomes a terminal-based AI coding agent powered by your local GPU, with zero API tokens required and no data leaves your machine. We (the xShellz Team) build and run borg and the Agent Shell boxes. We use this same local-model setup ourselves when iterating in air-gapped environments.

how do i use borg with my local ollama models?

First, make sure you have a model pulled in Ollama. With Ollama installed and the server running, pull a model like Llama 3.1:

ollama pull llama3.1:8b

Then install borg as a single Go binary:\n\n``` curl -fsSL https://turborg.com/install.sh | sh


Set two environment variables to point borg at your local instance. The binary installs as both `borg` and `turborg`; you can use either:

export BORG_MODEL_PROVIDER=ollama export BORG_MODEL=llama3.1:8b


Now run borg with any one-shot prompt:

borg "explain the database schema"


The agent will use your local GPU and never send a request off the machine. The same environment variables work inside the REPL (just run `borg` without arguments) or for one-shot tasks. The config file lives in `~/.config/borg/` if you want to make settings permanent.

## what is borg and how does it work with local models?

Borg is an AI coding agent that runs in your terminal. It is a single Go binary with no runtime to install, available from the [borg page](https://www.xshellz.com/borg). Borg supports a bring-your-own-model setup out of the box. Instead of paying for API calls to a remote service, you point borg at a local LLM server and let your own hardware handle every inference. This keeps your prompt history, code, and model interactions completely on your machine.

Local models are treated as just another provider. Borg communicates with Ollama using its native HTTP API. For LM Studio, you configure borg as if it were a generic OpenAI-compatible endpoint. The same configuration pattern works for remote services like OpenAI or OpenRouter, so you never need to change how you call the agent when switching between local and cloud models.

## how to connect borg to ollama or lm studio

For Ollama, the setup is straightforward. Ensure the Ollama server is running, then set the provider and model name:

export BORG_MODEL_PROVIDER=ollama export BORG_MODEL=llama3.1:8b


For LM Studio, start the local server from the LM Studio UI (it serves on port 1234 by default). Then configure borg to treat it as an OpenAI provider with a custom base URL:

export BORG_MODEL_PROVIDER=openai export BORG_OPENAI_BASE_URL=http://localhost:1234/v1 export BORG_MODEL=lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF


Replace the model name with whatever model you loaded in LM Studio. If you set an API key requirement in LM Studio, add `export BORG_OPENAI_API_KEY=<your-key>` or a dummy value. After setting these variables, running `borg` will immediately use that endpoint. Borg logs the model in use at the top of each session, so you can double-check the connection.

## can i run borg with my own api keys or local models?

Yes. Borg accepts both remote API keys (OpenAI, OpenRouter) and completely local models served by Ollama or LM Studio. You are never forced to use a hosted plan. If you want zero external dependencies, point borg at a local server and no API key is required. If you later need a more capable model for a complex refactor, switch the provider to `openai` or `openrouter` and supply your own key. Borg does not lock you into any single backend.

If you want to give the agent root access without risking your host, you can run it inside an [Agent Shell box](https://www.xshellz.com/agent-shell). Those boxes use [gVisor](https://gvisor.dev/docs/) to sandbox the agent with a fake root, so even a misbehaving model cannot touch your real machine. The root filesystem is ephemeral; only `/home` survives a reboot, while your local Ollama server stays safe on its own machine.

## is it cheaper to run borg with local models vs openai?

Yes, if you already own hardware capable of running the model. Local inference has no per-token billing. The only cost is the electricity your machine draws while the model is running. In contrast, heavy coding sessions with a remote model like GPT-4o can burn through several dollars per hour. A local 7B model on a modern GPU costs nothing once the hardware is in place, and it never hits a rate limit.

Borg itself is free to use as a binary. You only incur costs if you choose to use a hosted model from the [xShellz pricing page](https://www.xshellz.com/pricing) or bring your own paid API key. For sustained daily use, keeping the agent local is the obvious choice for both privacy and cost.

## Frequently asked questions

### Do I need a GPU to run borg with Ollama?
Not strictly. Many small models run on CPU through Ollama, though a GPU drastically speeds up completions. For a snappy terminal coding agent, a modern GPU with at least 6 GB VRAM is recommended for 7B parameter models. Borg does not require a GPU; it just connects to whatever Ollama backend you provide.

### Can I use borg with OpenRouter to access many models with one key?
Yes. Set `BORG_MODEL_PROVIDER=openrouter`, provide your API key, and borg will route requests to whichever model ID you specify. This lets you switch between dozens of hosted models without managing separate accounts, though you still pay per token unless using a free-tier model on OpenRouter.

### How do I verify that borg is actually using my local Ollama and not some remote service?
Borg prints the provider and model name at the top of each session. You can also check your Ollama logs for incoming requests. If you see processing activity in the Ollama console while borg is responding, the agent is hitting your local server. No network calls leave your machine for the model inference.