Running OpenCode with Local LLMs for Private AI Coding

One of the quietest objections to AI coding agents is that they send your source code to a third-party API. For proprietary systems, regulated code, or anything under NDA, that objection is a hard stop. OpenCode offers an alternative: run the agent against a local model through Ollama, and your code never leaves the machine.

We set this up for a client project that could not use cloud APIs. The goal was not to match Claude Sonnet. It was to find out whether a local model was useful at all for day-to-day coding tasks.

The Setup

The stack is simple on paper: Ollama serves a local model, OpenCode points at the Ollama endpoint, and the agent runs against http://localhost:11434. We used Qwen 2.5 Coder 14B on an M3 Max with 36 GB of unified memory. Smaller models work, but 14B was the smallest size that produced coherent multi-file edits.

Configuration lives in OpenCode’s provider settings. You add an Ollama provider, set the model name, and leave the API key blank. OpenCode then sends prompts to the local endpoint instead of Anthropic or OpenAI.

What Local Mode Handles Well

Three tasks worked reliably:

• Boilerplate generation. Creating a new API endpoint from an existing pattern, writing test stubs, and generating TypeScript types from a JSON sample. The local model followed existing conventions because the examples were in its immediate context.
• Small refactors. Renaming functions, extracting helpers, and updating call sites within a single module. The model made occasional import-path mistakes, but they were easy to catch in the diff.
• Code explanation. Asking “what does this function do?” or “why is this test failing?” produced useful answers because the answer required reasoning over code already loaded into context.

The common thread is that all three tasks fit inside the model’s context window and do not require deep architectural reasoning.

Where Local Mode Struggles

The local model fell down on anything that required planning across files. A task like “add pagination to every list endpoint” needs the agent to read route handlers, service functions, and response types across the codebase, then produce a consistent change. The local model either missed files or generated inconsistent implementations.

Speed was also a factor. A single prompt-response cycle against the local 14B model took 15-45 seconds depending on output length. Claude Sonnet over API returned in 3-8 seconds for similar prompts. Local inference is free, but it is not fast.

A Practical Split

The setup that worked best was a split workflow. Use the local model for:

• Writing new files from a clear pattern
• Explaining or summarizing existing code
• Tasks where latency does not matter

Switch to a cloud provider for:

• Multi-file refactors
• Debugging unfamiliar code paths
• Tasks where missing a file is expensive

OpenCode makes that switch easy because the model provider is just a config setting. You can run the same agent against Ollama in the morning and Claude in the afternoon without changing your workflow.

Hardware Notes

We tested on three machines:

• M3 Max, 36 GB RAM: Qwen 2.5 Coder 14B ran comfortably. 32K context worked without swapping.
• M2 Pro, 16 GB RAM: The same model was usable but slow. Context windows above 16K caused noticeable system slowdown.
• Linux desktop, RTX 4090: Faster than the Macs for the same model, but setup was more involved.

For occasional local use, 16 GB is enough. For daily local use, 32 GB or a dedicated GPU is strongly recommended.