Best local LLM for coding: how to choose and run it with Ollama or LM Studio
Hardware you need, Ollama vs LM Studio, which model families to try, quantization and context length — and how to use a local model with Claude Code or OpenCode.
Running a language model on your own machine has gone from a hobby to a real option for programming. Your code never leaves the computer, there is no per-token bill, and it works on a plane. But “best local LLM” lists go stale within weeks. This guide shows how to pick the best local LLM for coding on your hardware, how to run it with Ollama or LM Studio, and how to plug it into a coding agent — including Claude Code.
1. Why run an LLM locally for coding
- Privacy. Proprietary code, client projects, anything under an NDA stays on your machine.
- Cost. No usage limits and no API bill — you pay with hardware and electricity.
- Offline. Trains, planes, flaky Wi-Fi.
- Control. The model does not change under you overnight.
The honest trade-off: local models are still clearly behind the top cloud models on long, agentic tasks — multi-step changes across many files, where the agent has to plan, call tools and recover from its own mistakes. For autocomplete, explanations, small edits and scripts they are often good enough.
2. How much hardware you need
What matters most is memory: VRAM on a graphics card, or unified memory on an Apple Silicon Mac. A rough rule for the common 4-bit quantization is about 0.6 GB per billion parameters, plus room for the context:
| Your machine | Model size that fits | What to expect |
|---|---|---|
| 8 GB | ~7–8B | autocomplete, explanations, short snippets |
| 16 GB | ~14B, small coding models up to ~24B | solid help with single files |
| 24 GB GPU / 32 GB Mac | ~27–32B | the sweet spot; simple agent tasks |
| 64 GB+ Mac or multiple GPUs | large mixture-of-experts models | closest to cloud quality |
For mixture-of-experts (MoE) models, the memory you need follows the total parameter count, but the speed follows the much smaller number of active parameters — which is why MoE coding models feel fast for their size.
3. Ollama vs LM Studio vs llama.cpp
- Ollama — command line and a local API on port
11434. Best if you want to connect agents and editors. - LM Studio — a desktop app with a model browser and a chat window,
plus a local server on port
1234. Best for trying models out. - llama.cpp — the engine underneath many tools, for maximum control. On a Mac, models in the MLX format run especially well.
All of them run the same open-weight models; pick by how you like to work.
4. Best local LLMs for coding: how to choose
We will not give you a ranking — by the time you read it, it would be out of date. Instead, the families that consistently show up at the top for coding, and how to pick within them:
- Qwen coder models (Alibaba) — the most common choice for local coding, from small to large MoE versions. A good default.
- Devstral (Mistral) — built specifically for agentic coding; a strong pick for 16–24 GB machines.
- GLM (Z.ai) — the big versions are near the top of open models but need server-class hardware; the smaller “flash” variants fit a 24 GB card.
- gpt-oss (OpenAI) — open-weight models in a smaller and a larger size, good at tool calling.
How to choose: take the newest coding model from one of these families that fits your memory with room for context, then try two or three on the same real task from your own project. Your codebase is a better benchmark than any leaderboard.
5. Quantization and context length
Quantization stores the model's weights with fewer bits so it fits
into memory. Q4_K_M is the usual best trade-off between size and quality;
go higher (Q5, Q6, Q8) if you have memory to spare, lower only if you must.
Context length is the most common reason a local model “cannot code”. Coding agents send a long system prompt, tool definitions and file contents — easily tens of thousands of tokens. Ollama's default context depends on your memory and can be as small as a few thousand tokens, which silently cuts the agent's instructions. Set it explicitly:
OLLAMA_CONTEXT_LENGTH=32768 ollama serveOllama recommends at least 32K tokens for coding; agents are more reliable with 64K if your memory allows.
6. Using a local model with Claude Code, OpenCode or Codex
Claude Code with Ollama. Since version 0.14, Ollama speaks the Anthropic Messages API, so Claude Code can talk to it directly:
ollama pull qwen3-coder
export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_BASE_URL=http://localhost:11434
claude --model qwen3-coderThe token is a placeholder that Ollama ignores, but it has to be set. Expect the agent to be noticeably weaker than with Claude models on complex tasks, and test features such as subagents before you rely on them. Note that cloud-only features of Claude Code — Remote Control, cloud sessions — need a claude.ai account and do not work with a local model.
OpenCode is built to be model-agnostic: add Ollama or LM Studio as a provider in its configuration and pick the model. Codex can also use local open-source models through Ollama or LM Studio. More on the three agents in Claude Code vs Codex vs OpenCode.
7. What to expect in practice
- Great: autocomplete, explaining code, writing tests for a single function, regexes, shell one-liners, converting between formats.
- Good with a strong model: changes within one or two files with clear instructions.
- Still hard: long autonomous runs across a large codebase. Here the top cloud models are worth the money.
Many developers end up with a mix: a local model for private code and quick help, a cloud model for the big agentic tasks. Whatever you choose, keep reviewing what the model writes — see how to vibe code without losing control.
Do you see it differently? Write to support@coden-app.com.