Local LLMs on a 16GB M4 Mac: What Fits and What Breaks
A 16GB M4 Mac lets the GPU use about 11.45 GB (macOS 15) or 12.7 GB (macOS 26+). Which 4B-14B models fit at 4-bit, and why 20B+ breaks.
A 16GB M4 Mac does not give your model 16GB. Metal reports a recommended GPU working set, recommendedMaxWorkingSetSize, and on this machine it is about 11.45 GB on macOS 15 and about 12.7 GB on macOS 26 and later. A 16GB M4 Mac mini’s whisper.cpp log prints recommendedMaxWorkingSetSize = 11453.25 MB, which is two-thirds of 16 GiB. A 16GB M4 Mac mini on macOS 26.5.1 reports 12,713,115,648 bytes. llama.cpp and Ollama treat that number as the GPU memory limit. The model weights, the KV cache for your context window, and the runtime’s working buffers all have to fit inside it. Everything else on this page follows from that one budget.
The short answer: 4B to 12B models at 4-bit run fully on the GPU with room for a long context. Dense 14B models fit only at short context. At 20B and above, the popular gpt-oss-20b is over the budget on macOS 15 and borderline on macOS 26+, and 26B+ models at 4-bit are over it on both.
Why the GPU budget decides everything
Apple Silicon uses unified memory, so the CPU and GPU share one 16GB pool. macOS does not let the GPU wire all of it. Apple does not document the formula, but llama.cpp prints the value at startup, and the public logs are consistent:
- macOS 15 and earlier: a 16GB M1 MacBook Air logged 10922.67 MiB, which is the same 11,453 MB in decimal units. The 16GB M4 Mac mini above logged the same figure. Use this as the conservative case.
- macOS 26 and later: the default is higher. The 16GB M4 report above is 12.71 GB, about 74% of RAM.
Check your own machine the same way: start llama-server or ollama serve with debug logging and find the recommendedMaxWorkingSetSize line.
The budget matters more than anything else for two reasons.
- Over the budget, the runtime splits the model. Ollama places as many layers as fit on the GPU and runs the rest on the CPU. One user on a 16GB M1 Air reported that
codestral:22b-v0.1-q2_Kshowed 9.6 GB and “100% GPU” from the CLI, but 19 GB and “43%/57% CPU/GPU” when an editor plugin requested it, and generation became 2-5x slower. Same model, same machine. The difference was the memory the second request asked for. - The rest is all that macOS and your apps get. 16GB of RAM is 17.18 GB in decimal units. With the GPU budget fully used, that leaves about 5.7 GB on macOS 15 and about 4.5 GB on macOS 26+ for the OS, a browser, and your editor. If those need more, macOS compresses memory and then swaps to the SSD. Swap does not crash anything, but it slows everything.
So the practical question for any model is: file size + KV cache at your context length + working buffers, compared to 11.45 GB (macOS 15) or 12.7 GB (macOS 26+). If you want a browser and an IDE open at the same time, plan on staying a few GB under it.
What fits at which quantization
File sizes below are the actual Q4_K_M GGUF downloads from Unsloth’s Hugging Face repos (Qwen3.5 4B, Qwen3.5 9B, Gemma 4 E4B, Gemma 4 12B, Ministral 3 8B, Ministral 3 14B, gpt-oss-20b, Gemma 4 26B A4B, Qwen3.5 27B, Qwen3.5 35B A3B, Gemma 4 31B). The KV cache column is arithmetic from each model’s config.json (shown in the next section), at the default f16 cache type.
| Model (Q4_K_M) | File | KV cache at 32K tokens | Fits default budget (11.45 GB macOS 15 / 12.7 GB macOS 26+)? |
|---|---|---|---|
| Qwen3.5 4B | 2.74 GB | ~1.1 GB | Yes, with lots of room |
| Gemma 4 E4B | 4.98 GB | small (sliding window + shared layers) | Yes |
| Qwen3.5 9B | 5.68 GB | ~1.1 GB | Yes, with long context |
| Ministral 3 8B | 5.20 GB | ~4.6 GB | Yes, but 32K uses most of the headroom |
| Gemma 4 12B | 7.12 GB | under ~0.9 GB | Yes |
| Ministral 3 14B | 8.24 GB | ~5.4 GB (~1.3 GB at 8K) | Short context only (8K comfortable, 16K tight) |
| gpt-oss-20b | 11.62 GB (Ollama tag: 14 GB) | ~0.8 GB | No on macOS 15; borderline at short context on macOS 26+ (the 14 GB Ollama tag is over on both) |
| Gemma 4 26B A4B | 16.95 GB (UD-Q4_K_M, Unsloth) | n/a | No (2-bit files start at 9.9 GB) |
| Qwen3.5 27B | 16.74 GB | n/a | No (2-bit files start at 8.6 GB) |
| Qwen3.5 35B A3B | 22.02 GB | n/a | No |
| Gemma 4 31B | 18.32 GB | n/a | No |
Two notes on the rows that look odd.
Gemma 4 E4B is bigger than “4B” suggests. The Gemma 4 E4B model card lists E4B at 4.5B effective parameters but 8B total with its per-layer embedding tables. The file size follows the total.
gpt-oss-20b barely changes with quantization. Most of its weights already ship in a 4-bit MXFP4 format, so Unsloth’s Q4_K_M (11.62 GB) and Q8_0 (12.11 GB) are almost the same size. Ollama’s library page says the model runs “on systems with as little as 16GB memory” and lists the gpt-oss:20b download at 14GB. Both statements can be true: it runs, but with 14 GB of weights against an 11.45 GB (macOS 15) or 12.7 GB (macOS 26+) GPU budget, part of it will run on the CPU unless you use a smaller file or raise the limit (see below). With the 11.62 GB GGUF and a short context, it is borderline on macOS 26+ and still over on macOS 15.
The newest Qwen releases do not change this table. On Hugging Face, the Qwen organization published Qwen3.6 only as 27B and 35B-A3B, and Qwen3.8 only as 27B and larger. For a 16GB Mac, Qwen3.5’s 4B and 9B are still the newest Qwen sizes that fit.
The runner-up factor: context length
After model size, the KV cache is what pushes a model over the budget. Its size per token is:
2 (K and V) × attention layers × KV heads × head dimension × 2 bytes (f16)
The architectures differ a lot here, and this is the reason to prefer some 8B-12B models over others on 16GB:
- Qwen3.5 4B and 9B use full attention in only 8 of their 32 layers; the other 24 use linear attention with a fixed-size state (config). 8 × 2 × 4 KV heads × 256 × 2 bytes = 32 KiB per token, so 32K tokens cost about 1.1 GB.
- Ministral 3 8B uses full attention in all 34 layers with 8 KV heads of dimension 128 (config). That is 136 KiB per token, about 4.6 GB at 32K, four times Qwen3.5 9B.
- Ministral 3 14B has 40 such layers: 160 KiB per token, about 5.4 GB at 32K. With an 8.24 GB file, that is 13.6 GB total, which does not fit on either macOS version. At 8K it is about 9.6 GB, which does; at 16K it is about 10.9 GB, which leaves almost no room for buffers on macOS 15.
- Gemma 4 12B uses a 1,024-token sliding window in 40 of its 48 layers and a single KV head in the 8 global layers (config). The sliding part is a fixed ~0.34 GB, and the global part is at most 16 KiB per token, so 32K stays under about 0.9 GB.
The Ollama issue above is this effect in practice: the same quantized model doubled its memory footprint when a client asked for a larger context. Three settings control it:
OLLAMA_CONTEXT_LENGTHsets Ollama’s default window. Ollama’s context length docs tier the default by VRAM: 4k tokens below 24 GiB, which is the tier a 16GB Mac falls in. Editor plugins and agents often request much more per call.OLLAMA_NUM_PARALLELmultiplies it. The Ollama FAQ says required RAM scales with parallel requests × context length.OLLAMA_KV_CACHE_TYPE=q8_0halves KV cache memory with flash attention on, per the FAQ, andq4_0cuts it to about a quarter at some cost in precision. In llama.cpp the equivalent flags are--cache-type-kand--cache-type-v.
Raising the limit, and what it costs
You can move the GPU cap with sudo sysctl iogpu.wired_limit_mb=N. The mlx-lm README says N should be larger than the model size in megabytes but smaller than the machine’s memory. The setting resets at reboot.
N is in MiB. On 16GB, this is how gpt-oss-20b becomes possible fully on the GPU on either macOS version: a limit of 13,500 MiB (14.16 GB) covers its 11.62 GB GGUF plus a short-context cache. The cost is simple subtraction: 17.18 GB of RAM minus 14.16 GB leaves about 3 GB (2.8 GiB) for macOS and every open app. Expect memory pressure and swap as soon as you open a browser next to it. That is a setup for a closed-lid server, not a laptop you are also working on.
A 2-bit file of a 27B model (8.6 GB for Qwen3.5 27B UD-IQ2_XXS) fits without touching the limit. Whether a 2-bit 27B beats a 4-bit 9B on your task is a quality question this article cannot answer, so test both on your own prompts before you commit to the bigger one.
Speed: bandwidth sets the ceiling
Apple lists the M4 at 120GB/s of memory bandwidth. For a dense model, generating each token reads roughly all of the weights once, so 120 GB/s divided by the file size gives an upper bound on decode speed:
- Qwen3.5 4B Q4_K_M: 120 / 2.74 ≈ 44 tokens/s ceiling
- Qwen3.5 9B Q4_K_M: 120 / 5.68 ≈ 21 tokens/s ceiling
- Gemma 4 12B Q4_K_M: 120 / 7.12 ≈ 17 tokens/s ceiling
- Ministral 3 14B Q4_K_M: 120 / 8.24 ≈ 15 tokens/s ceiling
Real decode speed is below these numbers, and prompt processing (the wait before the first token) is a separate cost that grows with prompt length. Mixture-of-experts models such as gpt-oss-20b or Qwen3.5 35B A3B read only their active experts per token, so their ceiling is higher, but only if the whole model fits in memory. Once layers spill to the CPU, the split itself becomes the bottleneck.
Which runtime on 16GB
Ollama is the shortest path, and its default GGUF/Metal path is the one to use here. Its MLX preview from March 2026 asked for “a Mac with more than 32GB of unified memory”. A June 2026 update shows a gemma4:12b-mlx tag without stating a memory floor, so whether that path is comfortable on 16GB is unconfirmed here. Watch ollama ps: if the PROCESSOR column shows a CPU/GPU split, you are over the budget.
llama.cpp directly gives you the most control on a tight machine: exact context size, KV cache type, and the startup log line that shows your real budget.
LM Studio loads both GGUF and MLX models (MLX through its open-source mlx-engine). Its system requirements recommend 16GB+ RAM, so this machine is at the bottom of its supported range.
mlx-lm runs mlx-community 4-bit conversions. Those files are close in size to the GGUFs: Qwen3.5 9B 4-bit is 5.95 GB of safetensors, so the same budget math applies.
What this article did not test
- No model was run for this article. File sizes come from Hugging Face repos, the GPU budget from public llama.cpp logs, and KV cache sizes and speed ceilings from the arithmetic shown above.
- KV cache figures exclude runtime compute buffers and Qwen3.5’s fixed linear-attention state. Your loader log prints the real total.
- There are no measured tokens/sec here. The speed numbers are bandwidth ceilings, not benchmarks.
- No answer-quality comparison between models or quantization levels was made.
FAQ
How much memory can the GPU use on a 16GB M4 Mac?
Can a 16GB MacBook Air M4 run gpt-oss-20b?
What is the largest model a 16GB M4 Mac runs comfortably?
Why did my model suddenly get much slower in an editor plugin?
Related tools
Some links above are affiliate links. We may earn a commission if you sign up. See our disclosure for details.
Related reading
2026-11-18
How Much Mac Memory a 30B or 70B Local Model Needs
Weights plus KV cache must fit the GPU share of unified memory. What 32GB, 48GB, 64GB and 128GB Macs run at 30B and 70B, with the context math.
2026-11-18
Picking an MoE Model for a 24GB Mac with MLX
Total parameters decide whether an MoE model fits in 24GB; active parameters decide speed. File sizes for gpt-oss, Gemma 4, LFM2.5, Qwen and Nemotron.
2026-10-15
What Muse Spark Open's 816B size means for developers
Meta's new open-weight coding model runs like a 42B model but needs the memory of an 816B one. Here's what that means for running it.
2026-10-14
What Anthropic's Scion means for Git and AI coding
Anthropic swapped Git for a custom system called Scion after merges jumped to 30,000 a day. Here is the background and what it means for your workflow.
2026-09-07
Grok 4.6 Scores 26% and 88% on the Same Benchmark Line
xAI's own model card reports Terminal-Bench 3.0 at 26.0%. Artificial Analysis reports 88.4% on 2.1. Both are the same model. The model card is unusually honest about why — the problem is everyone quoting it drops the qualifiers.
Get the best tools, weekly
One email every Friday. No spam, unsubscribe anytime.