How Much Mac Memory a 30B or 70B Local Model Needs
Weights plus KV cache must fit the GPU share of unified memory. What 32GB, 48GB, 64GB and 128GB Macs run at 30B and 70B, with the context math.
A 4-bit Llama 3.3 70B is a 42.5 GB file, and on macOS 15 a 64GB Mac lets its GPU use about 48 GiB (51.5 GB) by default. That leaves room for roughly 22K tokens of context, not the 128K the model supports. The rule behind that number decides every question in this article: model weights plus the KV cache for your context must fit inside the GPU-usable share of unified memory. Total RAM is not the limit. The GPU share is.
This is the bigger-Mac follow-up to our 24GB M4 guide. That piece covers 7B-14B models on a base machine. This one covers the 32GB to 128GB tiers Apple sells now, the 27B-35B and 70B models people actually want to run on them, and the large mixture-of-experts (MoE) models that start to fit at 96GB and above.
Short answer by memory tier
The table below is the conservative case: the macOS 15 default GPU share (2/3 of RAM at 32GB, 3/4 above). macOS 26 and later give the GPU more by default; the values observed so far, and what they change, follow the table. “Raised” means you set iogpu.wired_limit_mb to leave 8 GiB for macOS.
| Model, quant (file size), macOS 15 default / raised | 32GB | 36GB | 48GB | 64GB | 96GB | 128GB |
|---|---|---|---|---|---|---|
| Qwen3.8-27B UD-Q4_K_M (16.6 GB) | 69K / 112K | 160K / 176K | 256K | 256K | 256K | 256K |
| Qwen3.8-27B Q8_0 (29.1 GB) | No | No | 118K / 182K | 256K | 256K | 256K |
| Gemma 4 31B Q4_K_M (18.3 GB) | 26K / 60K | 99K / 112K | 214K / 256K | 256K | 256K | 256K |
| Qwen3.6-35B-A3B UD-Q4_K_M (22.1 GB) | No / 97K | 250K / 256K | 256K | 256K | 256K | 256K |
| Llama 3.3 70B IQ2_M (24.1 GB) | No | 10K / 13K | 38K / 51K | 77K / 102K | 128K | 128K |
| Llama 3.3 70B IQ4_XS (37.9 GB) | No | No | No / 10K | 36K / 61K | 112K / 128K | 128K |
| Llama 3.3 70B Q4_K_M (42.5 GB) | No | No | No | 22K / 48K | 99K / 128K | 128K |
| Llama 3.3 70B Q6_K (57.9 GB) | No | No | No | No | 53K / 104K | 128K |
| gpt-oss-120b MXFP4 (63.4 GB) | No | No | No | No | 128K | 128K |
| Qwen3.5-122B-A10B Q4_K_M (76.5 GB) | No | No | No | No | No / 256K | 256K |
Each cell is the maximum context at the macOS 15 default limit / raised limit. One value means both settings reach the model’s maximum context. File sizes are the GGUF sizes listed on Hugging Face (unsloth/Qwen3.8-27B-GGUF, UD-Q4_K_M at 16.56 GB; unsloth/gemma-4-31B-it-GGUF; unsloth/Qwen3.6-35B-A3B-GGUF, UD-Q4_K_M at 22.13 GB; Llama 3.3 70B, gpt-oss-120b, Qwen3.5-122B-A10B). The context values are arithmetic, shown in the sections below, with an assumed 1.5 GB for runtime compute buffers and an FP16 KV cache. They are estimates, not measurements.
On macOS 26 and later, the default is higher where public logs exist: about 26.8 GB on a 32GB Mac and about 40.2 GB on a 48GB Mac. With the same method, that changes the 32GB and 48GB columns:
| Model, quant | 32GB, macOS 26+ default (~26.8 GB) | 48GB, macOS 26+ default (~40.2 GB) |
|---|---|---|
| Qwen3.8-27B UD-Q4_K_M | ~128K | 256K |
| Qwen3.8-27B Q8_0 | No | ~141K |
| Gemma 4 31B Q4_K_M | ~73K | ~232K |
| Qwen3.6-35B-A3B UD-Q4_K_M | ~146K | 256K |
| Llama 3.3 70B IQ2_M | ~3K | ~43K |
| Llama 3.3 70B IQ4_XS | No | ~2K |
No public macOS 26+ value has been reported yet for 36GB, 64GB, 96GB or 128GB, so those columns keep the macOS 15 figures.
Why the GPU share is the number that decides it
Apple Silicon has one memory pool, but macOS does not let the GPU wire all of it. Metal reports the default cap as recommendedMaxWorkingSetSize, and llama.cpp prints it at startup. Apple does not document the formula. On macOS 15 and earlier, public logs match 2/3 of RAM at 32GB and below and 3/4 above; the llama.cpp discussion on this limit traces this to a reserve of one-third on smaller machines and one-quarter on larger ones. On macOS 26 and later the default is higher: about 74% on 16GB and 24GB Macs and about 78% on 32GB and 48GB Macs, according to logs in ollama issue #15700 (an M5 MacBook Pro reporting 26,800.60 MB) and waired-agent issue #857 (an M5 Pro with 48GB on macOS 26.6.2 reporting 40,200,896,512 bytes).
| Unified memory | macOS 15 default | macOS 26+ default (observed) | Raised (RAM minus 8 GiB) |
|---|---|---|---|
| 32GB | 22.91 GB | ≈ 26.8 GB | 25.8 GB |
| 36GB | 28.99 GB | Not yet reported | 30.1 GB |
| 48GB | 38.65 GB | ≈ 40.2 GB | 42.9 GB |
| 64GB | 51.54 GB | Not yet reported | 60.1 GB |
| 96GB | 77.31 GB | Not yet reported | 94.5 GB |
| 128GB | 103.08 GB | Not yet reported | 128.8 GB |
All values in this table are decimal GB, the unit Hugging Face uses for file sizes. Check your own machine: start llama-server and read the recommendedMaxWorkingSetSize line in the log.
You can raise the limit with one command. On a 64GB Mac, sudo sysctl iogpu.wired_limit_mb=57344 gives the GPU 56 GiB. The setting resets at reboot unless you add it to /etc/sysctl.conf. The same llama.cpp discussion advises against 100%, because macOS needs memory outside the GPU’s wired share. If memory pressure turns yellow or red in Activity Monitor, lower the value.
One unit trap: Apple sells memory in binary units (64GB means 64 GiB), but Hugging Face lists file sizes in decimal GB. A 42.5 GB file is 39.6 GiB. The tables in this article convert everything to decimal GB before subtracting.
The second half of the budget: KV cache
Weights are a fixed cost. The KV cache grows with every token of context, and its cost per token depends on the model’s attention design, not its parameter count. The formula for a standard attention layer is:
2 (K and V) × layers × KV heads × head dim × 2 bytes (FP16)
Using each model’s published config.json:
- Llama 3.3 70B: 80 layers, 8 KV heads, head dim 128. That is 2 × 80 × 8 × 128 × 2 = 327,680 bytes, or 320 KiB per token. 32K tokens cost 10.7 GB. The full 128K window costs 43 GB, more than the 4-bit weights.
- Qwen3.8-27B: 64 layers, but only 16 use full attention. The other 48 are Gated DeltaNet linear-attention layers, according to the model card. Those keep a fixed-size state (about 0.15 GB at 32-bit) that does not grow with context. The 16 full layers have 4 KV heads at head dim 256: 2 × 16 × 4 × 256 × 2 = 64 KiB per token. 128K tokens cost 8.6 GB.
- Qwen3.6-35B-A3B: 10 full-attention layers out of 40, 2 KV heads, head dim 256: 20 KiB per token. The full 256K window costs 5.4 GB.
- Gemma 4 31B: 50 of 60 layers use a 1,024-token sliding window, which caps their cache at about 0.84 GB. The 10 global layers use 4 KV heads at head dim 512, so at most 80 KiB per token. Google’s model card says the global layers share keys and values, so a runtime that stores them once needs half that. The table uses the larger figure.
- gpt-oss-120b: 18 full-attention layers, 8 KV heads, head dim 64: 36 KiB per token. 128K costs 4.8 GB.
This is why the 27B and 70B rows in the table behave so differently. A 64GB Mac runs Qwen3.8-27B at Q8 with its full 256K window, but the same Mac holds only 22K tokens for a 4-bit Llama 3.3 70B. The 70B needs five times as much cache per token.
What each tier is good for
The tiers below match the configurations on Apple’s current spec pages for Mac mini, MacBook Pro and Mac Studio.
32GB (M6 Mac mini and 14-inch MacBook Pro). This is the entry tier for 27B-31B dense models. Qwen3.8-27B at UD-Q4_K_M leaves about 69K tokens of context at the macOS 15 default and about 128K at the macOS 26+ default. Gemma 4 31B at Q4_K_M fits with about 26K on macOS 15 and about 73K on macOS 26+. Qwen3.6-35B-A3B at UD-Q4_K_M needs the limit raised on macOS 15, but fits with about 146K context at the macOS 26+ default. A 70B does not fit at any useful quantization. On macOS 15 the 2-bit IQ2_M file (24.1 GB) does not fit at all; on macOS 26+ it fits with only about 3K context.
36GB (M5 Max base, MacBook Pro and Mac Studio). This tier adds only 4GB. On macOS 15 the default share also rises from 2/3 to 3/4, so the GPU budget grows by about 6 GB (22.9 GB to 29.0 GB). On macOS 26+ a 32GB Mac already gets about 78%, so 36GB adds only about 3 GB of GPU budget, assuming the same share applies. That is enough for Qwen3.6-35B-A3B at 4-bit with about 250K of its 256K context. A 70B fits only at 2-bit with about 10K context.
48GB (M5 Pro and M5 Max). This is the first tier where a 27B runs at Q8_0 (29.1 GB) with more than 100K context (about 118K on macOS 15, about 141K on macOS 26+). A 70B at 2-bit fits with about 38K to 43K context. A 70B at 4-bit (IQ4_XS) gets about 2K tokens at the macOS 26+ default and about 10K with the limit raised, so do not buy 48GB for that.
64GB (M5 Pro and M5 Max). This is the realistic floor for a 4-bit 70B. Llama 3.3 70B IQ4_XS (37.9 GB) gets 36K context at default and 61K raised. Q4_K_M gets 22K and 48K. For a coding agent that sends whole files, 22K is tight. Raise the limit, quantize the cache, or both. gpt-oss-120b (63.4 GB) does not fit.
96GB (M5 Ultra Mac Studio). A 4-bit 70B gets about 100K context at default and the full 128K with the limit raised. Q6_K fits too. gpt-oss-120b fits with its full 128K window. Qwen3.5-122B-A10B at Q4_K_M needs the raised limit.
128GB (M5 Max, MacBook Pro and Mac Studio). Every row in the table fits at its maximum context. This is also the only laptop tier where 120B-class MoE models fit at 4-bit.
The runner-up factor: bandwidth sets speed
Once a model fits, memory bandwidth decides how fast it runs. To generate each token, a dense model reads all of its weights once, so the decode ceiling is about bandwidth ÷ file size. Real throughput is lower, because compute and runtime overhead also take time. Treat these as upper bounds, not benchmarks. Bandwidth figures come from Apple’s spec pages.
| Chip (bandwidth) | Llama 3.3 70B Q4_K_M (42.5 GB) | Qwen3.8-27B UD-Q4_K_M (16.6 GB) |
|---|---|---|
| M6 (170 GB/s) | Does not fit at 32GB | ≤10 tok/s |
| M5 Pro (307 GB/s) | ≤7 tok/s | ≤18 tok/s |
| M5 Max, 36GB (460 GB/s) | Does not fit | ≤28 tok/s |
| M5 Max, 48GB and up (614 GB/s) | ≤14 tok/s | ≤37 tok/s |
| M5 Ultra (1.2 TB/s) | ≤28 tok/s | ≤72 tok/s |
This changes the answer in one case. A 64GB M5 Pro and a 64GB M5 Max hold the same models, but the Max has twice the bandwidth. If you plan to run a dense 70B every day, single-digit tokens per second on the Pro will feel slow, and the Max is the better purchase at the same memory size.
MoE models change the trade-off in the other direction. Qwen3.6-35B-A3B has 35B parameters but uses only 3B per token, according to its model card. gpt-oss-120b uses 5.1B of its 117B per token, according to OpenAI’s model card. These models need memory for all of their weights, but each token reads only a small fraction of them. As a result, they can decode much faster than a dense model of the same file size. If speed matters more than how a dense model handles hard prompts, an MoE model on a large-memory Mac can be the better choice than a dense 70B.
When a 70B is realistic, and when it is not the right target
In 2026, a 70B model on a Mac usually means Llama 3.3 70B, which Meta released in December 2024, or a fine-tune of a Llama 70B. Newer 70B dense releases exist from smaller groups: Swiss AI’s Apertus 70B (September 2025) is a separately trained base model, and Nous Research’s Hermes 4 70B (August 2025) is a fine-tune of Llama 3.1 70B. Both have the same memory footprint class as Llama 3.3 70B. The recent releases from Qwen, Google and OpenAI in this range are 27B-31B dense models (Qwen3.8-27B, Gemma 4 31B) or MoE models (Qwen3.6-35B-A3B, Qwen3.5-122B-A10B, gpt-oss-120b).
That leaves three conditions in which a 70B is a sensible target:
- You need a specific 70B model or fine-tune, or you have a workload where you already confirmed that it beats the newer 27B-31B models.
- You have 64GB or more and can accept 20K-60K of context, or you have 96GB or more for the full window.
- You have M5 Max or M5 Ultra bandwidth, so decode speed stays above about 10 tokens per second.
If any of these conditions is false, a recent 27B-35B model at Q6 or Q8 with long context is the practical choice. It uses less memory, has a bandwidth ceiling about 1.5x (Q8) to 2.5x (Q4) higher than a 4-bit 70B, and leaves room for an editor, a browser and Docker.
How to check a model against your Mac before you download it
- Find your GPU budget. Read
recommendedMaxWorkingSetSizefrom the llama.cpp startup log, or estimate it: 2/3 of RAM at 32GB and below and 3/4 above on macOS 15; about 74-78% of RAM on macOS 26 and later. - Subtract the GGUF or MLX file size. MLX 4-bit files are about the same size as GGUF Q4. For example, Qwen3.8-27B MLX 4-bit is 16.1GB.
- Subtract about 1.5 GB for runtime buffers.
- Divide the remainder by the model’s KV bytes per token. Use the formula above with the full-attention layer count from
config.json. - If the result is below the context length you need, choose a smaller quantization, quantize the KV cache, raise
iogpu.wired_limit_mb, or move up one memory tier.
FAQ
Can a 32GB Mac run a 30B model?
Is 48GB enough for a 70B model?
Should I buy 64GB or 128GB for local LLMs?
Is it safe to raise iogpu.wired_limit_mb?
Does M5 Pro or M5 Max matter if the memory is the same?
Related tools
Some links above are affiliate links. We may earn a commission if you sign up. See our disclosure for details.
Related reading
2026-11-18
Local LLMs on a 16GB M4 Mac: What Fits and What Breaks
A 16GB M4 Mac lets the GPU use about 11.45 GB (macOS 15) or 12.7 GB (macOS 26+). Which 4B-14B models fit at 4-bit, and why 20B+ breaks.
2026-11-18
Picking an MoE Model for a 24GB Mac with MLX
Total parameters decide whether an MoE model fits in 24GB; active parameters decide speed. File sizes for gpt-oss, Gemma 4, LFM2.5, Qwen and Nemotron.
2026-10-15
What Muse Spark Open's 816B size means for developers
Meta's new open-weight coding model runs like a 42B model but needs the memory of an 816B one. Here's what that means for running it.
2026-10-14
What Anthropic's Scion means for Git and AI coding
Anthropic swapped Git for a custom system called Scion after merges jumped to 30,000 a day. Here is the background and what it means for your workflow.
2026-09-07
Grok 4.6 Scores 26% and 88% on the Same Benchmark Line
xAI's own model card reports Terminal-Bench 3.0 at 26.0%. Artificial Analysis reports 88.4% on 2.1. Both are the same model. The model card is unusually honest about why — the problem is everyone quoting it drops the qualifiers.
Get the best tools, weekly
One email every Friday. No spam, unsubscribe anytime.