pickuma.
AI & Dev Tools

How Much Mac Memory a 30B or 70B Local Model Needs

Weights plus KV cache must fit the GPU share of unified memory. What 32GB, 48GB, 64GB and 128GB Macs run at 30B and 70B, with the context math.

9 min read

A 4-bit Llama 3.3 70B is a 42.5 GB file, and on macOS 15 a 64GB Mac lets its GPU use about 48 GiB (51.5 GB) by default. That leaves room for roughly 22K tokens of context, not the 128K the model supports. The rule behind that number decides every question in this article: model weights plus the KV cache for your context must fit inside the GPU-usable share of unified memory. Total RAM is not the limit. The GPU share is.

This is the bigger-Mac follow-up to our 24GB M4 guide. That piece covers 7B-14B models on a base machine. This one covers the 32GB to 128GB tiers Apple sells now, the 27B-35B and 70B models people actually want to run on them, and the large mixture-of-experts (MoE) models that start to fit at 96GB and above.

Short answer by memory tier

The table below is the conservative case: the macOS 15 default GPU share (2/3 of RAM at 32GB, 3/4 above). macOS 26 and later give the GPU more by default; the values observed so far, and what they change, follow the table. “Raised” means you set iogpu.wired_limit_mb to leave 8 GiB for macOS.

Model, quant (file size), macOS 15 default / raised32GB36GB48GB64GB96GB128GB
Qwen3.8-27B UD-Q4_K_M (16.6 GB)69K / 112K160K / 176K256K256K256K256K
Qwen3.8-27B Q8_0 (29.1 GB)NoNo118K / 182K256K256K256K
Gemma 4 31B Q4_K_M (18.3 GB)26K / 60K99K / 112K214K / 256K256K256K256K
Qwen3.6-35B-A3B UD-Q4_K_M (22.1 GB)No / 97K250K / 256K256K256K256K256K
Llama 3.3 70B IQ2_M (24.1 GB)No10K / 13K38K / 51K77K / 102K128K128K
Llama 3.3 70B IQ4_XS (37.9 GB)NoNoNo / 10K36K / 61K112K / 128K128K
Llama 3.3 70B Q4_K_M (42.5 GB)NoNoNo22K / 48K99K / 128K128K
Llama 3.3 70B Q6_K (57.9 GB)NoNoNoNo53K / 104K128K
gpt-oss-120b MXFP4 (63.4 GB)NoNoNoNo128K128K
Qwen3.5-122B-A10B Q4_K_M (76.5 GB)NoNoNoNoNo / 256K256K

Each cell is the maximum context at the macOS 15 default limit / raised limit. One value means both settings reach the model’s maximum context. File sizes are the GGUF sizes listed on Hugging Face (unsloth/Qwen3.8-27B-GGUF, UD-Q4_K_M at 16.56 GB; unsloth/gemma-4-31B-it-GGUF; unsloth/Qwen3.6-35B-A3B-GGUF, UD-Q4_K_M at 22.13 GB; Llama 3.3 70B, gpt-oss-120b, Qwen3.5-122B-A10B). The context values are arithmetic, shown in the sections below, with an assumed 1.5 GB for runtime compute buffers and an FP16 KV cache. They are estimates, not measurements.

On macOS 26 and later, the default is higher where public logs exist: about 26.8 GB on a 32GB Mac and about 40.2 GB on a 48GB Mac. With the same method, that changes the 32GB and 48GB columns:

Model, quant32GB, macOS 26+ default (~26.8 GB)48GB, macOS 26+ default (~40.2 GB)
Qwen3.8-27B UD-Q4_K_M~128K256K
Qwen3.8-27B Q8_0No~141K
Gemma 4 31B Q4_K_M~73K~232K
Qwen3.6-35B-A3B UD-Q4_K_M~146K256K
Llama 3.3 70B IQ2_M~3K~43K
Llama 3.3 70B IQ4_XSNo~2K

No public macOS 26+ value has been reported yet for 36GB, 64GB, 96GB or 128GB, so those columns keep the macOS 15 figures.

Why the GPU share is the number that decides it

Apple Silicon has one memory pool, but macOS does not let the GPU wire all of it. Metal reports the default cap as recommendedMaxWorkingSetSize, and llama.cpp prints it at startup. Apple does not document the formula. On macOS 15 and earlier, public logs match 2/3 of RAM at 32GB and below and 3/4 above; the llama.cpp discussion on this limit traces this to a reserve of one-third on smaller machines and one-quarter on larger ones. On macOS 26 and later the default is higher: about 74% on 16GB and 24GB Macs and about 78% on 32GB and 48GB Macs, according to logs in ollama issue #15700 (an M5 MacBook Pro reporting 26,800.60 MB) and waired-agent issue #857 (an M5 Pro with 48GB on macOS 26.6.2 reporting 40,200,896,512 bytes).

Unified memorymacOS 15 defaultmacOS 26+ default (observed)Raised (RAM minus 8 GiB)
32GB22.91 GB≈ 26.8 GB25.8 GB
36GB28.99 GBNot yet reported30.1 GB
48GB38.65 GB≈ 40.2 GB42.9 GB
64GB51.54 GBNot yet reported60.1 GB
96GB77.31 GBNot yet reported94.5 GB
128GB103.08 GBNot yet reported128.8 GB

All values in this table are decimal GB, the unit Hugging Face uses for file sizes. Check your own machine: start llama-server and read the recommendedMaxWorkingSetSize line in the log.

You can raise the limit with one command. On a 64GB Mac, sudo sysctl iogpu.wired_limit_mb=57344 gives the GPU 56 GiB. The setting resets at reboot unless you add it to /etc/sysctl.conf. The same llama.cpp discussion advises against 100%, because macOS needs memory outside the GPU’s wired share. If memory pressure turns yellow or red in Activity Monitor, lower the value.

One unit trap: Apple sells memory in binary units (64GB means 64 GiB), but Hugging Face lists file sizes in decimal GB. A 42.5 GB file is 39.6 GiB. The tables in this article convert everything to decimal GB before subtracting.

The second half of the budget: KV cache

Weights are a fixed cost. The KV cache grows with every token of context, and its cost per token depends on the model’s attention design, not its parameter count. The formula for a standard attention layer is:

2 (K and V) × layers × KV heads × head dim × 2 bytes (FP16)

Using each model’s published config.json:

  • Llama 3.3 70B: 80 layers, 8 KV heads, head dim 128. That is 2 × 80 × 8 × 128 × 2 = 327,680 bytes, or 320 KiB per token. 32K tokens cost 10.7 GB. The full 128K window costs 43 GB, more than the 4-bit weights.
  • Qwen3.8-27B: 64 layers, but only 16 use full attention. The other 48 are Gated DeltaNet linear-attention layers, according to the model card. Those keep a fixed-size state (about 0.15 GB at 32-bit) that does not grow with context. The 16 full layers have 4 KV heads at head dim 256: 2 × 16 × 4 × 256 × 2 = 64 KiB per token. 128K tokens cost 8.6 GB.
  • Qwen3.6-35B-A3B: 10 full-attention layers out of 40, 2 KV heads, head dim 256: 20 KiB per token. The full 256K window costs 5.4 GB.
  • Gemma 4 31B: 50 of 60 layers use a 1,024-token sliding window, which caps their cache at about 0.84 GB. The 10 global layers use 4 KV heads at head dim 512, so at most 80 KiB per token. Google’s model card says the global layers share keys and values, so a runtime that stores them once needs half that. The table uses the larger figure.
  • gpt-oss-120b: 18 full-attention layers, 8 KV heads, head dim 64: 36 KiB per token. 128K costs 4.8 GB.

This is why the 27B and 70B rows in the table behave so differently. A 64GB Mac runs Qwen3.8-27B at Q8 with its full 256K window, but the same Mac holds only 22K tokens for a 4-bit Llama 3.3 70B. The 70B needs five times as much cache per token.

What each tier is good for

The tiers below match the configurations on Apple’s current spec pages for Mac mini, MacBook Pro and Mac Studio.

32GB (M6 Mac mini and 14-inch MacBook Pro). This is the entry tier for 27B-31B dense models. Qwen3.8-27B at UD-Q4_K_M leaves about 69K tokens of context at the macOS 15 default and about 128K at the macOS 26+ default. Gemma 4 31B at Q4_K_M fits with about 26K on macOS 15 and about 73K on macOS 26+. Qwen3.6-35B-A3B at UD-Q4_K_M needs the limit raised on macOS 15, but fits with about 146K context at the macOS 26+ default. A 70B does not fit at any useful quantization. On macOS 15 the 2-bit IQ2_M file (24.1 GB) does not fit at all; on macOS 26+ it fits with only about 3K context.

36GB (M5 Max base, MacBook Pro and Mac Studio). This tier adds only 4GB. On macOS 15 the default share also rises from 2/3 to 3/4, so the GPU budget grows by about 6 GB (22.9 GB to 29.0 GB). On macOS 26+ a 32GB Mac already gets about 78%, so 36GB adds only about 3 GB of GPU budget, assuming the same share applies. That is enough for Qwen3.6-35B-A3B at 4-bit with about 250K of its 256K context. A 70B fits only at 2-bit with about 10K context.

48GB (M5 Pro and M5 Max). This is the first tier where a 27B runs at Q8_0 (29.1 GB) with more than 100K context (about 118K on macOS 15, about 141K on macOS 26+). A 70B at 2-bit fits with about 38K to 43K context. A 70B at 4-bit (IQ4_XS) gets about 2K tokens at the macOS 26+ default and about 10K with the limit raised, so do not buy 48GB for that.

64GB (M5 Pro and M5 Max). This is the realistic floor for a 4-bit 70B. Llama 3.3 70B IQ4_XS (37.9 GB) gets 36K context at default and 61K raised. Q4_K_M gets 22K and 48K. For a coding agent that sends whole files, 22K is tight. Raise the limit, quantize the cache, or both. gpt-oss-120b (63.4 GB) does not fit.

96GB (M5 Ultra Mac Studio). A 4-bit 70B gets about 100K context at default and the full 128K with the limit raised. Q6_K fits too. gpt-oss-120b fits with its full 128K window. Qwen3.5-122B-A10B at Q4_K_M needs the raised limit.

128GB (M5 Max, MacBook Pro and Mac Studio). Every row in the table fits at its maximum context. This is also the only laptop tier where 120B-class MoE models fit at 4-bit.

The runner-up factor: bandwidth sets speed

Once a model fits, memory bandwidth decides how fast it runs. To generate each token, a dense model reads all of its weights once, so the decode ceiling is about bandwidth ÷ file size. Real throughput is lower, because compute and runtime overhead also take time. Treat these as upper bounds, not benchmarks. Bandwidth figures come from Apple’s spec pages.

Chip (bandwidth)Llama 3.3 70B Q4_K_M (42.5 GB)Qwen3.8-27B UD-Q4_K_M (16.6 GB)
M6 (170 GB/s)Does not fit at 32GB≤10 tok/s
M5 Pro (307 GB/s)≤7 tok/s≤18 tok/s
M5 Max, 36GB (460 GB/s)Does not fit≤28 tok/s
M5 Max, 48GB and up (614 GB/s)≤14 tok/s≤37 tok/s
M5 Ultra (1.2 TB/s)≤28 tok/s≤72 tok/s

This changes the answer in one case. A 64GB M5 Pro and a 64GB M5 Max hold the same models, but the Max has twice the bandwidth. If you plan to run a dense 70B every day, single-digit tokens per second on the Pro will feel slow, and the Max is the better purchase at the same memory size.

MoE models change the trade-off in the other direction. Qwen3.6-35B-A3B has 35B parameters but uses only 3B per token, according to its model card. gpt-oss-120b uses 5.1B of its 117B per token, according to OpenAI’s model card. These models need memory for all of their weights, but each token reads only a small fraction of them. As a result, they can decode much faster than a dense model of the same file size. If speed matters more than how a dense model handles hard prompts, an MoE model on a large-memory Mac can be the better choice than a dense 70B.

When a 70B is realistic, and when it is not the right target

In 2026, a 70B model on a Mac usually means Llama 3.3 70B, which Meta released in December 2024, or a fine-tune of a Llama 70B. Newer 70B dense releases exist from smaller groups: Swiss AI’s Apertus 70B (September 2025) is a separately trained base model, and Nous Research’s Hermes 4 70B (August 2025) is a fine-tune of Llama 3.1 70B. Both have the same memory footprint class as Llama 3.3 70B. The recent releases from Qwen, Google and OpenAI in this range are 27B-31B dense models (Qwen3.8-27B, Gemma 4 31B) or MoE models (Qwen3.6-35B-A3B, Qwen3.5-122B-A10B, gpt-oss-120b).

That leaves three conditions in which a 70B is a sensible target:

  • You need a specific 70B model or fine-tune, or you have a workload where you already confirmed that it beats the newer 27B-31B models.
  • You have 64GB or more and can accept 20K-60K of context, or you have 96GB or more for the full window.
  • You have M5 Max or M5 Ultra bandwidth, so decode speed stays above about 10 tokens per second.

If any of these conditions is false, a recent 27B-35B model at Q6 or Q8 with long context is the practical choice. It uses less memory, has a bandwidth ceiling about 1.5x (Q8) to 2.5x (Q4) higher than a 4-bit 70B, and leaves room for an editor, a browser and Docker.

How to check a model against your Mac before you download it

  1. Find your GPU budget. Read recommendedMaxWorkingSetSize from the llama.cpp startup log, or estimate it: 2/3 of RAM at 32GB and below and 3/4 above on macOS 15; about 74-78% of RAM on macOS 26 and later.
  2. Subtract the GGUF or MLX file size. MLX 4-bit files are about the same size as GGUF Q4. For example, Qwen3.8-27B MLX 4-bit is 16.1GB.
  3. Subtract about 1.5 GB for runtime buffers.
  4. Divide the remainder by the model’s KV bytes per token. Use the formula above with the full-attention layer count from config.json.
  5. If the result is below the context length you need, choose a smaller quantization, quantize the KV cache, raise iogpu.wired_limit_mb, or move up one memory tier.

FAQ

Can a 32GB Mac run a 30B model?
Yes, at 4-bit. Qwen3.8-27B UD-Q4_K_M is a 16.6 GB file and leaves about 69K tokens of context at the macOS 15 default, or about 128K at the macOS 26+ default. Gemma 4 31B Q4_K_M (18.3 GB) fits with about 26K on macOS 15 and about 73K on macOS 26+. Qwen3.6-35B-A3B at UD-Q4_K_M (22.1 GB) needs the GPU limit raised on macOS 15 only; on macOS 26+ it fits at the default with about 146K context.
Is 48GB enough for a 70B model?
Only at 2-bit. Llama 3.3 70B IQ2_M (24.1 GB) fits with about 38K context on macOS 15 and about 43K on macOS 26+. The 4-bit IQ4_XS file (37.9 GB) gets about 2K context at the macOS 26+ default and about 10K with the GPU limit raised. For a 4-bit 70B, 64GB is the realistic minimum.
Should I buy 64GB or 128GB for local LLMs?
Buy 64GB if you mostly run 27B-35B models, or a 4-bit 70B with short context. Buy 128GB if you want a 70B with its full 128K context, Q6 or Q8 quality, or 120B-class MoE models such as gpt-oss-120b and Qwen3.5-122B-A10B.
Is it safe to raise iogpu.wired_limit_mb?
It is a normal sysctl that resets at reboot. Leave at least 8 GiB for macOS and your other apps. On a 32GB Mac with macOS 26 or later, the default is already higher than that, so leave it alone. If the system starts to swap or memory pressure turns red, lower the value. Do not set it to all of your RAM.
Does M5 Pro or M5 Max matter if the memory is the same?
Memory size decides what fits. Bandwidth decides speed. The M5 Max at 48GB and above has 614 GB/s against 307 GB/s on M5 Pro, so a dense model can decode up to about twice as fast on the Max.

Related tools

Some links above are affiliate links. We may earn a commission if you sign up. See our disclosure for details.

Related reading

See all AI & Dev Tools articles →

Get the best tools, weekly

One email every Friday. No spam, unsubscribe anytime.