How Cerebras runs a 3T-parameter model on 44 GB chips

Cerebras has published a technical walkthrough, How we deploy trillion-parameter frontier models on Cerebras. It explains how the company splits a model with about 3 trillion parameters across several of its wafer-sized chips, each of which holds only 44 GB of memory. The detail that decides what this means for you is a memory rule. According to Cerebras, every request in flight needs its own KV cache on each dense-layer wafer, so “the size of on-chip memory sets the ceiling on concurrent requests.”

The background you need

A few terms first, because the post assumes you know them.

Parameters are the numbers a model learned during training. A model’s size is usually given as its parameter count. To run the model, every parameter has to sit in memory somewhere. Cerebras says trillion-parameter frontier models need terabytes for their weights alone. Weights is another word for parameters.

Precision is how many bits each parameter uses. Fewer bits means a smaller memory footprint, usually with some loss of accuracy. In its sizing example, Cerebras works through what happens if all of Kimi K3’s parameters are stored at 4 bits each. That is a hypothetical for the calculation. It does not say how the model is stored in production.

On-chip memory versus off-chip memory. Most AI chips, including the GPUs you rent in the cloud, keep model weights in large memory banks next to the processor. Data has to travel between that memory and the compute units. SRAM is memory built into the chip itself. It is much faster to reach, but there is much less of it. Cerebras builds its chips from an entire silicon wafer instead of cutting the wafer into many small chips. That is what wafer-scale means. Even so, according to Cerebras, one WSE-3T wafer holds 44 GB of on-chip SRAM. That is far short of terabytes.

Layers are the stacked processing steps a model runs for each token. Kimi K3, according to Cerebras, has 69 linear-attention layers and 24 dense layers. Linear attention and standard (dense) attention are two ways a layer can look back over earlier text. They have different memory costs.

Activations are the intermediate results one layer hands to the next. They are small compared with the weights.

KV cache is short for key-value cache. As a model reads your prompt and writes its reply, attention layers store information about each earlier token so they do not have to recompute it. That stored information is the KV cache. It belongs to one request, and in standard attention it grows as the context gets longer. A long agent session uses far more KV cache than a short question.

Pipeline parallelism means splitting a model’s layers across several chips in order. Chip 1 runs the first layers, then passes its activations to chip 2, and so on. RDMA (remote direct memory access) is a way for machines to copy data straight into each other’s memory over the network with very little overhead.

The analogy Cerebras uses is an assembly line. Each wafer is a station. The tools (weights) stay bolted to the station, and the work in progress (activations) moves down the line. The analogy breaks in one useful place. A real station only needs room for its tools. Here, a station that runs a dense layer must also keep a separate notebook (the KV cache) for every request passing through. Those notebooks compete for the same limited shelf space as the tools.

What Cerebras actually described

The post does not announce a new chip or a price. It explains the deployment method, using the open-source Kimi K3 as the worked example. Here is what it lays out.

Layers fit whole on a wafer. Cerebras says each Kimi K3 layer needs about 16 GB. Each wafer has 44 GB, so entire layers typically fit on one chip. How many layers fit per wafer depends on the layer type. In most cases, the post says, the weights never have to move and layers do not need to be split across wafers.

Only activations cross between wafers. They travel over RDMA over Ethernet at roughly 2 microseconds per hop, according to Cerebras.

The pipeline is kept full. A common objection is that the wafers would sit idle waiting for their turn, which would be like running one request at a time. Cerebras says it avoids this by filling the pipeline with different requests. While stage 3 works on request A, stage 2 works on request B and stage 1 on request C. In production, the post says, Cerebras uses much finer-grained stages than one layer each, and many more of them.

Micro-batches set the speed-throughput tradeoff. Each stage can process a small group of requests at once, called a micro-batch. Cerebras says smaller micro-batches make each request faster, and larger ones increase total throughput. Requests in flight are roughly the number of stages times the micro-batch size.

Memory caps concurrency. Each dense-layer wafer already holds one dense layer. Whatever space is left becomes KV cache. Cerebras sizes the cache for a single 256k-token request, then divides the free space by that figure to get the number of concurrent requests. The post says it only needs to calculate KV cache for the dense layers. The published text does not spell out the reason.

Cerebras claims this setup runs multi-trillion-parameter models at 1000-2000+ tokens per second. It also cites benchmark scores for Kimi K3: 88.3% on Terminal-Bench 2.1 against 88.8% for GPT-5.6 Sol, and 91.2% on BrowseComp against 90.4%. The post adds that its performance comparisons come from third-party benchmarking or internal testing, and that results may vary by workload, configuration, date and model.

What it means if you use AI tools

The memory rule matters more than the speed headline. Speed per request and the number of users served at once pull against each other. Cerebras names both levers: micro-batch size, and how much on-chip memory is left for KV cache. A fast tokens-per-second figure tells you how quickly one reply streams. It does not tell you how many requests the hardware handles at that speed.

This is also why Cerebras itself says concurrency and batch size are not the right way to compare systems. It suggests total token throughput per unit of power, such as tokens per second per megawatt, measured at the same output speed. If you are choosing an inference provider for a coding agent or a high-volume app, that comparison is closer to what drives cost and capacity than a single speed number.

Context length is the second thing to keep in mind. Cerebras does its concurrency math in units of 256k-token requests. KV cache grows with context. So the capacity figures you see from any provider depend on what request length they assumed.

What to watch next:

• Independent benchmarks that report speed and concurrency together.
• Any tokens-per-second-per-megawatt figures, and whether they are measured at matched output speed.
• Whether providers say what context length their capacity claims assume.

What we don’t know yet

The published text leaves several things open.

• The actual numbers. The calculation results for total wafers, KV cache per request and total concurrent requests do not appear in the text, so the final wafer count and concurrency for Kimi K3 are not stated.
• Production precision. The 4-bit figure is a sizing hypothetical. The post does not say what precision Cerebras serves Kimi K3 at.
• Why only dense layers. The post says it only needs KV cache math for the dense layers but does not give the reason in the text. In general, linear-attention designs often keep a fixed-size running state instead of a cache that grows with context. That is our background explanation, not a reason Cerebras gives.
• Production stage layout. Cerebras says it uses much finer-grained stages in production but does not say how many or how large.
• Cost and power figures. Cerebras recommends tokens per second per megawatt as the metric, but the post does not publish that figure.
• Availability. The post invites readers to deploy models or use its inference cloud. It does not say whether Kimi K3 is offered there.