Anyone who regularly calls large model APIs to write code or run automated tasks has likely heard of Cerebras and its inference speed. Now that Moonshot AI has released Kimi K3, an open-weight flagship model with nearly 3 trillion parameters, a natural question for developers is whether they can run Kimi K3 directly on the Cerebras API to speed up their workloads. Yet as of October 25, Cerebras’s public shared inference catalog lists only two models, gpt-oss-120b and qwen-3.8-27b, with Kimi K3 nowhere to be found. Instead, it appears only in the dedicated enterprise model catalog, where customers must reserve capacity in advance and provision an organization-specific endpoint before they can use it.
Why can’t everyday developers access this model on a pay-as-you-go basis through the public API? We can find a clue in a technical blog post Cerebras published on October 22. The post walks through the hardware math of deploying Kimi K3. By its simplified calculation, deploying a single instance of Kimi K3 takes 59 wafer-scale chips. To understand why one model requires that many wafers, we first need to look at where Cerebras gets its inference speed.
Large language models generate responses one word or token at a time. The knowledge a model learns is stored in parameters called weights. Every time the model spits out a new word, the compute cores inside the processor have to read the weights needed for that step. On a standard GPU, the compute cores themselves are plenty fast, but the bottleneck is that the model weights live in GPU memory that sits outside the compute cores. For every single word generated, the GPU has to haul massive amounts of weights from external memory into the compute cores, spending most of its time just moving data back and forth. Making inference faster therefore comes down to cutting the distance and time required to move data.
Cerebras turns an entire silicon wafer into a single giant chip specifically to eliminate that data-movement distance. It builds memory directly alongside its 900,000 compute cores, a form of high-speed storage right next to the compute units known as on-chip memory. The model weights must be loaded into this on-chip memory ahead of time and stay fixed in place the entire time, allowing the compute cores to generate text at high speed right next to the weights without ever fetching data from outside the chip.
Still, the memory capacity of a single wafer is finite. According to Cerebras’s published hardware specifications, one wafer has only 44GB of on-chip memory, whereas popular large models today easily run into hundreds of gigabytes and clearly cannot fit on a single wafer. Why not store the weights in high-capacity off-chip memory and stream them into the wafer layer by layer for computation, as some media reports have suggested? Plenty of coverage, including an article from 36Kr and our own August article, described it that way, which was actually the training approach Cerebras introduced in 2022. During inference, if you had to haul weights from off-chip storage for every word generated, the wafer would be bottlenecked by off-chip memory bandwidth just like a conventional GPU.
The inference service Cerebras launched in August 2024 takes a different route by keeping the entire model’s weights in on-chip memory. Because a single 44GB wafer cannot hold a large model, the system slices the model along network layer boundaries and distributes different layers across multiple wafer systems running in series. Each wafer only computes the handful of network layers assigned to it, producing intermediate results called activations. One wafer passes those activations along to the next wafer to continue the calculation, while the weights themselves remain stationary in each wafer’s on-chip memory. As that launch announcement noted, running a 20-billion-parameter model takes 1 wafer system, while running a 70-billion-parameter model requires chaining 4 wafer systems together. Keeping weights on-chip is what buys generation speed, and the price is that the bigger the model, the more wafers you need.
The basic unit a model uses to process text is a token, roughly equivalent to a Chinese character or half an English word. For developers who simply call an API, a model’s internal architecture usually only affects GPU memory footprint or the price of long-context requests. In a wafer system where weights stay resident on-chip, however, the number of layers and the structure of the model directly dictate how many physical wafers sit in the data center.
We can work through the math starting from the blog post Cerebras published on October 22. According to the K3 technical report, Kimi K3 has roughly 2.8 trillion parameters across 93 network layers. Kimi K3 is a mixture-of-experts model composed of many distinct expert modules internally. Although it activates only a fraction of its parameters to generate each token, the system cannot know in advance which parameters will be needed next, so all parameters must reside in on-chip memory. To simplify the math, the Cerebras blog assumes all 3 trillion parameters are stored at 4-bit precision, meaning each parameter takes half a byte. That puts the total weight size of the model at roughly 1.5TB, or about 16GB per layer averaged across 93 network layers. Since a single wafer has 44GB of on-chip memory, fitting two layers of weights takes 32GB, leaving 12GB, which is not enough for a third layer. By the blog’s calculation, if wafers only had to hold model weights, fitting all 93 layers would require 47 wafers.
On-chip memory, however, has to hold more than just weights; it also needs to set aside space for the context cache of the request currently being processed. As a model generates a response word by word, each step requires caching intermediate results from the input prompt read so far and the words already generated in order to produce the next word. This space for recording history is the context cache, commonly called the KV cache. Without it, the model would have to recompute the entire preceding text from scratch every time it generated a single new word. In a wafer system, this context cache also consumes precious on-chip memory.
Kimi K3’s hybrid attention architecture saves a substantial number of wafers here. Large models mainly handle context in one of two ways: full attention or linear attention. With full attention, every time the model outputs a new word, it looks back at the context cache left behind by every previous word. As the input and output text grows longer, the memory consumed by the full-attention cache grows proportionally. Linear attention, by contrast, continuously compresses historical information into a fixed-size state. No matter how long the context gets, the state occupied by a single request never grows any larger. That fixed state still takes up memory, of course, and processing long inputs still incurs compute cost, but it does not endlessly eat up on-chip memory as the text lengthens.
According to the Kimi K3 technical report, its 93 network layers mix these two structures in a three-to-one ratio: 69 linear-attention layers + 24 full-attention layers. Because the cache for those 69 linear-attention layers does not balloon with sequence length, each wafer can still hold two layers of weights as usual: 34 wafers hold two layers each and the last wafer holds one layer, using 35 wafers in total. Meanwhile, the 24 full-attention layers need plenty of room for long-context caching, so the blog places only one 16GB layer of weights on each wafer and dedicates the remaining roughly 28GB of on-chip memory entirely to the cache, meaning those 24 layers require 24 dedicated wafers of their own. Adding the two parts together, 35 wafers plus 24 wafers comes out to 59 wafers in total.
Putting these numbers side by side makes the hardware ledger clear. Compared with the 47 wafers needed for weights alone, the extra 12 wafers in the 59-wafer total are set aside specifically to provide long-context cache space for those 24 full-attention layers. If all 93 layers used full attention, following the exact same formula from the blog would require 93 wafers, or 34 more than 59. By pairing three linear-attention layers with every one full-attention layer in a hybrid design, Moonshot AI saves 34 wafers on this ledger when deployed on a weight-resident wafer system.
Once you work through the math of those 59 wafers, the question at the beginning of this article answers itself. Kimi K3’s presence in the dedicated enterprise catalog simply means the platform is capable of deploying the model on a dedicated basis for enterprise customers. It does not necessarily mean 59 wafers are sitting idle in a data center ready for instant use at any moment. Cerebras has not officially explained why it splits its catalog this way. In our view, however, a deployment that strings dozens of wafers together in series and keeps all model weights permanently resident on-chip is very hard to sell piecemeal out of a shared public pool.
A public API is fundamentally a ride-sharing business that spreads hardware costs across scattered requests from a large pool of developers. Yet as long as a Kimi K3 deployment is running, the on-chip memory across dozens of wafers must remain packed with its weights, making it impossible to switch those chips over to other models on the fly. Moreover, once a model is partitioned across dozens of wafers, activations and request states must be handed off from one wafer to the next in sequence, tying dozens of machines together like a continuously running assembly line and concentrating operational overhead in cross-wafer scheduling. We suspect that a system where the entire production line has to spin up as a single unit is tough to parcel out for retail pay-as-you-go traffic; leasing it out as a full block of dedicated capacity to enterprise customers is the only way the economics pencil out.
This sales model also fits a broader industry trend: inference speed is becoming a separately priced commodity. Alongside raw model capability, speed is emerging as a second major axis of competition among large model APIs. Not only has Cerebras placed high-speed service for ultra-large models into its dedicated enterprise catalog, Moonshot AI has also priced speed as a standalone tier inside its own coding tools. According to the official Kimi model documentation, Kimi has already rolled out K3’s successor, K3.1 Pro, making K3 the previous-generation flagship, and K3.1 Pro comes with a high-speed variant called K3.1 Pro Highspeed. It matches the standard K3.1 Pro in model capability while generating tokens roughly 2 to 3 times as fast and consuming about 2 times the usage quota. Developers who want faster speeds on flagship large models either have to book dedicated capacity from a chipmaker or spend extra quota with the model provider.
If we actually went ahead and bought dedicated capacity, how much faster would Kimi K3 on Cerebras make our workloads? No third-party benchmarks have surfaced through public channels so far, so the outside world still does not know just how fast Kimi K3 runs on Cerebras in practice. Although the blog mentions that multi-trillion-parameter models can reach speeds of 1,000 to over 2,000 tokens per second, it never specifies whether that figure comes from actual Kimi K3 measurements or how such a test was conducted. Even the 59-wafer figure earlier is a simplified calculation used by the blog to illustrate the underlying principles; the post says nothing about the exact production deployment configuration or whether it is already live today.
The closest available data comes from an enterprise service announcement Cerebras published this May. In early May, the independent benchmarking firm Artificial Analysis tested an older model, Kimi K2.6, on a private Cerebras endpoint, and the announcement cited those test results. The test sent a request containing roughly 10,000 tokens of input. The model first had to generate a large amount of hidden reasoning content in the background before outputting a final 500-token answer. On that identical request, the official Kimi endpoint took 163.7 seconds, whereas the private Cerebras endpoint generated 981 tokens per second and finished the entire run in just 5.6 seconds. Our reading is that inside both the 5.6 seconds and the 163.7 seconds, the vast majority of the time went toward generating that hidden reasoning content, which explains why faster output speed created such a dramatic gap. Still, those numbers come from vendor-cited tests on an older model and cannot be taken directly as Kimi K3’s real-world performance.
Stepping back, even if a developer did run Kimi K3 on Cerebras, that raw speed applies only to the model’s output generation phase. As we noted in our September article benchmarking real-world inference speeds, in coding agent tasks such as automated programming, prompts are typically 100 to 600 times longer than the model’s output and are interleaved with frequent local tool calls. At the time, we benchmarked the Qwen 27B model hosted on Cerebras and measured a median pure output speed of 1040 tokens per second. In real coding tasks, however, because prompts are so long, the model has to finish reading the entire prompt before emitting its first token, stretching out the time to first token. Once that wait time is factored in, even excluding local tool execution time, the effective speed drops from 1040 tokens per second to 357 tokens per second. The FAQ in the official Kimi model documentation likewise states explicitly that high-speed mode only accelerates model output and has no effect on tool calls such as reading or writing files and executing shell commands. When tool execution makes up a large share of a task, the overall speedup is far less noticeable.
Returning to our opening question: why isn’t Kimi K3 on the public API? Cerebras achieves its high speed by keeping both model weights and caches inside on-chip memory, which means a 3-trillion-parameter Kimi K3 ties up dozens of wafer-scale chips at once. In our view, that hardware math makes the model impractical to slice into a shared public pool for pay-as-you-go retail traffic, and that is why it appears only in the dedicated enterprise catalog.