Key takeaways
- Cerebras said Nov. 17 that its disaggregated inference setup, with AMD Instinct GPUs handling prefill and CS-3 systems handling decode, is in production.
- In a Nov. 14 test on a shared GPT-OSS-120B endpoint, Cerebras reported five times the throughput per megawatt of a CS-3 only configuration with no loss in generation speed.
- Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute already have access, and Cerebras did not publish pricing.
In this story 3 sections
Cerebras disaggregated inference is now running in production, the company said Nov. 17, splitting each AI request between two types of hardware: AMD Instinct GPUs read and process the prompt, and Cerebras CS-3 systems write the answer. Cerebras Systems (NASDAQ: CBRS) said the setup produced about five times as many tokens per megawatt as running the same work on CS-3 machines alone, without slowing the speed at which text is generated.
The company described the system in a Cerebras blog post that came less than a week after it reported third quarter results, covered in our Cerebras earnings story.
Why Cerebras is handing prompts to GPUs
Serving a large language model involves two phases with different demands:
- Prefill, where the model reads the full prompt and builds its intermediate state, known as the KV cache. This step leans heavily on raw compute.
- Decode, where the model produces its response one token at a time. This step depends mostly on how fast the chip can pull model weights from memory.
Cerebras builds wafer scale processors packed with on-chip SRAM, which it has long pitched as an edge for fast output. Under the new design, those systems spend their time on decode, while GPUs from Advanced Micro Devices (NASDAQ: AMD) take on the prompt work. "Moving prompt processing to GPUs allows Cerebras systems to devote their serving capacity to response generation," the post said.
A software layer Cerebras calls KV Portal moves the KV cache and other model state from the GPU pool to the Cerebras pool over RDMA, a networking method that lets machines exchange memory directly. A separate framework routes incoming requests between the two pools and tracks the health and capacity of each replica. For prefill, Cerebras said it uses open source engines such as vLLM or SGLang.
How the 5x figure was measured
Cerebras based its claim on a shared endpoint for OpenAI's open weight GPT-OSS-120B model on the Cerebras Inference Cloud on Nov. 14. The company compared the split setup with a baseline in which CS-3 systems handled both phases and reported five times the throughput per megawatt. GPT-OSS-120B is the only model named in the test.
The post does not give absolute figures such as tokens per second or response latency, and it does not say which AMD Instinct GPU model sits in the prefill pool. A disclaimer at the bottom of the page says results may vary by workload, configuration and model.
Power explains why Cerebras chose this metric. The company said it has about 700 megawatts of planned data center capacity and aims to roll out disaggregated inference across hundreds of megawatts by the end of 2027. More output per megawatt means more tokens to sell from the same power budget, which the post frames as greater revenue potential for inference providers.
Who can use it now
Cerebras is offering the design in two ways:
- Through its own cloud, where the disaggregated setup adds capacity for Cerebras Inference Cloud customers.
- Inside partner data centers, using a Cerebras reference architecture. Partners can take the complete software stack, or a decode only kit that plugs the Cerebras decode engine and KV Portal into their own framework and prefill engine.
Amazon Web Services, the cloud unit of Amazon (NASDAQ: AMZN), along with Fireworks AI, Gimlet Labs and General Compute already have access, according to Cerebras, which said it is widening the program to more partners through a contact form. It did not publish pricing.
The pairing stands out because AMD is also building its own inference technology, including its purchase of chip designer Taalas, which closed Nov. 17 as covered in our AMD acquisitions report. Cerebras did not say whether AMD played any role in the new system beyond supplying the GPUs. For more chip and data center coverage, visit our tech section.
Share this story
Spotted an error? Tell us


