Just in
TechCerebras Disaggregated Inference Pairs AMD GPUs With CS-3 SystemsAIMicrosoft Ignite 2026: Code 2, Perception and Cobalt 200 Are LiveTechOracle Critical Patch Update November 2026: 655 Fixes, 11 for JavaTechAMD Acquisitions: World Labs and Taalas Deals Close in Two DaysTechAnthropic IPO Date: Nov. 19 Pricing Assumed as Nasdaq Approves ANTAIGPT-6.2 Astra and Sol Arrive as Astra API Prices Fall 20%
Tech

Cerebras Disaggregated Inference Pairs AMD GPUs With CS-3 Systems

Cerebras now sends prompt processing to AMD Instinct GPUs and output generation to its CS-3 systems. The company said the split delivered five times the tokens per megawatt in a Nov. 14 test.

USA Publishing Group3 min read
Share
Glowing central processor on a detailed circuit board, representing the AI chips behind Cerebras disaggregated inference
Photo: Brecht Corbeel / Unsplash

Key takeaways

  • Cerebras said Nov. 17 that its disaggregated inference setup, with AMD Instinct GPUs handling prefill and CS-3 systems handling decode, is in production.
  • In a Nov. 14 test on a shared GPT-OSS-120B endpoint, Cerebras reported five times the throughput per megawatt of a CS-3 only configuration with no loss in generation speed.
  • Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute already have access, and Cerebras did not publish pricing.
In this story 3 sections
  1. Why Cerebras is handing prompts to GPUs
  2. How the 5x figure was measured
  3. Who can use it now

Cerebras disaggregated inference is now running in production, the company said Nov. 17, splitting each AI request between two types of hardware: AMD Instinct GPUs read and process the prompt, and Cerebras CS-3 systems write the answer. Cerebras Systems (NASDAQ: CBRS) said the setup produced about five times as many tokens per megawatt as running the same work on CS-3 machines alone, without slowing the speed at which text is generated.

The company described the system in a Cerebras blog post that came less than a week after it reported third quarter results, covered in our Cerebras earnings story.

Why Cerebras is handing prompts to GPUs

Serving a large language model involves two phases with different demands:

  1. Prefill, where the model reads the full prompt and builds its intermediate state, known as the KV cache. This step leans heavily on raw compute.
  2. Decode, where the model produces its response one token at a time. This step depends mostly on how fast the chip can pull model weights from memory.

Cerebras builds wafer scale processors packed with on-chip SRAM, which it has long pitched as an edge for fast output. Under the new design, those systems spend their time on decode, while GPUs from Advanced Micro Devices (NASDAQ: AMD) take on the prompt work. "Moving prompt processing to GPUs allows Cerebras systems to devote their serving capacity to response generation," the post said.

A software layer Cerebras calls KV Portal moves the KV cache and other model state from the GPU pool to the Cerebras pool over RDMA, a networking method that lets machines exchange memory directly. A separate framework routes incoming requests between the two pools and tracks the health and capacity of each replica. For prefill, Cerebras said it uses open source engines such as vLLM or SGLang.

How the 5x figure was measured

Cerebras based its claim on a shared endpoint for OpenAI's open weight GPT-OSS-120B model on the Cerebras Inference Cloud on Nov. 14. The company compared the split setup with a baseline in which CS-3 systems handled both phases and reported five times the throughput per megawatt. GPT-OSS-120B is the only model named in the test.

The post does not give absolute figures such as tokens per second or response latency, and it does not say which AMD Instinct GPU model sits in the prefill pool. A disclaimer at the bottom of the page says results may vary by workload, configuration and model.

Power explains why Cerebras chose this metric. The company said it has about 700 megawatts of planned data center capacity and aims to roll out disaggregated inference across hundreds of megawatts by the end of 2027. More output per megawatt means more tokens to sell from the same power budget, which the post frames as greater revenue potential for inference providers.

Who can use it now

Cerebras is offering the design in two ways:

  • Through its own cloud, where the disaggregated setup adds capacity for Cerebras Inference Cloud customers.
  • Inside partner data centers, using a Cerebras reference architecture. Partners can take the complete software stack, or a decode only kit that plugs the Cerebras decode engine and KV Portal into their own framework and prefill engine.

Amazon Web Services, the cloud unit of Amazon (NASDAQ: AMZN), along with Fireworks AI, Gimlet Labs and General Compute already have access, according to Cerebras, which said it is widening the program to more partners through a contact form. It did not publish pricing.

The pairing stands out because AMD is also building its own inference technology, including its purchase of chip designer Taalas, which closed Nov. 17 as covered in our AMD acquisitions report. Cerebras did not say whether AMD played any role in the new system beyond supplying the GPUs. For more chip and data center coverage, visit our tech section.

Cerebras Disaggregated Inference Uses AMD GPUs