The Biggest Chip Ever Built — Why OpenAI Runs On It | Cerebras CEO Andrew Feldman

Summary

Andrew Feldman says the next AI bottleneck is inference, not training, and that the right metric is tokens per second per user because latency shapes chat, reasoning, and multi-step agent workflows. He walks through the chip stack—from CPUs and GPUs to TPUs, Trainium, and ASICs—and argues that HBM, CoWoS packaging, and 3nm capacity are the real supply-chain constraints, which Cerebras sidesteps with a 5nm wafer-scale design built around SRAM and redundancy. Feldman also frames reasoning, verification, guardrails, and multimodal/video as compute-expanding trends that favor faster systems. The conversation closes on megawatt-scale infrastructure, including a large OpenAI deal, AWS’s disaggregated prefill/decode setup, and Feldman’s belief that faster AI will eventually reshape SaaS by making custom tools easier to generate than to buy.

Notes

• Cerebras describes its wafer-scale processor as the largest chip ever built in the computer industry.
• The company’s core claim is that larger chips can return answers faster because they move information more quickly.
• Feldman says Cerebras solved a problem nobody had solved before, but the market did not immediately care.
• He frames AI as moving from novelty toward production usefulness, with inference becoming the dominant bottleneck.
• The right performance metric is tokens per second per user, from first token to last token.
• Latency matters especially for chat and multi-turn agentic workflows, where waiting compounds across turns.
• Feldman treats AI as a broadband-like transition where speed changes the product category, not just the user experience.
• He lays out the chip landscape as a progression from CPU to GPU to TPU, Trainium, and other specialized ASICs.
• He argues the industry has been steadily specializing hardware away from general-purpose compute.
• Cerebras says AI demand is outpacing silicon supply rather than the reverse.
• OpenAI is presented as a customer that anticipated capacity shortages early and locked in supply.
• He says the AI supply chain has three major bottlenecks: HBM memory, CoWoS packaging, and 3nm capacity.
• HBM is the top bottleneck because only a few companies make it and they are sold out.
• Cerebras says its architecture avoids HBM, CoWoS, and 3nm constraints by using a 5nm wafer-scale design.
• The design relies on SRAM and a very large on-chip memory bandwidth advantage.
• Feldman says agentic AI increases CPU demand because actions like browsing and retrieval are CPU-executed.
• He distinguishes Cerebras from model-specific chip approaches by emphasizing underlying sparse linear algebra.
• The company aims to stay useful as model families change rather than hard-coding today’s favorite architecture.
• The chip took years to get working and entered an 18-month burn period of about $8 million per month.
• The project is presented as a deep-tech Everest: long, expensive, and uncertain before first success.
• Cerebras says it can shut down failed tiles and keep the wafer-scale machine running.
• The company’s architecture uses redundancy across roughly a million identical tiles to tolerate faults.
• Feldman says Cerebras handles not just inference but RL, traditional training, pretraining, and fine-tuning.
• The system is positioned as useful across several workload types, not only serving tokens.
• He says GPU training depends on difficult distributed compute across many devices.
• He says distributed GPU systems need a fabric for constant sharing, which is why Mellanox mattered.
• Cerebras says training with model data parallel is simpler than tensor model parallel.
• Reasoning is described as several drafts, with solving, review, and improvement before the final answer.
• Reasoning increases inference compute, and slow compute can become crippling.
• He argues reasoning makes Cerebras stronger because GPU slowness becomes more painful.
• Verification is framed as a guardrail step that checks whether an answer is wrong.
• Guardrails also consume compute because they require review by a model or scoring mechanism.
• He suggests small models can check the claims of large models, like data checkers in journalism.
• Cerebras says it is fastest on one of Google’s multimodal models.
• He argues multimodal AI matters because most text includes charts, graphs, and illustrations.
• Video is positioned as the next frontier, but one that is extremely compute intensive.
• Cerebras sells hardware to customer data centers and also offers rented capacity through its own data centers and cloud.
• The company serves AI-native firms, enterprises, and governments through both hardware and cloud access.
• Feldman describes a changing revenue mix, with hardware growing relative to other lines.
• The OpenAI deal is measured in megawatts, underscoring power as the binding constraint.
• Cerebras says OpenAI gets a full cloud solution through an API connection.
• Nordic data centers are attractive because of low-cost power, clean power, and cheap cooling.
• Transport latency matters, and he cites fiber running at roughly two-thirds the speed of light.
• AWS uses a disaggregated setup where Trainium handles prefill and Cerebras handles decode.
• He says customers care about fast tokens, which pushed Cerebras from pure chip supplier into data-center services.
• Customers can start on demand, then move to dedicated deployments after proving value with A/B tests.
• Feldman says the on-demand environment is available directly on the Cerebras site.
• He argues CUDA’s moat is shrinking as top models move to other stacks.
• He says inference has no moat because switching to Cerebras in the cloud is trivial.
• Cerebras’s defensibility comes from doing things others cannot do with its architecture.
• TSMC agreed to Cerebras’s chip-manufacturing proposal directly in the meeting.
• The collaboration required real but limited changes to TSMC’s process.
• He says chip design is tied to a specific fab’s rules, which makes switching foundries very hard.
• Cerebras says it brings chips back to the U.S., repackages and assembles them there, then ships from the U.S.
• Feldman predicts today’s models will soon look outdated and weak compared with future systems.
• He argues AI will unbundle SaaS by enabling custom tools built on demand.