OpenAI’s Compute Chief: We Can’t Build Fast Enough | Sachin Katti

Summary

Sachin Katti argues that OpenAI’s central problem is not overbuilding compute but failing to build it fast enough to meet demand, which he says is consumed immediately when brought online. He describes AI infrastructure as a new industrial stack: supercomputers running at massive scale, liquid cooling everywhere, and power systems that require grid upgrades, new generation, transmission, and in some cases behind-the-meter gas turbines. He says nuclear energy is attractive as a dense, clean source, but the near-term constraint is still physical build capacity across turbines, transformers, electricians, plumbers, and construction. On the compute side, he explains why OpenAI is pursuing custom chips like Jalapeño to maximize tokens per watt, especially as inference becomes a larger share of workload. He also outlines Stargate as an umbrella strategy across hyperscalers, neoclouds, chip partners, and OpenAI-built infrastructure, and says AI is already starting to help design the next generation of AI hardware.

Notes

• OpenAI’s compute demand currently exceeds supply; anything brought online is used immediately.
• The bigger risk is underbuilding, not overbuilding.
• OpenAI frames its buildout as one of the largest infrastructure projects ever.
• AI data centers are described as large supercomputers.
• He uses the phrase “giant factories” turning electrons into tokens.
• Liquid cooling is mandatory because these chips run too hot for air cooling.
• Cooling has to happen across the data hall, the chips, the interconnects, and even power-distribution hardware.
• Liquid cooling is established, but scaling it reliably and cheaply is the real challenge.
• New liquids and materials are being explored to improve heat transfer.
• Better cooling lets chips run hotter, which increases memory bandwidth and FLOPs.
• OpenAI says it is investing in generation and transmission so it does not draw power away from the grid.
• If spare grid capacity is unavailable, OpenAI may fund new gas, solar, or hydro generation.
• Transmission lines, transformers, and substations are part of the data-center investment stack.
• When grid limits are hit, OpenAI is exploring behind-the-meter, on-site power.
• In the U.S., the near-term on-site option is gas turbines.
• Gas turbines are limited by supply-chain availability.
• Nuclear power is viewed as the ideal dense, clean source for large data centers.
• Jalapeño is OpenAI’s custom chip effort for inference efficiency.
• The optimization target for Jalapeño is tokens per watt.
• OpenAI says inference may already be the majority of its compute.
• Katti rejects a hard separation between training and inference.
• OpenAI compares its historical risk to a simple pattern: slowing down usually leads to regret later.
• The downside surprise is failing to build all the compute OpenAI wants.
• The physical world, not demand, is the limiting factor.
• Katti acknowledges community disruption but argues data centers are net positive locally.
• Site selection starts with land, permitting, and distance from neighborhoods.
• Access to power, strong gas availability, and labor are core siting criteria.
• The Industrial Compute team covers the full lifecycle from land and power to operations and allocation.
• OpenAI uses a portfolio approach rather than relying on one build model.
• Hyperscalers are expected to supply a major share of OpenAI’s compute.
• OpenAI’s compute sources include Microsoft, AWS, Google, Oracle, CoreWeave, and chip partners.
• Stargate is the umbrella strategy for scaling compute across multiple build-and-operate models.
• The Abilene, Texas site is already training OpenAI’s newest models.
• Oracle is building more public data centers in Michigan, Texas, and other places.
• OpenAI says it is the tenant or offtake party while partners handle financing.
• Jalapeño moved from design to tape-out in nine months.
• Three reasons Jalapeño moved fast: experienced designers, Broadcom, and knowing the target workload.
• AI is already helping design chips, and Katti expects it to design future AI systems too.
• MRC is a networking protocol for clusters of around 100,000 GPUs.
• MRC uses multipath spraying to tolerate failures across many routes.
• Bottlenecks span permitting, gas turbines, transformers, and labor.
• There is a shortage of electricians and plumbers, which is becoming more acute.
• OpenAI’s “guaranteed capacity” product is effectively guaranteed tokens.
• Enterprise customers want assurance that intelligence tokens will be available.
• Orbital compute is exciting, but it will only be one part of the overall arsenal.
• Orbital compute depends on cheap launch economics and cheap hardware economics.