Stanford MS&E435 Economics of the AI Supercycle | Spring 2026 | Infrastructure, Capstone Case

Summary

This episode frames AI as an industrial compute supercycle, with OpenAI’s industrial compute team responsible for the full stack from chips and memory to power generation, distribution, cooling, and land. The speaker argues that OpenAI’s revenue has tracked compute availability and utilization, and that the center of gravity is shifting from training to inference as products like ChatGPT and Codex become more agentic and more token-intensive. He emphasizes that agent workflows create complex compute graphs with tool calls, search, VM execution, and closed-loop iteration, which will require heterogeneous accelerators and better orchestration rather than a pure GPU monoculture. The conversation also highlights major bottlenecks in fabs, memory, ASML, grid stability, and labor, plus a belief that AI will increasingly help design the next generation of chips and low-level software. Long term, he expects profits to migrate from infrastructure to platforms and apps, but says the lowest layers of the stack remain the most compelling current investment area.

Notes

• OpenAI industrial compute spans the full physical supply chain, not just GPUs.
• Intel is described as benefiting from supply constraints and an agent-driven CPU comeback.
• OpenAI’s industrial compute job is to deliver compute across training and inference.
• Revenue and compute have both tripled year over year for the last three years.
• Codex reportedly saw double-digit growth within two weeks of the 5.5 launch.
• Codex is moving beyond coding into general-purpose knowledge work.
• Inference is already the majority of compute and is expected to exceed 80% of future compute.
• OpenAI wants every token to be cheaper, more intelligent, and needed in smaller quantities per task.
• OpenAI’s compute stack includes chips, memory, networking, power, cooling, buildings, generation, distribution, and land.
• Gigawatt-scale buildouts are operational coordination problems, not just procurement problems.
• The chips are described as brittle and highly sensitive to cooling and power fluctuations.
• Synchronized training jobs can move grid load by hundreds of megawatts.
• OpenAI is thinking about supply-chain de-risking through fabs, memory factories, natural gas, and nuclear.
• The speaker imagines a world where every person could have a GPU.
• A GPU is estimated at 1 to 2 kW, implying 7 terawatts for 7 billion people.
• The U.S. hyperscaler buildout is described as around 100 gigawatts of compute.
• That scale could consume a double-digit share of U.S. capacity.
• Compute advantages show up in more generous limits and fewer restrictions for users.
• Agentic AI is defined as closed-loop behavior, not just answering a prompt.
• Agentic workloads involve more complex compute graphs with tools, search, and execution environments.
• The goal is to make the human the bottleneck rather than the AI.
• Workload-specialized accelerators matter, including fast inference and long-context memory systems.
• The market will need multiple chip varieties because TSMC wafer allocation is multi-customer by design.
• Even inference-heavy services still favor concentration at large sites.
• Time to first token for agentic workloads is still about 400 to 500 milliseconds.
• Codex now uses 400k context, with all tokens computed before the first output token.
• Prefill latency dominates because the full context must be processed before generation starts.
• Faster token generation can expose previously hidden app and API latency bottlenecks.
• Latency reduction is expected to raise engagement, revenue, and retention.
• Current AI systems are still simplistic, with big compute units tied to one HBM layer.
• AI is already being used to design the next chip and low-level software.
• The chip design cycle is around three years from concept to production.
• Infra captures profits now, but value is expected to move up the stack over time.
• Foundational infrastructure is framed as an underbuilt and durable investment area.
• Technical depth plus scale make foundational infrastructure hard to replicate.
• Model wrappers are seen as weak businesses.
• The first $10 trillion company could be Nvidia.
• The biggest structural bottleneck is fab capacity across logic and memory.
• ASML is described as the deepest choke point in the supply chain.
• Near-term bottlenecks are expected in orchestration software, harnesses, and token efficiency.
• Longer-term bottlenecks are expected in memory architectures around the compute unit.
• Stargate is framed as a delivery and timing problem for compute, not just a capacity target.
• A gigawatt is described as about $70 billion of spend and half a million GPUs.
• Open-weight models have a role, but frontier intelligence still needs orders of magnitude more compute.
• A six-month lead in intelligence is portrayed as a very large strategic advantage.