Stanford MS&E435 Economics of the AI Supercycle | Spring 2026 | Enterprise Internal Knowledge

Summary

Yash Patel, founder and CEO of Applied Compute and a former OpenAI post-training researcher, lays out a roadmap of AI progress from AlexNet and transformers through scaling laws, RLHF/RLVR, and reasoning models driven by test-time compute. He argues the main bottleneck has shifted repeatedly—from architecture and compute to data, then post-training signal quality, and now to continual learning from sparse real-world feedback. A major theme is that code and math are ideal RL domains because rewards are verifiable via compilation and tests, while enterprise customers need bespoke evals because “good” and “bad” differ by organization. Patel uses DoorDash, Cognition/Windsurf, and Cursor as examples of specialized systems that combine model training, context, telemetry, and tight latency constraints. He is bullish on transformer scaling and Nvidia-like compute suppliers, but says compute scarcity, data access, and the difficulty of creating new RL tasks will keep pushing innovation toward synthetic data, robotics/egocentric data, and better hardware.

Notes

• Yash Patel is introduced as Applied Compute's founder/CEO and a former OpenAI post-training researcher who went from Stanford directly into frontier model work.
• He says he met Sam Altman through mutual friends and later emailed him after ChatGPT launched to get into the OpenAI residency.
• AlexNet is framed as the pivotal deep learning breakthrough that showed scaling compute and data could dramatically improve predictive performance.
• Transformers and self-attention are described as the architecture that made large-scale language model training practical on GPUs.
• He places GPT-3 in the scaling-laws era and says Chinchilla formalized compute-optimal scaling with both larger models and more data.
• He says GPT-4 was a step change, then OpenAI's O model introduced test-time compute as a new scaling axis.
• Chain-of-thought is described as an emergent behavior rather than something explicitly trained by humans.
• He argues the bottleneck has shifted toward continual learning from extremely sparse real-world reward.
• Pretraining is still described as internet-scale next-token prediction over enormous corpora, which is not very data efficient.
• RL training is attractive because code and math provide deterministic, verifiable rewards through compilation and unit tests.
• He says code is also attractive because synthetic data is easy to generate and the internet contains a huge prior of code tokens.
• He says frontier labs have approached the limit of available data and need huge capital expenditure to keep training at the frontier.
• He expects pretraining innovation to come from better architectures that use existing data more efficiently, not from simply consuming more internet-scale data.
• He frames RL environments as a way to trade more compute for less high-quality data per rollout.
• Evals matter because they set the roadmap, and SWE-bench helped kick off the code-model race.
• He says enterprises need their own internal evals because definitions of good and bad differ by organization.
• Applied Compute is positioned as the specialization layer that helps enterprises optimize beyond general-purpose foundation models.
• He says DoorDash's menu extraction problem is hard because merchant inputs are unstructured and the modifier/add-on rules are highly specific.
• The DoorDash solution used human corrections, ground-truth comparison, and direct optimization against error rate.
• He describes one Applied Compute deployment as a vision model with transformer architecture.
• He argues enterprises will keep paying for frontier-model access now because the payoff from custom models is immediate and the world is fragmented.
• He says RL can use an order of magnitude less compute than pretraining and is more appealing than SFT or RLHF for customization.
• He cites DeepSeek as evidence that RL training can be a small fraction of pretraining compute, and that RL compute is rising.
• He adds that the share of compute devoted to RL is increasing meaningfully.
• A Cognition/Windsurf production model is described as catching bugs in under two seconds after a save event.
• He says product teams can use small, heavily post-trained models to navigate the performance-cost-latency frontier better than general models.
• He says Ramp Labs trained an RL model for fast spreadsheet search.
• Continual learning is defined as learning from sparse production rewards and updating deployed systems based on downstream consequences.
• He says the main blocker for continual learning is access to the right data and the right humans for feedback.
• Cursor's Composer is presented as an online-learning coding model trained on production telemetry over an open-source base model.
• He says Cursor's improvement loop runs over days or weeks, with roughly a couple hours per training step.
• Offline RLVR can replay the same task hundreds or thousands of times in parallel, unlike production systems.
• He proposes denoising gradients with massive batches of conversations, with each step taking hours.
• He says context-based learning spends compute offline to analyze documents and agent traces, then extracts learnings for downstream performance.
• He claims performance rose sharply at different reasoning efforts without increasing token usage.
• He says future gains will likely come from weight updates, context, and the harness.
• He says scaling transformers is working and is more likely to reveal the better architecture than designing one from first principles.
• He cites the non-transformer critique that humans do not need pretraining-scale data to learn language representations.
• He says chip and compute investments make it hard to pivot away from transformers, since the current ecosystem is optimized for them.
• He says compute scarcity is a major problem because demand is outpacing supply.
• He expects innovation in energy sources and more efficient chips, and says he would personally look into hardware.
• He says Nvidia will likely keep winning by supplying frontier labs, while noting a reported 75% margin on its chips.
• He suggests labs could someday spend a couple hundred billion dollars on in-house chip design and co-develop models and hardware.
• He is less bullish on the data market because improving the model makes the next RL task harder to define and sell.
• He says synthetic data generation is growing because RL often exploits a generator-verifier gap, especially in code.
• He expects the next data wave to move toward robotics and egocentric GoPro-style collection.
• He names Image 2 / Image Duo as a favorite AI product because it helps non-designers turn paper or documents into visual walkthroughs.