Rich Sutton and Khurram Javed: Why AI Models Stop Learning, and How to Start It Again

Summary

Rich Sutton makes the case that continual learning is the natural default for intelligence: agents act, learn, and update all the time, while the field has awkwardly treated learning as something separate from use. He and Khurram Javed connect this to the Bitter Lesson, arguing that systems should scale via computation and learning rather than injected human knowledge, and that LLMs are only a partial win because the Internet is finite and models do not really learn after deployment. Their “big world” view says the environment is vastly larger than any fixed simulator or dataset, so synthetic data and human-built simulations cannot be the full answer. They discuss continual learning as an algorithmic problem—catastrophic forgetting, per-weight step sizes, generate-and-test in feature space, and continual backprop with new random units. The episode closes on Oak Lab’s goal: a small team building a self-maintaining, energy-efficient system that can keep learning, plan with learned models, and eventually scale to something like a trillion-parameter mind running on 20 watts.

Notes

• Sutton’s baseline claim is that continual learning is just learning, and the field is the odd one for treating it as a special case.
• He situates that view as a long-held belief rather than a recent reaction to the LLM era.
• The Bitter Lesson, in Sutton’s framing, means favoring algorithms that scale with computation over systems that mainly scale with human knowledge.
• He treats LLMs as both proof and limitation of that lesson: they scaled impressively, but they still depend on finite Internet data.
• Sutton and Javed argue synthetic data is not a universal escape hatch because humans still decide what gets generated.
• Javed’s “big world hypothesis” is that the environment is far more complex than any agent or simulator, so the learning problem never really ends.
• They distinguish simulators from real continual learning by asking who keeps fixing the simulation: humans or the agent itself.
• In Javed’s view, agents should learn models from their own experience and repair bad models through continual updates.
• Sutton says the key missing capability is learning a model and then planning with it, especially when the model is not hand-specified.
• The Alberta plan was meant to turn broad ideas into concrete research steps, with continual deep learning called out as one of the most important.
• They frame continual deep learning as an algorithmic gap, not just an infrastructure or deployment problem.
• Naive single-sample online learning causes catastrophic forgetting by damaging earlier knowledge.
• Javed says some products approximate continual learning by batching data from many users, but most real settings are single streams.
• Sutton’s proposed fix starts with per-weight step-size optimization so older knowledge is not overwritten uniformly.
• He pairs that with generate-and-test in feature space to create new units instead of relying only on gradient descent.
• Sutton describes continual backprop as backprop plus periodic injection of randomly initialized units to preserve novelty over time.
• Javed says you likely cannot retrofit this algorithm onto an existing foundation model; you need to train from scratch.
• Sutton’s longer-term ambition is a self-maintaining system covering both small-scale and large-scale knowledge uniformly.
• The episode’s north star is a trillion-parameter system that can run on roughly 20 watts.
• Sutton translates the 20-watt target into Moore’s-law terms, arguing that a 2,000-watt system today could become 20 watts in about 10 years.
• Javed says the big labs are in a local minimum: new paradigms will likely get worse before they get better, which product-focused organizations resist.
• The company is aiming for a single design that can yield many minds adapted to different environments, not one universal mind.
• Javed ties that goal back to the big world hypothesis: no single system can absorb infinitely many things, so multiple systems will learn from different experience.
• They plan to start small and hire people who already share the paradigm, rather than scale headcount quickly.
• Sutton says animal learning is inspirational, but he rejects the idea that animals learn by supervised learning.
• He argues that no animal learns by supervised learning because bodies are not given target muscle twitches.
• He says school is a distraction as a model of intelligence because it is historically recent and not part of evolved learning.
• Sutton says the real challenge is spanning sensory-motor learning and higher abstraction in one system.
• He says the goal is to cover both low-level skills and abstract reasoning.