Memory and Continual Learning: Engram's Dan Biderman and Jessy Lin

Summary

Dan Biderman and Jessy Lin describe Engram as a “neolab” built around memory and continual learning, with the goal of training per-team models that internalize workspace knowledge instead of repeatedly re-reading documents through prompts or RAG. They argue the core technical problem is deciding what should be stored in weights versus left external, and they emphasize white-box access, adapter fine-tuning, and other training methods as the path to making models learn from real work. The payoff they claim is dramatic: up to 100x fewer inference tokens on company-specific tasks, plus a model that can keep getting better as it sees more of a team’s context. The conversation also ranges into retrieval limits, KV-cache bloat, multimodal competition, and a neuroscience-inspired product vision in which each person or team has a personalized model that acts like an associative interface to the data plane.

Notes

• Engram is positioned as a memory-and-continual-learning company, not a traditional pretraining/post-training shop.
• The hosts frame the central bottleneck as adapting to new context, not making models smarter in a generic sense.
• The product is per-team models inside workspaces that improve on the things users care about over time.
• Adapter fine-tuning is part of the method stack, alongside LoRAs, prefixes, sparse architectures, supervised fine-tuning, RL, and policy distillation.
• White-box access to weights is required; open-source transformer models are the easiest deployment path.
• The company argues that internalizing repeated context can cut token consumption by two orders of magnitude.
• For many workplace tasks, they think the right answer can be produced in about 100 tokens instead of 100,000.
• They believe bespoke tasks can stay ahead of frontier models by roughly three to six months.
• The initial wedge is teams because they are more disciplined about collecting context over years.
• The hosts reject a clean split between memorization and learning, arguing that some memory is necessary to build higher-level concepts.
• Deep learning has blurred the old database-versus-algorithm split, but enterprise adoption is re-separating them through context engineering.
• Frontier labs are seen as prioritizing AGI-scale generic capability, while memory and continual learning remain more product-oriented work.
• The short-term pain point is inference cost when agents run for days and keep re-reading the same files.
• They argue repeated file reading across a company should be learned, not re-queried over and over.
• A key benefit of internalizing context is that the model can make associations users did not explicitly ask for.
• Retrieval systems fail partly because models do not know what to search for, not just because storage is hard.
• KV cache is described as a major infrastructure bottleneck, with a single Wikipedia article using about 80 GB of HBM on a 70B Llama example.
• They want offline compute to compress cached context by about 1000x.
• A visible proof point for continual learning would be an internal model that gets better after being taught over time.
• The desired end state is that time spent on a model improves quality in the domains users care about.
• They want learning on the fly to be safe and economically useful, not harmful.
• Product design likely needs separate memory boundaries for personal and work contexts.
• A long-term goal is safe transfer of skills and methods across jobs without leaking company IP.
• Jessy Lin asks why language beat vision in AI progress and whether vision can come back.
• Shaun Maguire expects language and vision to merge into a more unified multimodal system over time.
• Dan Biderman says language was unexpectedly hard because words sit in sparse high-dimensional spaces, yet models far larger than vision models still work well.
• Jessy Lin offers an information-theoretic explanation: biological vision has far higher bitrate than audio.
• Dan Biderman says humans are strongly vision-biased, but text is good enough for much knowledge work.
• Dan Biderman says vision-language training often leads language to dominate the joint system.
• Memory and navigation in neuroscience are used as an analogy for Engram’s long-term product direction.