Grok Bots and How CPUs are used in Agentic AI

Summary

Vikram Sekar and Austin Lyons use Grokbot as a springboard to argue that the real significance of easy-to-use agentic AI products is not the app itself, but the compute stack they require. They describe a cloud-VM sandbox with tools and sign-ins bundled inside, reducing setup and security burden for users while pushing more work onto server CPUs. The discussion distinguishes the host CPU that keeps a GPU fed from a second class of dedicated spillover compute they call “agentic CPUs.” They debate design tradeoffs across core count, single-core speed, power efficiency, memory coherency, and rack-scale orchestration, citing examples like AMD’s 256-core parts, Intel’s 288-core CPUs, and Intel’s P-rack/E-rack positioning. The broader thesis is that mainstream adoption of agentic AI could create massive incremental demand for cloud CPU cores, but software orchestration across heterogeneous hardware remains a key bottleneck.

Notes

• Grokbot is presented as an agentic AI platform, and the hosts treat it as an inflection point for compute demand rather than just a product launch.
• The hosts describe Grokbot as an agentic harness that automatically spins up multiple agents from a simple UI.
• Grokbot is framed as a cloud VM product with tools, sign-ins, and sandboxing inside one dedicated machine.
• The cloud VM model centralizes the agent runtime and removes the need for many separate integrations.
• The Mac Mini craze is used as an example of users wanting a local sandbox to run untrusted agent code.
• Security and isolation are described as a major reason users externalize agent execution into a separate machine or VM.
• The conversation ties the move from local sandboxes to cloud VMs to broader CPU demand.
• The hosts contrast self-hosted local boxes with cloud-hosted execution as two deployment models for agents.
• A third model is implied: dedicated cloud infrastructure for agents, beyond ad hoc local boxes and generic VMs.
• The GPU is repeatedly described as the value-creating “genius,” while the CPU is the execution layer.
• The CPU is cast as the assistant that carries out delegated work after the GPU decides what should happen.
• The host-node CPU must stay responsive and keep the GPU continuously fed with tasks.
• Fast per-core performance matters because the host CPU is constantly checking GPU completion and handing off work immediately.
• The hosts emphasize low-latency communication between CPU and GPU, especially in coherent systems.
• Memory coherency is treated as a desirable property because the assistant CPU can directly see and update the GPU’s working state.
• The hosts introduce “agentic CPU” as a term for CPUs dedicated to spillover agent work.
• Agentic CPUs are framed as dedicated racks for tasks the host node cannot comfortably absorb.
• Vikram uses an office-building analogy to explain partitioning many cores into work teams.
• Large multi-core CPUs are attractive because they can be partitioned into many small agent departments.
• AMD’s 256-core and Intel’s 288-core CPUs are cited as examples of scale-friendly agentic compute.
• CPU procurement is discussed in terms of cores per watt or per dollar, depending on the deployment constraints.
• Power budgets force buyers to optimize for cores, threads, and price rather than only raw performance.
• Intel is said to have launched P-rack and E-rack products for agentic AI workloads.
• The episode highlights heterogeneous rack design as a response to different agentic workload profiles.
• Rack-scale orchestration across mixed hardware is described as a difficult software-layer problem.
• The software layer must coordinate how CPUs and GPUs interact and move data across vendors and architectures.
• Grokbot is expected to increase CPU demand by making agentic AI accessible to more people.
• More mainstream adoption of agentic AI is expected to create many VMs and many more CPU calls.
• Austin argues that 10 million users could plausibly translate into 10 million cloud VMs and, with 100 agents each, a billion cores.
• The cloud VM path is described as the “easy button” for mainstream agentic AI adoption.
• The hosts expect normie adoption of agentic AI to be good for server CPUs.
• The episode closes with an open question about whether workloads truly land on specialized agentic CPUs or just general-purpose VMs.
• The hosts ask vendors to clarify the real workload placement tradeoffs between agentic CPUs and ordinary CPUs.