20VC: "Anti-Data Centres is a Chinese Psyop" | How Many Planned Data Centers Will Actually Get Built? | Is Energy AI's Biggest Bottleneck? With Thomas Sohmers, Co-Founder @ Positron

Summary

Thomas Sohmers, co-founder of Positron AI, lays out a highly technical case for why AI inference needs different hardware than training: training is compute-bound and parallelizable, while inference is autoregressive and memory-bound. He argues the industry’s real constraint has been the widening gap between GPU FLOPs growth and memory-bandwidth growth, and says KV caching, quantization, and new attention mechanisms are now central to serving economics. Sohmers is also skeptical of claims that frontier AI labs are structurally unprofitable, arguing they could be highly profitable if they stopped training. The conversation turns sharply political on data centers, where he dismisses water/power criticism, claims anti-data-center sentiment is “almost entirely a Chinese psyop,” and says the U.S. should lean into domestic buildout. He closes by arguing that the next wave of AI demand will come from long-context agents, autonomous workflows, and ever-larger frontier systems that need much more memory capacity.

Notes

• Positron is building inference hardware from chips through rack-scale systems for generative AI.
• Inference is the core product focus, not training.
• Training is compute-bound and can be massively parallelized.
• Inference is memory-bound because each generated token must reread model weights autoregressively.
• The key structural difference is that training can crush through full corpora, while inference must generate token by token in order.
• He says GPU FLOPs improved about 120x from 2014 to 2024, while memory bandwidth improved only 17x.
• The compute-memory ratio has diverged, making memory-bound workloads more constrained over time.
• SRAM has lagged because the lowest-level memory primitive is a six-transistor cell that has not improved much for decades.
• He says SRAM’s architectural primitive has changed little in 30 to 40 years.
• The transformer era became obvious with GPT-3 and then ChatGPT.
• Cache reads are about 1/1000th the cost of recomputation, which drives provider margins.
• He says cached reads can generate obscene margins for model providers.
• Frontier labs are not structurally unprofitable in his view.
• He says frontier pacing could reinforce anti-tech sentiment and give ammunition to Luddites.
• His deeper fear is technological centralization that turns everyone else into serfs.
• He argues restricting who can do matrix multiplications would be a civilizational regression.
• He says anti-data-center politics has become a left-right consensus and is, in his words, largely a Chinese psyop.
• He disputes water and power criticism as misinformation.
• He says a single In-N-Out uses more water than the largest U.S. data centers.
• He says many data centers use closed-loop liquid cooling.
• He thinks many planned data centers can still get built by moving to friendlier geographies.
• He highlights Nevada and western desert land as a buildout opportunity.
• He says data-center projects often arrive with their own generation capacity.
• He argues grid interconnection could lower prices by adding supply.
• Energy is a foundational bottleneck for intelligence scaling.
• Positron’s base case is turning 500 MW of Nvidia-equivalent demand into 100 MW.
• He says economics may be a more binding constraint than physical energy supply.
• He worries about sovereign debt and currency devaluation more than company default risk.
• KV caching stores K and V matrices from the prompt and generated tokens to avoid recomputation.
• KV caching is an explicit tradeoff: more memory for less repeated compute.
• Attention compute grows quadratically with sequence length, while KV-cache storage grows linearly.
• He says KV-cache management becomes hard across many users and long contexts.
• Quantization has moved from FP32 to FP16, BF16, FP8, and now FP4 in some systems.
• Naive 16-bit to 4-bit quantization saves 75% of space but can severely hurt quality.
• Advanced quantization can compress values to about 4.5 bits per value while keeping losses near 1%.
• KV caching is optional, but the best workloads are highly cacheable.
• Semianalysis’s AgentX benchmark reportedly found about 96% of Claude code-session tokens are cacheable.
• Long-context sessions can dwarf model weights in memory footprint.
• A tiered memory hierarchy is the standard deployment pattern for serving.
• Accelerator memory, host memory, and NVMe are used in layers.
• For agents, context capacity is often more important than raw model capability.
• He expects frontier models to keep improving.
• He would pay 10x more per token for 10x more output value.
• He estimates the top four model companies handle 80% to 85% of tokens.
• Local models may increase cloud usage by autonomously deciding when to call bigger models.
• He believes scaling laws still hold and model sizes can keep growing past 50T and 100T parameters.
• He says GPT-6 Astra felt as magical as GPT-3.5 at launch.
• He says Astra can solve hard codebase problems one-shot.
• He describes an LLM-assisted RTL-to-GDS chip-design flow that finished in a little over 50 hours and hit timing above 1GHz.
• Positron’s next generation will have 8x more memory capacity than Nvidia’s top memory SKU.
• Context windows beyond 1M tokens are hard because memory cost scales quadratically.
• Chinese model labs, especially DeepSeek, have driven important attention-memory innovations under export constraints.
• DeepSeek v3 reduced KV-cache size with multi-head latent attention.
• Gated DeltaNet is one of his favorite mechanisms because it can cut attention time by about 75%.
• He says MLA-style compression is not free and likely costs capability.
• Advertised million-token contexts can still have poor recall above 64k tokens.
• He uses RULER-style needle-in-a-haystack tests to compare long-context performance.
• He claims GPT-5.6 scores about 70% and GPT-6 Astra over 95% on that benchmark.
• He thinks token price matters less than value per token.
• He estimates token value may be 100x to 1000x higher than prior generations.
• He expects pricing to move toward cost per useful result, though token pricing will likely persist.
• He sees the next big token growth wave coming from local LLMs that autonomously trigger bigger-model calls.