Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

Summary

This episode is a deep technical and strategic case study of Etched, the AI hardware startup founded by Gavin Uberti and Rob Wachen. The company’s thesis is that inference is not a generic chip problem: prefill is primarily a FLOPs-and-thermal challenge, while decode is a memory-bandwidth-and-interconnect challenge, so the hardware must be designed around those distinct bottlenecks. Etched says it has built a rack-scale product with custom power delivery, board design, and interconnects, and it claims its first-generation low-voltage inference system runs at under half the voltage of any other AI chip. The founders also emphasize speed of execution, vertical integration, and a talent model that mixes elite “Legends” with scrappy young builders. They close by arguing that inference will grow into a massive economic layer, with token production, concurrency, and availability becoming defining metrics for the next generation of infrastructure.

Notes

• Etched is positioned as a full inference systems company: rack, chip, power delivery, board, interconnect, and manufacturing are all part of the product.
• The founders split inference into two distinct workloads: prefill and decode.
• Prefill is treated as a data-loading and KV-cache staging problem, not a token-generation problem.
• For prefill, the company thinks FLOPs density and real-world MFU matter more than headline peak FLOPs.
• Rob says GPUs cannot simply be improved by adding FLOPs because thermal throttling caps usable performance.
• Etched’s core power bet is a new low-voltage inference mechanism.
• The first-generation low-voltage system is said to run at under half the voltage of any other AI chip.
• Decode is framed as a memory-bandwidth problem above all else.
• Etched says its custom interconnect stack is more than 5x better on latency than current GPU cluster behavior.
• The architecture is intentionally different from pre-ChatGPT AI chips across voltage domains, power planes, packaging, and boards.
• Rob argues inference adoption will be driven by productivity, scientific discovery, medicine, education, and concurrency.
• The current paid-AI user base is described as tiny relative to the global population.
• They expect inference to scale from small clusters to thousands or tens of thousands of chips.
• Rob’s motivation traces back to stage-four bone cancer in high school.
• He says surviving the illness changed what mattered and made him want to do something impactful.
• GPT-4V and image upload convinced Rob that the world would need much more AI infrastructure.
• Gavin’s first job at Xnor taught him that data movement, not math, dominates decode performance.
• He says most optimization work is moving data within and across chips.
• Gavin’s FTC robotics background reinforced a winner-take-all, narrow-focus mentality.
• The team says it set the world record score and ranked third globally for software development by OPR.
• Etched’s operating style is speed, product focus, and aggressive parallelization.
• Vertical integration is central, with the team preferring no vendor wherever possible.
• Etched says it is the only startup building both its own rack and its own chips.
• They validated thermal hardware before silicon returned using mock chips and cold plates.
• Etched runs a Taiwan factory, cloned test stations, a 2 MW on-site data center, and 24/7 development cycles.
• The team uses day and night shifts to accelerate bring-up.
• Hiring is explicitly bimodal: world-class “Legends” plus scrappy younger builders.
• Persistence matters in recruiting; many hires only close after repeated conversations.
• Brian Leyler is highlighted as a key legend hire from Nvidia’s HGX and DGX teams.
• The founders’ shorthand for the scrappy side of the talent mix is memorable: “chips on shoulders. Put chips in data centers.”
• Sanford, described as a world robotics champion, built a cold-plate contraption in a week to de-risk a power question.
• Contrarian founders say opportunistic talent tends to self-select out.
• The company is intentionally hard to join: pre-product, supply-constrained, and run by very young founders.
• The founders say the company must build the machine that builds the product, not just the first product itself.
• Etched’s Bangalore push was a save-the-tapeout effort that moved a dozen engineers overseas for six months.
• They used 12-hour handoffs to sustain a 24-hour development loop.
• The execution model is team-based; the founders emphasize recruiting great people to solve the binding constraint.
• Prefetching means doing everything possible before silicon returns.
• They shipped racks to customers, emulated the chip on 700+ FPGAs, and built thermal mockups before silicon return.
• Around-the-clock shifts helped them hit the 40-day bring-up target.
• Token capacity is the key business metric in Etched’s view.
• The natural business boundaries are the chip layer below and the model layer above.
• Etched uses human-AI hybrid kernel engineering and cares deeply about overlap and data movement.
• The company avoided generic graph compilers in favor of a kernels-first approach.
• A serious clock-domain-crossing bug forced them to align two clocks within 50 picoseconds.
• The fix used phase shifting and a drifting mechanism and took about two weeks.
• Their operating assumption is that hard problems are solvable through many experiments, even if most fail.
• They told investors they would spend about $100 million over the next 12 months.
• A detailed 30-page memo failed to convince major Valley investors at first.
• They originally thought the company might be possible on roughly $30 million and used debt financing to bridge the gap.
• The eventual Series A closed at $103 million from soft commitments.
• TSMC and Synopsys supported Etched with unusually favorable terms before the round closed.
• A TSMC executive followed up after a technical conversation and said he wanted to work with Etched.
• Patrick’s broader investing point is that experts called the company stupid, but base-rate thinking can miss rare opportunities.