“If there were a common failure reason, we'd have figured it out and fixed it. It is a long tail of constant discovery.”

“The coolest problem that's also the biggest problem is how we can non invasively write to the nervous system in sleep and waking states.”

“تمت الإجابة على كل جزء من هذه الأجزاء الثلاثة المختلفة للمعادلة بطريقة مختلفة، لكن الأكثر تعقيدًا كان تقدير الإنتاجية لكل وحدة معالجة رسومية (GPU)، لأننا اضطررنا لدمج الكثير من بيانات الاستدلال الخاصة بنا المتاحة للجمهور، ولكن على شرائح ونماذج لم تصدر بعد.”

“Each of these three different parts of the equation got answered in a different manner. But the most complex was estimating the throughput per gpu because we had to combine a lot of our inference x data that's already public, but on chips that are not out yet and on models that are not out yet. So we had to build an inference simulator where we had to estimate, okay, if Nvidia comes up with Feynman with certain set of characteristics, what does it mean for the throughput? If memory gets despec ed like it has happened, what's the impact on throughput? If OpenAI trains 10 trillion, 15 trillion, $20 trillion parameter model, what does it mean for throughput?”

“Each seed investment is an option bet. You're buying an option to see if it's an outlier.”

“The infrastructure to support AI wasn't going to be centralized. It was going to be distributed. Where energy was low cost and abundant energy prices actually come down.”

“The FICO short thesis has been that FICO risks losing its mortgage monopoly. It raised prices 1,600% over the last five years and has angered the entire mortgage ecosystem. The lesson here is don't piss off your regulator.”

“the obvious chart to me is just like the big step up for Reuben as a GPU is the memory bandwidth, which is like as you go from HBM3 to HBM4, you get almost a three times improvement in memory bandwidth. then that roughly seems to correspond with this you know three times improvement on the big middle parts of the curve where inference is memory bandwidth constraint.”