Dwarkesh Podcast

“the famous fact here is that an H100 costs more to rent now than it did 3 years ago, even though we have much superior technology. And we have much more compute in the world because as models get smarter, the opportunity cost of compute gets higher.”

“in the long run, it might make more sense to think of the size of our economy or the size of our civilization as the raw energy use that we do rather than GDP.”

“The idea that the United States cannot compete against that with mostly or fully automated solar panel manufacturing in the United States, which has cheaper natural gas by far, abundant oil, abundant human resources, great financial capacity, world-leading automation, et cetera, et cetera, is crazy. Like we could literally copy-paste solar manufacturing factories.”

“the answer is no, we're not totally screwed because batteries actually do the same job that the grid does... The grid transports power from one place to another... it's actually performing a spatial arbitrage... Now we have batteries. Batteries store power at one time of day and they release it at another time of day. So batteries are performing a temporal arbitrage”

“The learning rate for natural gas is nowhere near as steep as solar, which just tells you that it's easy to make solar panels, much easier to make solar panels. There are actually very few manufactured products which are easier to make. The Wright-Sawyer coefficient is 43%. So every time we double cumulative production, we get a 43% reduction in cost.”

“it turns out that Putin, Russia is much more vulnerable to blockade than the other way around, 'cause it's very easy to, close up the Baltic and Black Seas.”

“obviously robot arms are being manufactured in China. And so if you live in this world where The hardware is just incredibly valuable to ramp up manufacturing of because each robot can produce some fraction of the value that a human worker can produce”

“pre-AGI education is useful, post-AGI education is fun.”

“even Waymo and so on has very few cars and they're doing that roughly speaking because they're not economical... they had to make it uneconomical. So they have all these... costs, not just marginal costs... but also the CapEx of the entire thing.”

“Putin, he's trying to do empire. In the age of nationalism. Well, it's a non-starter. And he's dumping all of his ordnance on Ukraine, leaving Siberia wide open. And Xi Jinping has moved right in even before the Ukraine war. That's this Belt Road Initiative is peeling away the former Russian sphere of influence in Central Asia.”

“databases in fact will be a big thing that'll grow. In fact, if I think about all of the office artifacts being structured better data, the ability to do the joins between structured and unstructured better because of the agenting, what that'll grow the underlying, what is infrastructure business.”

“the thing that is the biggest competitor for any new accelerator is kind of even the previous generation of NVIDIA, right? I mean, in a fleet, what I'm going to look at is the overall TCO.”

“But like Oracle is going from like 1/5 your size to bigger than you by end of 2027. And while it's not a Microsoft-level quality of return on invested capital, right, they're still making 35% gross margins, right?”

“Software as a service, incredibly low incremental cost per user... This is why not Microsoft, but the SaaS companies have underperformed massively in the markets because the COGS of AI is just so high and that just completely breaks how these business models work.”

“companies that are fierce competitors starting to collaborate on AI safety. You may have seen OpenAI and Anthropic”

“it appears that Gemini have found a way to get more out of pre-training.”

“They're doing so well, but the economic impact seems to be dramatically behind. And it's almost like, it's, it's very difficult to make sense of how can the model on the one hand do these amazing things.”

“At some point though, pre-training will run out of data. The data is very clearly finite.”

“Like on tax policy, if we keep racking up our debt, it may get us out of the corner today, but is it going to really back us into a corner later on? And this is where Americans need to think a little harder about long-term implications of things.”

“If the capabilities were actually at AGI level, people would be willing to spend trillions of dollars a year buying tokens that these models produce. Knowledge workers across the world cumulatively earn tens of trillions of dollars a year in wages. And the reason that labs are orders of magnitude off this figure right now is that the models are nowhere near as capable as human knowledge workers.”

“But people are trying to launder the prestige that free-training scaling has, which is almost as predictable as a physical law of the universe, to justify bullish predictions about reinforcement learning from verifiable reward, for which we have no well publicly known trend.”

“If every iPhone was also a brain scanner, you would not have this problem and you would be training AI with the brain signals.”

“It can use the same Lean and same proof to do formally verified software. I think that's going to be a really powerful piece of cybersecurity. That's relevant for all sorts of other AI hacking the world stuff.”

“We are going to make solar. Okay. Yeah, great. Both SpaceX and Tesla are, are building towards 100 gigawatts a year of, of, uh, solar cell production.”

“If you take like refining of ore, I'd say roughly China does more, does twice as much ore refining on average as the rest of the world combined.”

“the interest payments to the national debt exceed the military budget, which is a trillion dollars. So over a trillion dollars just in interest payments.”

“China is extremely competitive in manufacturing. So, um, I think there's going to be a massive flood of Chinese vehicles and other, basically, most manufactured things.”

“the most valuable companies currently by market cap their output is digital. So NVIDIA's output is FTPing files to Taiwan.”

“the fabs today all basically use machines from like 5 companies. Yeah. You know, so. You know, you've got ASML, Tokyo Electron, KLA-Sencor”

“we've secured all the, all the chip fab production that we can.”

“I'd say my biggest concern actually is, is memory. Um, so the, I think there's, there's a, the, the path to creating logic chips is more obvious than the path to, um, having sufficient memory to support logic chips. That's why you see DDR prices going ballistic”

“It's the, it's the, the, the vanes and blades in the turbines, um, that are the limiting factor because the, the casting may, it's, it's like a very specialized process to cast the blades and vanes”

“I actually worry more about the drug approval process where I think AI models are going to greatly accelerate the rate at which we discover drugs and just the pipeline will get jammed up, like the pipeline will not be prepared to, like, process all of the stuff that's going through it.”

“when, for whatever reason, the models have those skills, then robotics will be revolutionised, both the design of robots because the models will be much better than humans at that, and also the ability to kind of control robots.”

“A thing I've been trying to fight for is export controls on chips to China. And that's in the national security interests of the US... And yet it doesn't happen. And we sell the chips because there's so much money, there's so much money riding on it.”

“However, that bottleneck goes away with AI. There are 100 million CCTV cameras in America, and you can get pretty good open-source multimodal models for 10 cents per million input tokens. So if you process a frame every 10 seconds, and if each frame is, say, 1,000 tokens, then for $30 billion, you can process every single camera in America.”

“while this year might cost $30 billion, next year it'll cost $3 billion, the year after that $300 million, and by 2030 it'll be less expensive to monitor every single nook and cranny in this country than it is to remodel the White House.”

“by late 2027, or certainly by 2028, there's going to be such wide diffusion that even open source models will be able to match the performance that the frontier had 12 months prior.”

“the main factor is going to be for reducing the number of people is modularizing things and making them in factories in Asia... these areas are going to ship more and more built-out sections of the data center”

“electrician wages probably double or triple again, right? And there's going to be a lot of new people entering that field and there's going to be a ton of people who make money.”

“I'm quite bullish that they're going to be able to do these things over the next 5 to 10 years. Really scale up production, really kick it into high gear.”

“memory vendors are just like sort of to some extent raising a lot of price... So who is able to accrue all the margin dollars is actually, you know, potentially the cloud, potentially the chip vendors and the memory vendors”

“that means the end consumer is paying $250 more for an iPhone... Apple has to either pass it on to the consumer... or B, they have to eat it.”

“TSMC is not raising prices, but memory vendors are... until TSMC or ASML like break out and they're like, no, actually we're going to charge a lot more.”

“Google had Nano Banana and Gemini 3, which caused their user metrics to skyrocket. And leadership at Google was like, oh... we have to double compute every... 6 months”

“Who saw the energy bottleneck before others asymmetrically went to, you know, Siemens, Mitsubishi, and of course GE Vernova and bought up turbine capacity. And now they're able to charge excess amounts for deploying these turbines”

“Bloom Energy's doing fuel cells. We've been like very positive on them for like a year and a half now because they have like such a capability to increase their production and their payback period for production increase is like very fast”

“I think at least this year we're going to see margins for the model vendors go up a lot, right? Because they're so capacity constrained, they have to destroy demand”

“because you are so limited on semiconductors... what is the value I can derive out of this chip today... an H100 is worth more today than it was 3 years ago.”

“who holds the card is the infra providers for charging margin... as you go upstream to, oh, well, who has access to all the memory and logic capacity? Well, it's, it's NVIDIA for the most part. They've signed a lot of long-term contracts.”