Compute demand depends on price, and a glut will come

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 15:21

   So help us contextualize, Sam, maybe like how compute constrained do you feel today? And when you look at the build out over the course of the next two to three years, do you think you'll ever get to the point where you're not compute constrained? We talk about this question of, is there ever enough compute a lot? I think the answer is the best way to think about this is like energy or something. You can talk about demand for energy at a certain price point, but you can't talk about demand for energy without talking about different demand at different price levels.

2. 16:00

   price of compute per like unit of intelligence or whatever however you want to think about it fell by a factor of 100 tomorrow you would see usage go up by much more than 100 and there'd be a lot of things that people would love to do with that compute that just make no economic sense at the current cost but there would be new kind of demand so i think the the now on the other hand is the models get even smarter and you can use these models to cure cancer or discover novel physics or drive a bunch of humanoid robots to construct a space station or whatever crazy thing you want, then maybe there's huge willingness to pay a much higher rate cost per unit of intelligence for a much higher level of intelligence. That we don't know yet, but I would bet there will be.

3. 16:41

   So I think when you talk about capacity, it's like a cost per unit and capability per unit. And you have to kind of, without those curves, it's sort of a made up note. It's not a super well-specified problem. Yeah, I mean, I think the one thing that, Sam, you've talked about, which I think is the right way to think about is that if intelligence is more of a log of compute, then you try and really make sure you keep getting efficient.

4. 17:08

   And so that means the tokens per dollar per watt and the economic value that the society gets out of it is what we should maximize and reduce the costs. And so that's where, if you sort of where the Jevons paradox point is that, which is you keep reducing it, commoditizing in some sense intelligence, so that it becomes the real driver of GDP growth all around. Unfortunately, it's something closer to log of intelligence equals log of compute, but we may figure out better scaling laws and we may figure out how to do this.

5. 17:38

   We heard from both Microsoft and Google yesterday, both said their cloud businesses would have been growing faster if they have more GPUs. You know, I asked Jensen on this pod if there was any chance over the course of the next five years we would have a compute glut. And he said it's virtually non-existent chance in the next two to three years. And I assume you guys would both agree with Jensen that while we can't see out five, six, seven years, certainly over the course of the next two to three years for the reasons we just discussed, that it's almost a non-existent chance that you have excess compute.

6. 18:11

   Well, I mean, I think the cycles of demand and supply in this particular case, you can't really predict, right? I mean, even the point is, what's the secular trend? The secular trend is what Sam said, which is at the end of the day, because quite frankly, the biggest issue we are now having is not a compute blood, but it's a power. And it's sort of the ability to get the builds done fast enough close to power. So if you can't do that, you may actually have a bunch of chips sitting in inventory that I can't plug in.

7. 18:43

   In fact, that is my problem today, right? It's not a supply issue of chips. It's actually the fact that I don't have warm shells to plug into. And so how some supply chain constraints emerge, tough to predict, because the demand is just going, you know, is tough to predict, right? I mean, it's not like Sam and I would want to be sitting here saying, oh, my God, we are less short on compute. it's because we just were not that good at being able to project out what the demand would really look like. So I think that that's, and by the way, the worldwide side, right? It's one thing to sort of talk about one segment in one country, but it's about, you know, really getting it out to everywhere in the world. And so there will be constraints and how we work through them is going to be the most important thing. It won't be a linear path for sure. There will come a glut for

8. 19:30

   sure. And whether that's like in two to three years or five to six, Satya and I can't tell you, But like it's going to happen at some point, probably several points along the way. Like this is – there is something deep about human psychology here and bubbles and also as Satya said, like it's such a complex supply chain. Weird stuff gets built. The technological landscape shifts in big ways. So, you know, if a very cheap form of energy comes online soon at mass scale, then a lot of people are going to be extremely – with existing contracts they've signed, if we can continue this unbelievable reduction in cost per unit of intelligence, let's say it's been averaging like 40x for a given level per year, you know, that's like a very scary exponent from an infrastructure buildout standpoint.

9. 20:19

   Now, again, we're taking the bet that there will be a lot more demand as that gets cheaper. but I have some fear that it's just like, man, we keep going with these breakthroughs and everybody can run like a personal AGI on their laptop and we just did an insane thing here. Some people are going to get really burned like has happened in every other tech infrastructure cycle at some points along the way. I think that's really well said. And you have to hold those two simultaneous truths. We had that happen in 2000, 2001, and yet the internet became much bigger and produced much greater outcomes for society than anybody estimated in that period.

10. 20:53

    of time. Yeah, but I think that the one thing that Sam said is not talked about enough, which is, for example, the optimizations that OpenAI has done on the inference stack for a given GPU, I mean, it's kind of like, you know, we talk about the Moore's law improvement on one end, but the software improvements are much more exponential than that. Someday, we will make an incredible consumer device that can run a GPT-5 or GPT-6 capable model completely locally.