The 100 AI chip startups will consolidate to a few winners

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 14:26

   To us, it's even more provocative. Historically, there have not been 100 competing processor vendors in any industry ever, right? So is this a more heterogeneous future? Will we see, is this a temporary thing? It'll convert back to a few? Yeah, and I see it as a more temporary thing and it will convert is what I expect. And there's probably three different reasons that I see that to be the case. One of them is this emerging heterogeneity that you see in the compute environment where, okay, now I have specialized pre-fill versus decode.

2. 15:02

   Well, you know, and now people are saying, well, we really need a specialized tier for mid-fill, not just pre-fill, right? So now I have early pre-fill and now I have mid-fill, which is, you know, different. Well, maybe speculate and verify if we listen to OpenAI here, right? Yeah, you know, and all of a sudden my compute fleet becomes more and more granular across the workload. And I think any time that you've seen that in history is not sustainable.

3. 15:27

   And then all of a sudden, right, when, you know, as you look at that, and now people are saying, well, as I go to reasoning models, I want something that looks more like a CPU again. So, you know, all of a sudden I didn't want that level, you know, that didn't become the predominant portion of my compute capacity. I needed more of it over here, right, and model and model workflows and, you know, so on.

4. 15:49

   You know, so I really don't like, I'll say at-scale specialization, right, because I think the workloads are going to continue to moderate and migrate so significantly. First, second is I think today's models are about to go through some evolutionary breakthroughs as well. I think the LLM is sort of reaching the limits. And now as people look to how do I really model 3D, where a flattish LLM doesn't work well when you go to molecules and chemicals and imaging.

5. 16:22

   So I think we'll see limits of that that will cause different shifts in the algorithmic domain. I also think that some of the most interesting workloads become where you start, you know, I'll say bringing HPC-like things back into AI-like things, where all of a sudden, you know, 64-bit precision counts again. What would be an example of that? Well, you know, imagine that my AI model is now inducing the five most interesting domains for a chemical analysis.

6. 16:52

   Okay, those chemical algorithms are going to run with high precision. which is going to be the dominant piece of the workload. Getting to the five algorithms I need to run or running the five algorithms on different chemical or biological systems. So I do think that as we optimize our AI systems, they will lead us back to some of the things that have been more traditional high-performance computing.

7. 17:18

   So I see that aspect of workload as well. So all of that said is this extreme specialization to me and the compute architect, I sort of don't like it because I don't think it's going to be what the workloads will look like two, three, four years from now. So that's one reason I don't see all of these AI chips as they get more and more specialized for portions of the compute workload to necessarily be right.

8. 17:43

   You know, second is, right, there's like a hundred of them now, right? You know, there's multiple optical ones, there's probabilistic ones and so on. And you know, they're not all going to win and they're not all going to win because at the end of the day, you have to get scale. on these things. And scale requires, you know, you have to win, you have to get capital, you have to get workloads onto it.

9. 18:01

   So I think it sort of defies logic that you're going to have a hundred of these things. So I see them, you know, coming back as a result of, you know, simple, you know, capital market share, et cetera, you know, which are the winning teams, you know, winning designs, winning architecture. So I see the workload driving that, you know, I see the natural, you know, industry consolidations, you know, that will occur.

10. 18:24

    And then I also expect that the winners will pick some winners, right? You know, where, you know, an open AI, an NVIDIA, an Anthropic will say, I like this one because it isn't just hardware. It's how do the hardware and software code evolve, right? You know, for it. And I do expect that there's going to be, hmm, you know, here's this 10 I could pick for this. I like that one. and it does require investment in the software and the workload evolution to take advantage of those platforms.

11. 18:55

    So for those three reasons, yeah, we're going to see a narrowing of the field. And the deployment complexity as well. Sorry, go ahead. Makes total sense. And I think I agree in most parts, let me still try to stir a little bit of controversy here. You, Guido, stirring controversy? No.