AI speeds chip design, but fab, memory and power still bind

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 9:17

   We're off to do a great new chip together. We're going to... We're building the AI. Right, right. And it's the one that's going to leap ahead because we have understanding of the AI... It's 2026. It's 2026. We have understanding of the AI workloads. We have understanding then of how to compose that into the right, multiply, accumulate, register structure. Perfect visibility with model workloads of tomorrow as well.

2. 9:37

   Yeah, and all of that kind of stuff. And let's say we turn our AI tools loose. We are great. Within three months, we have an awesome design, right? for that. We've been a little bit conservative on all the analog components of it. Now, we still have a couple of really major bottlenecks today. One is, right, we still have nine months of silicon processing time, right? Something like that. So, I can design the thing in three months, but I can't actually get it into real silicon at scale for nine months. Okay, that sucks.

3. 10:08

   So, we have to really see that bottleneck improve if we're going to have this innovation because we know... Are you talking about the validation and verification or even beyond that? Well, you know, first, I can't get anything out of FAB in less than three months. And that's if I have, you know, like supercharged design flows, right? So I said, you know, how do I compress that, right? You know, for it.

4. 10:29

   And then I got to get into advanced packages. 3D packages are getting complex. So that's like another three months, right? You know, and then I have to put it into a rack scale solution because nothing's a chip anymore. It's a rack. Yeah, so we're nine months. You know, it took me three months to design it, but it's nine months until I can actually start to use it. So how do we start to compress those aspects of design?

5. 10:51

   And I think we need new forms of lithography to enable that. We need design flows that don't require $50 million of mass cost until I can get things into prototyping. So to me, how do you compress that to a month or two? Because I can now do the design in three months. How can I have that done in a month or two? Because if it takes me a year and a half until I actually get scale and software on it.

6. 11:15

   Okay, my understanding of the AI workloads is no longer applicable to the chip that I designed. Right, you know, so, you know, all of those... We've seen that play out live right now, I think, with the AI chips, yeah. Right, you know, it's almost, you know, and we can look at it historically. It was like, you know, Graphcore wasn't a bad design, but the world moved on, right? You know, and, you know, in that.

7. 11:35

   So, all of those aspects somewhat obviate the fact that AI has made the design piece easy all of those manufacturing scale issues to me become a huge bottleneck, right? Another bottleneck, of course, is memory. Yep. Right, you know, memory bandwidth. We noticed, yeah. Right, and, you know, for that, as I've described, HBM is a hideous memory. It's just the best one that we've got. We were asking about that.

8. 11:59

   It's a terrible memory. You know, I get terrible bit density. I'm limited to shoreline bandwidth, the power, right, issues, thermal issues, DRAM doesn't like to be hot. Concentrating everything in one spot. Yeah, it's terrible to give, and AI is a memory compute workload. Yes. Right, so for that, so I need to compress that development and embodiment into real scale. I need to fix memory associated with this.

9. 12:32

   So to me, that's another critical bottleneck for us to attack. And power, right? You know, power in, you know, thermal, and most of the power simulation aspects today are bad, right? You know, you don't have good 3D modeling tools for power, power dissipation, hotspots, etc. You know, my voltage in, right, you know, I'm essentially guard banding almost my full power envelope. Correct. Right, so essentially I'm losing, you know, 40% of my power in guard banding, so I got to get much better at power modeling.

10. 13:04

    and then my power dissipation techniques are bad. The delivery model has to change too. Yeah, so all of these things, to me those are the next big bottlenecks. How do I improve each one of those and now if I've done that, okay, now we're on to an exciting era for what's next. So thank you for those great AI tools, but I got big challenges even when those are fully deployed.