Lab instruments need onboard AI because dumb data capture bottlenecks science

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 45:51

   what it would actually do when it's made. Maybe that's fine because maybe once it's made, It will do amazing things we can't even imagine. Good. Okay. I was going to move to the automating the lab side, where you've closed the loop on your experimentation. You can talk about robotic arms. Talk about basically just everything you've done here. My favorite quote from you was that every piece of equipment in your lab is going to have 140 IQ.

2. 46:16

   Okay, what does that mean? In the early days, as we were scaling things up, we realized that we had huge bottlenecks imposed just by operating machinery. So for one set of machinery, we would have technicians and scientists looking for particular morphology and trying to see, okay, what actually were we making, see if this is consistent with our intentions. And we really quickly, as we scale up the lab, came into these bottlenecks where it just wasn't keeping up.

3. 46:44

   So we started doing some programmatic approaches to capturing the data on SEM. And the data that was being surfaced from this very simple program where it sort of takes a field of view, captures, zooms in, captures, was just not all that useful. It was a little too dumb for what we really wanted to take care of. And so at that point, it was then really pertinent to build AI systems directly onto the machines to start controlling these things.

4. 47:12

   And now they have the full context as to what we're trying to achieve. So what was the intent of the experiment? What were we trying to synthesize? What were some other experimental evidence? And it's actually looking in the machine and capturing that data. And this is really valuable now because, you know, like we've been talking about these hidden variables and not being able to capture everything.

5. 47:31

   But if you can do more intelligent data capture at the time of that experiment, your data for future AI systems and future computational predictions is that much better. You'll have just a richer set of data. And so that's sort of what we mean by just, you know, everything on the lab has to be incredibly intelligent to just make the data as useful as possible. So that was sort of the motivation and inspiration behind that.

6. 47:57

   What's the state of the art there in terms of like... intelligence on every device right well i think uh one interesting aspect um is what's the latency exactly of controlling that instrument because there's cloud latency but then there's also device compute which that's right and then there's sort of a a time scale associated with different like physical processes and if calling out to an api or something is simply too slow so the amount of reasoning or tokens or tool calls is just not matched to the latency of that actual process, then that's infeasible. It sounds like self-driving cars, right?

7. 48:36

   Real quick, I'm actually surprised to hear that because the latency I'd imagine for reasoning seems small compared to a lot of these things take hours to run, right? Or maybe your experiments are much faster or high-tube or something. I'm just surprised to hear that. So ultimately, the experiments do take many hours, days, but there could be particular steps where latency wouldn't matter. Yeah, you might want to have finer control.

8. 49:04

   Another thing is how expensive it is because the same reason some of these models can be really slow in analyzing data also makes them very expensive. Then finally, the human patience. If a human wants the analysis of a certain XRD pattern and they have to wait two hours before they get a good result, that's very different, I think, than if they can get it in like two minutes. Oh, okay. Yeah, and then the reasoning, you're not doing like a single call to that model, right?

9. 49:31

   Like you're doing potentially many, many tokens to kind of get to these patterns. Like run simulations in the loop. Exactly. Deep research in the loop. It's not a single tool call. It's not a single inference thread. So then the latency can blow up quite a bit. I see what you mean. Yeah. So how much of your latency is tool calls? And I assume tool calls is largely like DFT or computational techniques in the F4.

10. 49:53

    Yeah. So how much of your latency is derived from those versus like actual reasoning? It's really process dependent. Okay, yeah. And then the other thing I think about is also, I guess, building up from small things to bigger things, where I assume that the general temptation or the typical development is incremental, where, like, everything is human-operated and then you find ways in which to automate it and then you sort of build up from there.

11. 50:19

    I worry that sometimes that that is the way that people evolve things, but that's a local minima, optimal. Like, shortcut is an optimization. Exactly, when, like, actually you should get a humanoid and just put them in there. I think we're in the opinion that solving humanoids would actually be slower to kind of getting to some of our goals. Just checking. Again, a lot of this is just, like, you do this every day, we, like, see this, but I don't know the reality of the situation.

12. 50:44

    A lot of people are hyperhumanoids. Yeah, I mean, I think like maybe one process is by having like a mix of humans and automation, you can identify really quickly, like what are some of the bottlenecks in the experimental process? And you start alleviating those bottlenecks one at a time. And so, for example, if there's some really tricky dexterity task that humans are excellent at, but you'd have to spend months automating machine, maybe don't spend a ton of time there.

13. 51:11

    And maybe like a more promising thing is something very routine that takes up a huge amount of scientists and technicians time, and it's easy to automate. So I think there's like this pragmatism to it. But I mean, ultimately, what we want from the lab is a huge quantity of data, high quality data, diverse data. And that those are our goals. And full autonomy is a non goal. It's sort of in the service that we use automation in the service of achieving the goals on the data.

14. 51:38

    yeah um but also i mean i think another aspect to automation is uh robots will just be uh make fewer mistakes potentially in the lab and it's been really helpful having ai systems with a full view over all of our data because sometimes if there's like a permutation in data uh then it can identify it. example, one of our steps at one point had a cyclic error because one of the machines was loaded incorrectly and the patterns were inconsistent. So the AI was reading through these things and it says, well, given what was run, this is not expected. And it's looking at this basket of data longitudinally. And then it realized if I do this like cyclic permutation and reverse it, everything is consistent. And then we're able to go back to the physical infrastructure and understand that a mistake had been made in the loading.

15. 52:30

    And I think managing data quality is so foundational to doing AI in the physical world. And these are the types of things that we're building in. And so you can improve the operating process. But at that point, you're like, okay, this is another great opportunity for automation. How do you make this just so reliable, so durable that we never have those types of mistakes again? Yeah, I mean, reducing the noise flow