Why AI is best at finding counterexamples to conjectures

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 23:07

   That question has two directions. If you think about what you would need to prove in either direction, which direction of proof would be more amenable to AI? If P were equal to NP, then you would expect that ought to be very amenable to AI to find that because you could just, you just need to find an algorithm. And then, of course, give an analysis and show that it works. OK, but the search for algorithms is very, very parallelizable, right?

2. 23:37

   You could set 10,000 agents to try 10,000 different ideas for algorithms, right? And if just one of them works, then you win. So, you know, one pattern that we've seen is that AI has been especially good at finding counterexamples to widely believed conjectures. those conjectures have been false, okay? It did this, so with the Navier-Stokes, for example, where the answer was there is finite time blow up in these equations, so there is a counterexample to the smoothness of the solutions.

3. 24:17

   It did this with what was called the Jacobian conjecture, right, which turned out to be false, and there was famously, I guess, a counterexample that was found by fable during the world cup and that uh the the guy who uh had fable find it just just announced it via tweet okay and the a complete listing of the counterexample which was really just like this polynomial that takes three lines to specify was was right there in the tweet you know the the unit distance problem of paul eardish was another example where where ai found a counter example so so i would say there's sort of two two things that ai has going for it here one is that it's not uh blinded by preconceptions the way that we are right like it's not discouraged by like oh all the all the experts that believe this is true i'm surely going to fail and finding a counter example no if you tell it to look for a counter

4. 25:18

   example it's just going to keep looking right and the second thing is that looking for counter examples as i said is highly parallelizable okay so you can often like you know you just have to try a zillion different exotic constructions each one might be totally unmotivated might have some very high chance of failure you know and and a human would just very very quickly want to give up in doing that right it just it's just very very unrewarding to do and the ai doesn't care right it'll just keep looking at counter examples until it until it finds something okay so this is not the only thing that ai can do it can also prove you know so you know some of our long-standing you know actually many of our long-standing conjectures have turned out to be true and ai has proven them okay but it's been especially good at finding counter examples when when the

5. 26:12

   counterexamples have existed. You know, some people, they use that as a way to say that the AI is not that impressive. Yeah, right. Well, this is, you know, this was, this is what I was saying before about the sort of the infinite goalpost moving, right? AI saw, you know, like struggles with arithmetic, people laugh at it, right? And they say, okay, wake me up when it can actually do math. Okay, then it does like all high school math, right? All college level problem sets. People say, well, it doesn't count because it's not doing real research. Okay, then AI starts solving research problems. They say, well, it doesn't count. You know, obviously it could do research problems. No one ever doubted that. Of course, you know, those same people did doubt it six months ago. Pick me up when it can solve a really hard research problem, like a millennium problem,

6. 26:59

   right? Then it solves a millennium problem. And they say, well, it doesn't really count because it was just a counterexample, right? And it's like, no matter what happens, people will look at it after the fact and they will find a reason why it didn't right so like let's say that that within the next six months or a year as i think is probably more likely than not suppose that another millennium problem gets solved and suppose that the ai in this case proves that the the that the conjectured statement was true right like do do we agree in advance that that will count Or is there going to be a different excuse then?

7. 27:38

   OpenAI, Anthropic, and Cursor all use this product to make their lives easier. WorkOS is an API platform that adds enterprise features like SSO, SCIM, and RBAC to your app.