The alpha, twice a week
That was 1 min instead of 1 h 43 m.
Get the best new moments in your inbox every Wednesday and Saturday. It's free.
Check your inbox
We sent a link to your address. Click it to finish signing up. Not there in a minute? Look in spam.
or
Latent Space · 1 min instead of 1 h 43 m · Oct 2, 2026
One expert insight can erase a trillion tokens of search
Alex Zhang
Researcher, MIT
Shawn Wang (swyx)Host · Co-founder, AI Engineer
Wrote “The Rise of the AI Engineer” and founded Smol AI, maker of AI News.
About this segment
Alex says throwing compute at a problem can eventually explore the solution space. But someone who knows the problem can surface one insight that makes that brute-force spend unnecessary.
From “Recursive Language Models — Alex Zhang, MIT PhD”, Oct 2, 2026. Watch the full episode
“if you bring in someone who knows something about the problem, they can uncover something for the model that would like erase that one trillion token spend.”
Takeaways
- Domain experts still act as strong verifiers and steerers of AI
- Burning a trillion tokens can be replaced by one expert insight
- Efficiency still matters since compute can't be thrown at everything
In this episode
Full episode- 36:18RLMs generalize across tasks because the strategy stays the same4:44
- 17:12Academics should take big bets that industry labs won't4:02
9:38One expert insight can erase a trillion tokens of search1:19
Transcript
Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.
- 9:50
Like if you are knowledge or if you know what to do and you're also like I think the thing that we've kind of discovered with all these agents. swarms and things like this is like when you throw enough compute at a problem, you like can sufficiently explore solutions to that problem. But oftentimes, like, you know, maybe you can burn like 100 billion or a trillion tokens on something. But if you bring in someone who knows something about the problem, they can uncover something for the model that would like erase that one trillion token spend. I mean, it's not it's not super clear, like what exactly the trends are here. But I think like there are so many problems in the wild still right now that we want to solve and like we can't afford to just always, you know, throw as much compute as possible at it. Like
- 10:34
there is still an efficiency aspect of all of these things that is super, super important. Is there like a theoretical right answer that you just calculate based on physics and then you just get close to the physics?