Real Alpha

a16z Podcast · 5 min instead of 59 min · Oct 14, 2026

Why distillation explains less of China's AI progress than assumed

Nathan Lambert

Co-founder & executive director, Trillium Labs

Led post-training for Ai2's open Olmo and Tülu models, wrote the RLHF Book and writes the Interconnects newsletter.

  • Erik TorenbergHost · General partner, Andreessen Horowitz

    Helped build Product Hunt; founded On Deck, Village Global and Turpentine.

About this segment

Lambert says Chinese labs match US talent but lack compute, and treats distillation as a way to buy compute. He argues the open literature shows distillation is hard to apply to modern RL-heavy post-training.

From “Can Open Models Keep Up With OpenAI and Anthropic? | Nathan Lambert”, Oct 14, 2026. Watch the full episode

“And distillation is a way to effectively spend more money to get what is compute equivalent. Like outputs from a language model are a function of compute.”
Nathan Lambertat 9:37 · a16z Podcast9:37

Takeaways

  1. An average OpenAI researcher can reportedly access about 5,000 H100 equivalents
  2. Distillation is a way to buy compute: model outputs are a function of compute
  3. Open labs struggle to gain from SFT on top reasoning traces, so the effect is months
Watch 4:44 from 8:53

In this episode

Full episode

  1. 8:53Why distillation explains less of China's AI progress than assumed4:44

  2. 24:51Frontier gains outside math and code come from buying data4:40Up next
  3. 54:55Scaling laws argue against a fast takeoff, but cyber risk is real3:41