OpenAI’s Chief Scientist on Continual Learning Hype, RL Beyond Code, & Future Alignment Directions

Summary · 5 min read instead of 59 min

Jakub Pachocki is chief scientist of OpenAI. He joined the company at the start of 2017, when he says it was an academic lab, and has stayed through its move to scaling and the launch of ChatGPT. He did a PhD in a field that overlaps with one of the First Proof problems.

Pachocki says OpenAI still targets a research-intern-level system by September and a fully automated researcher by March 2028, citing the growth of Codex and math results as signs it is on track. He expects reinforcement learning to extend to harder tasks through longer horizons, and he sees in-context learning and general harnesses reducing the need for custom RL pipelines. He argues continual learning is what scaling GPT-style models already pursues. He backs chain-of-thought monitoring as an alignment tool and treats value alignment as a question of generalization. He says the concentration of power in highly automated organizations is a societal problem without obvious solutions.

Minute by minute

1. 1:49

   Research intern timeline still on track. Pachocki says coding tools grew explosively and OpenAI now uses Codex for most of its coding. He sees this and recent math and physics results as signs the September research-intern goal is on track. He says the intern differs from a full automated researcher in how long it works autonomously and how specific its tasks must be, and he does not expect a system that can be told to solve alignment this year.

2. 5:17

   Math benchmarks give way to real-world use. Pachocki says math served as a measurable North Star because solutions are easy to check and problems can be arbitrarily hard. Models have now reached IMO gold level and are moving into research-level mathematics. He says OpenAI is shifting its focus to making models useful for its own research, the economy and applied sciences, because it believes the models are capable enough to change how things are done.

3. 8:22

   Hard-to-verify tasks are long-horizon tasks. Pachocki expects RL gains to spread beyond math and code. He argues that hard-to-evaluate tasks and long-horizon tasks share the same difficulty, since even a clearly specified year-long project leaves the first day open-ended. He says models must learn to judge partial progress, and that pre-training alone would lengthen horizons over time. He also says OpenAI is buying a lot of compute and still believes the bitter lesson.

4. 12:28

   In-context learning may beat custom RL. Asked whether builders should run their own RL, Pachocki says RL is data-efficient but in-context learning is more so, and he expects it to improve. He is unsure that replicating current RL pipelines is the right path. He says harness implementation should not be a limitation for long, expects more general harnesses, and says Codex already works well beyond coding. Long term, he says AI should meet people where they are.

5. 17:31

   Research org now depends on near-term model quality. Pachocki says the immediate quality of OpenAI's models is about to determine the pace of its research, which requires rewiring intuitions about running a research organization. Normally he would not focus on short-term quality. On compute, OpenAI explicitly budgets a large share to the most scalable methods, even when that is not the most efficient allocation, so that important work is not diluted across many projects.

6. 21:09

   Research has decoupled from product strategy. On Anthropic's early lead in coding products, Pachocki says success comes from focusing the product on the next application of the technology. OpenAI treated coding products as secondary to its main priorities. He says ChatGPT fits OpenAI's vision but does not represent everything the technology enables, so research has increasingly decoupled from short-term product strategy. He expects Codex users to trust models with more autonomy, and models to work for a couple of days soon.

7. 24:16

   Continual learning is what scaling already builds. Pachocki says he is a little confused by the continual-learning movement. In his view the excitement around GPT models was that they can learn in context, and RL teaches them to do so more efficiently. He agrees continual learning is the goal but says it is not ignored or off OpenAI's path. He thinks scaling pre-training and RL has made the most progress and that more will come from scale.

8. 28:58

   First Proof results raised his urgency. Pachocki describes OpenAI's work on the First Proof challenge, where mathematicians released unpublished problems on a week-long deadline. James Lee prompted a new model by hand and it solved problems, including one from Pachocki's PhD field, in about an hour. He says this raised his sense of urgency. On the pattern-matching criticism, he points to AlphaGo and OpenAI's Dota bot as systems that found new strategies.

9. 34:12

   AI science may pair general and specialized models. Pachocki says he would build around the capabilities of the technology, not its limitations. If models can design many experiments, building labs for them makes sense, and humans will work with AI scientists in a loop. He says language models are probably not the most efficient route for tasks like protein folding or Go models, so separate models may be trained for them and paired with a general researcher model.

10. 37:36

    Chain-of-thought monitoring aids alignment. Pachocki says reasoning models are not supervised directly on their chains of thought, so those chains may reveal motivations the final output hides. He compares this to mechanistic interpretability, with the advantage that chains of thought are in English. He explains that OpenAI hid them in the o1 preview to avoid pressure to train them. He calls the approach one tool, not a full solution, and cites scheming research that relies on it.

11. 44:30

    Value alignment is a generalization problem. Pachocki says the long-term challenge is how models behave in situations unlike their training, so value alignment is about what values a model falls back on. OpenAI is investing in how generalization relates to pre-training data. His belief in a research path to a very good outcome has grown, but his timelines to very capable models have shortened. He says the industry must be ready to take tradeoffs and possibly slow development.

12. 47:56

    OpenAI's phases and the current urgency. Pachocki says he keeps his focus on high-quality experiments and honesty about results, and that fundamentals have not changed, though urgency has. He describes phases: an academic lab when he joined in 2017, a shift to buying compute and scaling, then ChatGPT, which surprised him as a text-first success. He says OpenAI now sees itself deploying models that are economically transformative.

13. 54:35

    Automation concentrates power in few hands. Pachocki says automating much intellectual work brings big problems, including jobs and concentration of wealth, and he has no obvious solutions. He says an automated lab or company could be controlled by a very small number of people, even without robots. He calls governance of such organizations a new question for society, and says alignment and monitorability challenges involve policymakers, not only AI researchers.

Published Apr 9, 2026.