Robots That Predict the Future: Dyna-2 & Physical AI — Jason Ma, Dyna Robotics

Summary · 6 min read instead of 1 h 21 m

Jason Ma is cofounder and chief scientist of Dyna Robotics, which he started at the end of 2024 and which builds its own robots and models. He studied reinforcement learning for robotics in his PhD at Penn, then worked at NVIDIA, where his Eureka project had GPT-4 write reward functions for simulated pen spinning, and at Google DeepMind on using Gemini for robotics. Dyna's robots fold napkins at Din Tai Fung and towels for Best Western.

Ma argues that robots should predict the future: Dyna-2, which he calls the first world-action model trained on more than 1 million hours of data, predicts future video and uses it to choose actions, an approach he says carries fewer human biases than models that predict only actions. Because robotics has no internet of data, Dyna pretrains on egocentric human video, teaches each task with a couple of hours of robot data or less, and is exploring synthetic environments as a new dimension of scaling. He treats robotics as a whole-system problem, so Dyna builds its own wheeled semi-humanoid, its own inference stack and an agentic harness that runs hour-long jobs like laundry. He separates pilots from real deployments, citing Din Tai Fung, where Dyna's robots fold thousands of napkins a day across many US locations and which has signed a much bigger contract. Ma names dexterous hands that overheat and data that misses senses like touch as the main bottlenecks.

Minute by minute

1. 2:04

   World-action models predict, then act. Ma says Dyna-2, released last month, is the first world-action model trained on more than 1 million hours of data. From recent camera video, joint angles and a language instruction, it predicts future video and uses that to produce actions, which he likens to chain of thought. It is a diffusion transformer: the transformer takes in the most data, and diffusion, or its simpler-to-tune variant flow matching, refines noise into smooth, precise motion.

2. 8:35

   Pretrain on humans, adapt on robots. Ma says Dyna trains one general base model on as much data as it can find, over a million hours of human data, then teaches each task with a few demonstrations. With no internet for robotics, he says what any one company can collect is "nowhere near sufficient." Egocentric human video gives a strong prior on how the world moves, but the human-robot gap means costlier robot data is still needed for low-level dexterity, and he calls a million hours just the beginning.

3. 15:38

   Synthetic data is the next frontier. Ma calls synthetic environments one of the next frontiers and a new dimension of scaling. When his Eureka paper had GPT-4 generate simulated environments and reward functions, models lacked good vision and 3D understanding; now, he says, frontier models' vision puts the technology at a tipping point, and Dyna is exploring using them to generate environments it cares about. Its world-action model can also generate its own training data, which he calls a self-fulfilling flywheel.

4. 18:43

   Deployed models must run on the robot. Ma says robots can't tolerate the seconds of latency people accept from chatbots: a robot can't "hold a knife and not do anything for three seconds," and gravity doesn't wait. So the deployed model must be small enough to serve on the robot itself, though the model trained first can be large and then distilled, as in language models. Dyna builds its own inference stack for every model it deploys.

5. 21:36

   Build what matters, buy what's mature. Ma says Dyna buys mature parts such as lidars and cameras but is building its own actuators, which must be easy to model in simulation, and sized the robot against real people so human data transfers. He says no US company truly builds robots from scratch in the US, since raw materials come from elsewhere, and that hardware is under-emphasized. Having mostly built models at DeepMind and NVIDIA, he says he saw the ceiling of not controlling the hardware.

6. 26:58

   A wheeled semi-humanoid with a waist. Ma calls the robot a semi-humanoid: a humanoid upper body with four degrees of freedom in the waist, on wheels, because legs use more energy, are less stable and make safety very hard to guarantee. People move their torsos constantly without thinking, he says, and a body that can't do that transfers less from human data. The new robot had little data of its own but learned tasks quickly, which he credits to pretraining on human data before the hardware was ready.

7. 31:27

   Quick to teach, limited by hardware. Ma says a new task takes a little teleoperated data, a couple of hours or just tens of minutes; when the model is 70% or 80% there, the team studies failures and collects more, which he calls "the robotics version of reinforcement learning from human feedback." The laundry job already runs about an hour, but he says broader tasks are held back by dexterous hands that overheat and by vision-based data that misses senses like touch and hearing.

8. 37:58

   Din Tai Fung scaled past a pilot. Ma says Dyna folds towels for Best Western, but its biggest scaled customer is Din Tai Fung, where its robots fold thousands of napkins a day to a precise standard across many US locations. He argues the hard part is not a pilot but a customer choosing to deploy across many sites, which to his knowledge hasn't happened for similar general-purpose robots. Din Tai Fung signed a much bigger contract that adds cutting food, and he says one set of weights keeps folds consistent across sites.

9. 43:00

   Scale the right data, iterate hardware. Ma's bitter lesson for robotics is that scaling works as in other fields: keep algorithms simple instead of adding hand-built priors, and scale the right data, which depends on picking the right sensors. He expects a human form to win eventually but says slow, costly hardware iteration favors deploying a semi-humanoid now over spending five to ten years on a humanoid, noting Optimus has scrapped designs. He says the model handles about 100 continuous action dimensions smoothly.

10. 48:02

    An agentic harness for hour-long jobs. Ma says long physical jobs need high-level decisions, such as doing other work while the washer runs, then moving towels to the dryer the moment it finishes. Learning that in the low-level action model would be extremely inefficient, so Dyna wraps an agentic harness around its world-action model to compose skills, much as language models moved from autocomplete to agents. He argues the value lies in workflows that run unsupervised for hours, not 30-second tasks.

11. 50:56

    AI-native, aiming for hundreds of robots. Ma calls Dyna an extremely AI-native company: when GPT-6 came out it held a company-wide hackathon that same week, and many features built there are already in production. What stood out to him was GPT-6's ability to construct virtual environments for simulation. Dyna has deployed tens of robots, much of the work in laundry and restaurants, and aims for hundreds next year, treating each task as a wedge that validates its general platform.

12. 53:56

    Predict raw video, not abstractions. Ma says some companies bet on scaling robot data or intermediate collection devices, while Dyna owns its hardware, scales human data and deploys early to find true bottlenecks; he notes a graveyard of robotics companies with shiny robots. He argues robotics is about predicting the future, so a model should predict raw camera input; VLAs predict only actions or abstractions that encode human bias. He could be wrong, he says, if a vision-plus-touch model proves 100x more data efficient.

13. 1:01:10

    From AlphaGo to founding Dyna. Ma grew up playing Go, and AlphaGo, released when he was a high school senior, drew him to self-improving systems and to reinforcement learning, which he applied to robots in his PhD at Penn. At NVIDIA his Eureka project had a coding model write reward functions so a simulated hand could learn pen spinning; at Google DeepMind he worked on Gemini for robotics. Wanting real use rather than "grading your own homework," he started Dyna at the end of 2024.

14. 1:06:06

    Research driven by real deployments. Ma calls Dyna a research-plus-product company, like OpenAI shipping research into ChatGPT: deployments set the problems, such as generalization, whole-body control and dexterity, and engineers invent tasks to test them. About 10 people work on models, with more on hardware. He named Dyna after Richard Sutton's Dyna paper and the Ultraman of his birth year, and says robotics is still very demo-oriented; Dyna wants millions of robots deployed.