A Top SI Video Company Just Gave Away a Robot Brain That Learned From Watching Videos

The hard part of building a useful robot has never really been the arm or the motor. It’s the brain. Teaching a machine to pick up a mug, flip it over and set it on a tray takes mountains of training data, and the usual way to get that data is painfully slow: a person steers the robot by remote control, over and over, while cameras record every move.

Runway thinks there’s a shortcut, and on Wednesday it gave that shortcut away. The New York company, best known for SI tools that make video, released Praxis-1, a model that controls robots, with its full weights open to anyone under the Apache 2.0 license. That means a startup, a university lab or a guy in his garage can download it, change it and build a commercial product on top of it without paying Runway a dime.

Runway first showed off Praxis-1 at the end of September and promised to release it openly once partners had kicked the tires. It kept that promise. The weights, the code to run it and the code to retrain it are posted on Hugging Face and GitHub.

A robot that learned by watching

The big idea is simple enough to explain at the dinner table. People film and upload a staggering amount of everyday life: cooking, packing boxes, fixing things, cleaning up. Runway already trains huge models on that kind of video to generate realistic clips. Praxis-1 points the same knowledge at a robot.

“Most robot policies are bottlenecked by robot data, which is scarce and expensive to collect,” Runway’s chief technology officer, Kamil Sindi, told The Robot Report when the model was announced. “Praxis-1 learns mostly from third-person video, built on the same large-scale pretraining behind Runway’s world models, so it already understands how objects behave and how tasks unfold.”

Under the hood, Praxis-1 is a 27-billion-parameter model that does two things at once. It imagines what the scene in front of the robot will look like over the next moment, and it works out the motion that would get it there. Runway’s research page says each call returns about 1.6 seconds of movement. The robot carries out that chunk, looks again through its cameras, and asks for the next one.

Runway also made a claim that, if it holds up, matters a lot. In its own tests, a robot policy trained first on ordinary web video did just about as well as one trained on expensive remote-control robot footage. One placed objects with a final error of 16.1 centimeters, the other 16.0, according to Runway’s comparison. If plain video is nearly as good a teacher, the supply of training material goes from scarce to nearly endless.

How it scored, and what that score means

Runway says Praxis-1 is the top performer of its kind on RoboLab-120, a simulation test built by NVIDIA with 120 tabletop chores for a single robot arm. Each task gets 10 tries. Praxis-1 finished 555 of the 1,200 attempts, or 46.2%.

That beat FLUX 3 Action at 42.9%, NVIDIA’s Cosmos3 Nano at 36.8% and π0.5 at 28.0%, the numbers Runway published. Runway says it’s the best world action model on the board, ahead of every open-weight rival.

Keep two things in mind before anyone starts calling this robot genius. First, these are Runway’s numbers, posted on Runway’s own launch page, from a computer simulation, not a warehouse floor. Second, 46% means the best open model still fails more often than it succeeds on these chores. Robots aren’t folding your laundry next month.

The more practical test is how fast the model picks up a body it has never seen. Runway retrained Praxis-1 on Google’s older RT-1 robot, which wasn’t in its training data at all. It hit 80.0% on a standard simulated test, compared with 52.4% for the robot’s original control software, the company says.

Three companies already running it

Before Wednesday, Praxis-1 was only in the hands of a few early partners. Runway says Noble Machines, Standard Bots and Ultra have been running it on their own machines: a legged humanoid, a six-axis factory arm and a two-armed warehouse robot. Very different bodies, one brain.

“A big takeaway so far is that a single model can adapt across very different embodiments, from bimanual arms to humanoids, with light fine-tuning,” Sindi told The Robot Report in October.

That’s the business case. A robot maker doesn’t have to build a brain from scratch. It grabs Praxis-1 and teaches it its own arm, cameras and jobs. Runway says that tuning can be done with a lightweight method that doesn’t retrain the whole model, and the model runs on NVIDIA hardware all the way from big data center chips down to the Jetson Thor computer that sits inside a robot.

Why give it away?

Runway’s answer is partly patriotic and partly strategic. “U.S. leadership in physical AI will depend on investment in both hardware and software,” the company wrote, “and on a level of interoperability and openness from American models that physical use cases don’t have yet.”

Read that against the backdrop of the past two years, when Chinese labs pushed out a steady stream of free, capable SI models and won a lot of developers that way. Robots are the next battlefield. Runway is betting that if American robot builders have a strong free model to start from, they’ll build on an American foundation instead of a foreign one.

There’s a commercial angle too. Runway says the release builds on its role as a founding member of the Cosmos Coalition, an effort with NVIDIA to push open models for robots. It lets developers try Praxis-1 on its own developer platform and invites companies planning real deployments to call its robotics team. Give away the engine, and the customers come to you for the rest.

Runway also has company. NVIDIA unveiled its own GR00T 2 robot model on Nov. 9, and Google DeepMind rolled out a new Gemini Robotics model the same week. The race to give robots a general-purpose brain is crowded and getting faster. What sets Runway apart this week is that anyone can take its model home today, open it up and see exactly what it does.