Top of the Leaderboard, Half the Tasks Failed: Reading Runway’s Open-Source Praxis-1
先看那个数字:46.2%。在 NVIDIA 的 RoboLab-120 榜单上,Runway 开源的世界动作模型 Praxis-1 拿到了目前的最高成功率,可换个角度说,1,200 集任务里有一半以上是失败的。同一张榜上,FLUX 3 Action 是 42.9%,Cosmos3 Nano 是 36.8%,π0.5 是 28.0%。Praxis-1 领先第二名 3.3 个百分点,领先 π0.5 18.2 个点。”第一”和”能用”是两回事:抓取、放置、避障、多物体干扰,每一环都会掉分,榜首离产线可用还有一段距离。
Start with the number: 46.2%. On NVIDIA’s RoboLab-120 leaderboard, Runway’s open-source world action model Praxis-1 posts the highest success rate so far, yet put the other way, more than half of the 1,200 episodes failed. On the same board, FLUX 3 Action scores 42.9%, Cosmos3 Nano 36.8% and π0.5 28.0%. Praxis-1 leads second place by 3.3 points and π0.5 by 18.2. “First” and “usable” are different things: grasping, placing, obstacle avoidance and distractor objects each cost points, and the top of a leaderboard is still some distance from a production line.
榜单上的几个对手,类别其实并不相同。Praxis-1 是同时输出视频预测和动作的世界动作模型;π0.5 是视觉语言动作模型,也就是 VLA;FLUX 3 Action 和 Cosmos3 Nano 更接近生成式或世界模型一类。官方的原话是”超过所有开放权重 VLA”,这句话成立,但不等于超过所有开放权重模型,两个范围差得不小。另外还有一层关系值得知道:榜单由 NVIDIA 发布,Runway 是 Cosmos Coalition 的创始成员,NVIDIA 还为 Praxis-1 写了后训练教程。这不代表榜单不公,但读者最好心里有数。
The rivals on the board are not all the same kind of model. Praxis-1 is a world action model that predicts video and actions together; π0.5 is a vision-language-action model, or VLA; FLUX 3 Action and Cosmos3 Nano sit closer to generative or world-model approaches. The official wording is “beats all open-weight VLAs,” which holds, but it does not mean beating every open-weight model, and the two scopes differ considerably. One more relationship is worth knowing: the leaderboard comes from NVIDIA, Runway is a founding member of the Cosmos Coalition, and NVIDIA wrote a post-training tutorial for Praxis-1. That does not make the ranking unfair, but readers should have it in mind.
架构方面,官方披露得很少:总参数 270 亿,结构是混合专家扩散 Transformer,输出视频与动作。激活参数、上下文长度、动作空间维度,官方都没有公布。对 MoE 来说,真正决定算力的是激活参数,总参数只决定显存。官方称它开箱即可运行在 RTX 5090 和 Jetson Thor 上,部署清单还包括 H100、B200 和 RTX PRO 6000。能跑到机器人本体的那块板子上,说明激活参数大概不会太大,但这只是推断,不是数据。估算部署成本时,别按 27B 稠密模型来算。
On architecture, the vendor discloses little: 27 billion total parameters, a mixture-of-experts diffusion transformer, outputting video and actions. Active parameters, context length and action-space dimensions are all unpublished. For an MoE, compute is set by active parameters, while total parameters only set memory. The vendor says it runs out of the box on an RTX 5090 and a Jetson Thor, and its deployment list also includes H100, B200 and RTX PRO 6000. Running on the board a robot carries suggests the active count is probably modest, but that is inference, not data. When estimating deployment cost, do not budget it like a 27B dense model.
我觉得比榜单更有价值的,是”一个检查点适配多种机器人”。三个发布合作方用的是三种完全不同的本体:Noble Machines 的腿式人形、Standard Bots 的六轴工业臂、Ultra 的双臂仓储机器人,靠轻微微调适配。更硬的数字来自 SimplerEnv:模型在 Google 的 RT-1 机器人上微调,这台机器人的数据从未进入训练集,成功率达到 80.0%,RT-1 原策略是 52.4%。对比的是 2022 年的基线,所以进步看起来偏大,但它说明了模型可以迁移到没见过的本体上,而且只需少量微调。
What I find more valuable than the leaderboard is “one checkpoint, many robots.” The three launch partners run three very different bodies: Noble Machines’ legged humanoid, Standard Bots’ six-axis industrial arm and Ultra’s dual-arm warehouse robot, adapted with light fine-tuning. A harder number comes from SimplerEnv: fine-tuned on Google’s RT-1 robot, whose data never entered the training set, the model reaches 80.0% success, against 52.4% for the original RT-1 policy. The baseline dates from 2022, so the gain looks larger than it is, but it shows the model can transfer to an unseen body with only light fine-tuning.
背后还有一条训练结论:官方研究称,网页视频和遥操作机器人视频作为预训练数据,效果相当。如果这条成立,瓶颈就从”弄不到机器人数据”变成”能吸收多少通用视频”。机器人数据昂贵,网页视频便宜,预训练的成本结构会因此改变。对做机器人策略的团队,这一条比 46.2% 更值得关注,不过它目前只是官方研究的结论,还需要更多人复现。
Behind this sits a training finding: the vendor’s research says web video and teleoperated robot video work about equally well as pretraining data. If that holds, the bottleneck shifts from “we cannot get robot data” to “how much general video can the model absorb.” Robot data is expensive and web video is cheap, so the cost structure of pretraining would change. For teams building robot policies, this matters more than 46.2%, though for now it is the vendor’s own research conclusion and needs more replication.
再说许可证。权重采用 Apache 2.0:可以商用,可以修改,修改后分发也不必开源自己的改动,没有地域限制,没有营收门槛。Runway 放出的不只是权重,还有推理代码和微调代码,全量微调与 LoRA 都支持。渠道也铺得比较全:GitHub 的 runwayml/praxis-1-oss,Hugging Face 的 runwayml/praxis-1,原生支持 LeRobot,推理引擎有 vLLM-Omni,不想自己部署可以用托管的 Reactor,另有 Runway Dev 在线试用,以及 NVIDIA 的 NeMo Automodel 后训练教程。官方给出的开放理由很直白:物理 AI 的领先既取决于软硬件投入,也取决于开放程度,而物理场景目前还缺少这种开放。
Now the license. The weights are under Apache 2.0: commercial use is allowed, modification is allowed, distributing your changes does not require open-sourcing them, and there are no regional limits or revenue thresholds. Runway released more than weights: inference and fine-tuning code come too, with both full fine-tuning and LoRA supported. The channels are fairly complete: GitHub at runwayml/praxis-1-oss, Hugging Face at runwayml/praxis-1, native LeRobot support, vLLM-Omni as an inference engine, the hosted Reactor if you would rather not deploy it yourself, Runway Dev for trying it online, and NVIDIA’s NeMo Automodel post-training tutorial. The stated reason is blunt: leadership in physical AI depends on hardware and software investment and on openness, and physical settings still lack that openness.
最后是几处读的时候要核对的地方。第一,46.2% 既是第一,也不到一半,只写”登顶”容易让人以为已经可用。第二,领先第二名只有 3.3 个点,对 π0.5 的 18.2 个点是跨类别比较。第三,榜单方与厂商有合作关系。第四,激活参数没给,27B 只是总数。第五,80.0% 对 52.4% 是微调之后对旧基线,不是零样本。第六,模型 9 月就已发布,这次是开放权重,写成”新一代模型发布”会误导。我的判断是,这次最值得记的不是 46.2%,而是干净的 Apache 2.0:商用、微调代码、推理代码都在。真要落地,先在自己的机器人和任务上跑一遍,那个成功率才有意义。你更看重商用许可,还是成功率?
Finally, a few points to check while reading. One: 46.2% is first place and also under half, and writing only “tops the board” suggests it is already usable. Two: the lead over second place is just 3.3 points, and the 18.2 over π0.5 compares across categories. Three: the leaderboard’s owner has ties to the vendor. Four: active parameters are not given, so 27B is only a total. Five: 80.0% against 52.4% is after fine-tuning, against an old baseline, not zero-shot. Six: the model was announced in September and this release is the open weights, so calling it a new-generation launch misleads. My verdict is that the thing to remember is not 46.2% but the clean Apache 2.0, with commercial use, fine-tuning code and inference code all included. To deploy it, first run it on your own robot and tasks, because only that success rate means anything. Which do you value more, the commercial license or the success rate?