Paper review · October 30, 2026
Sensorimotor Alignment: a Platonic offline metric that predicts robot policy success rates
Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success
The authors propose S², a training-free metric that measures how well a model's visual and language representations match the structure of robot motion trajectories. It correlates positively with task success rates without costly closed-loop evaluation.
Problem
Robot foundation models are evaluated with closed-loop rollouts in simulation or real environments, which is slow, noisy and expensive. Existing offline proxy metrics, such as VQA scores or video quality metrics, measure only passive perception or generation quality, so they do not reflect whether representations are organized for action.
Method
The work extends the Platonic Representation Hypothesis (PRH) to robotics, assuming that more capable models have sensory representations more closely aligned with motion structure. For each episode in robot datasets such as DROID and BridgeDataV2, one image frame and the instruction are fed into the model's encoder to obtain a sensory embedding. The end-effector position, rotation and gripper state over the next h steps serve as the motion representation. k-nearest neighbors are computed separately in sensory space (cosine distance) and motion space (DTW distance), and S² is defined as how much the two sets overlap (a CKNNA variant, k=10).
Results
- 12 VLAs: Pearson r=+0.717 between S² and SimplerEnv success rate (95% CI [+0.436, +0.912]).
- 8 raw VLM backbones: S² correlated with success rate after fine-tuning into VLAs at r=+0.681 (p=0.031). VQA scores showed almost no correlation, at r=−0.154.
- 6 WAMs gave r=+0.718 (p=0.054), and 5 real-robot policies (results reported by MolmoBot, 40 tasks) gave r=+0.831 (p=0.041). In a comparison of 3 WAMs, S² (r=+0.999) matched the success-rate ranking better than SSIM, PSNR and FID.
- Negative control: when bimanual WAMs were measured on single-arm data (DROID), the correlation fell from +0.999 to +0.246, showing that the metric responds to embodiment mismatch.
- Noting that fine-tuned VLA success rates correlate with S² computed on vision features rather than text features, the authors offer a representation-level explanation that VLAs rely more on visual information than on instructions.
Limitations
The number of models is small, 3–12 per family. The p-values are one-sided and not corrected for multiple comparisons, and the WAM correlation, at p=0.054, sits at the edge of conventional significance. The real-robot results reuse figures for 5 policies reported in another paper (MolmoBot). The results are correlational and do not show that S² causes success rates. The scope is limited to manipulation tasks.
Why it matters
It can serve as a low-cost pre-screening tool for deciding which models to evaluate first when closed-loop evaluation is expensive, such as when choosing a VLA backbone or comparing many checkpoints. It also offers a representation-level diagnostic view that success rates alone do not show.
Key terms
- Platonic Representation Hypothesis(PRH)
- The hypothesis that as models grow larger and more capable, representations from different modalities such as images and text converge toward a shared statistical structure of reality.
- VLA (Vision-Language-Action) model
- A model that takes images and language instructions as input and directly outputs robot actions. It is usually built by fine-tuning a VLM on robot data.
- WAM(World Action Model)
- A robot model built on a video generation model that both predicts future scenes and outputs actions.
- Mutual nearest-neighbor alignment
- A method that measures the structural similarity of two representations by how much each sample's k-nearest-neighbor sets overlap across the two spaces.
Original
Authors: Yunfei Xie, Howard H. Qian, Shiyi Lan, Kaiyu Hang, Chen Wei · Posted 2026-10-22 · License arXiv 비독점 배포 라이선스 · Based on the full text
This review is an AI-written summary of the paper. Please treat the original paper as the reference.