On November 12, 2026, researchers from Shanghai Jiao Tong University, Tsinghua University, and Sharpa Robotics published an arXiv preprint introducing SALA (Semantically Aligned Latent Actions), an embodied foundation model framework designed to scale vision-language-action (VLA) robot policies across massive unlabelled human video and diverse robotic embodiments without requiring paired demonstrations.
What happened: Announcing SALA and dual-axis scaling
In their preprint titled SALA: Semantically Aligned Latent Actions Enable Dual-Axis Scaling of Embodied Foundation Models (arXiv:2611.17110), authors Jiachun Jin, Zetong Zhou, Haonan Wang, Xiao Yang, Jun Zhu, and corresponding author Zhijie Deng address a primary bottleneck in embodied artificial intelligence: the scarcity and steep expense of physical robot teleoperation data. While physical teleoperation requires specialized hardware rigs and human operators collecting data minute by minute, vast repositories of passive egocentric human video already exist in digital archives. Prior attempts to train robot policies on video have relied on world-action models (WAMs) or latent action models (LAMs), both of which struggle to bridge the visual and mechanical gap between human bodies and multi-jointed robots.
The research team demonstrated that SALA bridges this gap by enforcing semantic correspondence rather than pure visual reconstruction. By mapping both human demonstration videos and robot teleoperation traces into a shared, semantically structured latent action space, the framework achieves dual-axis scaling. Along the pre-training axis, scaling the volume of general human video consistently reduces action-prediction errors on downstream robot tasks. Along the post-training axis, incorporating task-specific human video demonstrations significantly improves physical task execution rates on real hardware without collecting additional teleoperated robot trajectories.
Key details: The SALA architecture, latent action tokenizer, and benchmark metrics
The SALA architecture separates cross-embodiment feature learning into a modular pipeline combining a latent action tokenizer, a visual-language backbone, and a dedicated flow-matching action expert. The tokenizer encodes video transitions across observation pairs into 8 discrete latent tokens using Soft Vector Quantization (SoftVQ) with a codebook of 256 entries. To prevent representations from clustering based on background textures or lighting, the visual decoder reconstructs frozen DINOv2 visual features rather than raw pixels. Simultaneously, atomic-action textual descriptions embedded via frozen text encoders (CLIP ViT-B/32 in simulation and Qwen3-Embedding-4B in physical experiments) define soft relational neighborhoods. These semantic neighborhoods guide human latent action tokens toward corresponding robot latents, while a robot action decoder grounds the space in physical control dynamics.
The downstream policy couples a 4-billion-parameter vision-language model (Qwen3-VL-4B-Instruct) with a 600-million-parameter Qwen3-style action expert transformer that attends jointly across all attention layers. Physical execution was validated on a Sharpa North wheeled humanoid robot equipped with dual 22-degree-of-freedom Sharpa Wave dexterous hands, controlling 63 joint-space dimensions. In simulated trials on the RoboTwin 2.0 dual-arm bimanual manipulation benchmark across 50 tasks, SALA achieved a 64.4% average success rate, substantially outperforming leading world-action models such as FastWAM (41.3%), LingBot-VA (57.1%), and OpenWAM (45.4%), as well as earlier latent action models like UniVLA (27.1%) and RotVLA (51.2%).
The following table summarizes comparative empirical results reported across simulated environments and physical humanoid manipulation tasks:
| Evaluation Environment | Model / Architecture | Clean Scene Success (%) | Random Scene Success (%) | Average Success Rate (%) |
|---|---|---|---|---|
| RoboTwin 2.0 Simulation | UniVLA (LAM baseline) | 48.6% | 5.6% | 27.1% |
| RoboTwin 2.0 Simulation | FastWAM (World-Action Model) | 79.4% | 3.2% | 41.3% |
| RoboTwin 2.0 Simulation | LingBot-VA (WAM baseline) | 75.8% | 38.4% | 57.1% |
| RoboTwin 2.0 Simulation | RotVLA (Rotational Latent VLA) | 74.4% | 28.0% | 51.2% |
| RoboTwin 2.0 Simulation | SALA (Proposed Framework) | 80.2% | 48.6% | 64.4% |
| Real-World Sharpa North (4 Tasks) | Vanilla VLA (Teleoperation only) | N/A | N/A | 27.5% |
| Real-World Sharpa North (4 Tasks) | GR00T N1.7 (NVIDIA Baseline) | N/A | N/A | 41.3% |
| Real-World Sharpa North (4 Tasks) | SALA (Human Pre- and Post-Training) | N/A | N/A | 72.5% |
In real-world deployment across four challenging manipulation tasks (gear assembly, bimanual pouring, flower insertion, and spoon scooping), SALA achieved a 72.5% average success rate across 20 trials per task. This represented a 45.0 percentage point improvement over a vanilla VLA trained purely on teleoperation data (27.5%) and a 31.25 percentage point advantage over NVIDIA’s GR00T N1.7 humanoid policy baseline (41.25%). Task breakdowns showed SALA securing 75% on gear assembly, 80% on bimanual pouring, 55% on flower insertion, and 80% on spoon scooping.
Background and context: The data bottleneck in vision-language-action models
The rise of embodied foundation models has mirrored the scaling of large language models, yet physical robotics has continually collided with a severe physical bottleneck. While web-scale text and image datasets encompass billions of tokens, robotics datasets such as the Open X-Embodiment dataset remain bounded by the sheer labor hours needed to steer robot arms or teleoperate manipulators. Training generalist policies that generalize across varied environments typically demands tens of thousands of physical trajectory demonstrations.
To overcome this limitation, researchers have sought methods to exploit passive video data. World-action models attempt to predict future video frames pixel-by-pixel, but generative video synthesis incurs high computational overhead and risks physical hallucinations. Earlier latent action models attempted to infer unobserved control commands directly from consecutive frames, but unsupervised clustering invariably grouped actions by visual backgrounds rather than physical kinematics. When an algorithm cannot differentiate between a human stirring a cup and a robot stirring a cup because the room appearance differs, cross-embodiment transfer fails. SALA represents a pivot toward semantic alignment, using natural language descriptions of atomic actions as a grounding bridge so that identical tasks map to compatible latent actions regardless of embodiment.
Industry and academic reactions
The publication of SALA has sparked discussion across the robotics and artificial intelligence research communities. On academic forums and open peer repositories like Papers.cool, roboticists noted that demonstrating dual-axis scaling provides empirical evidence that video pre-training directly enhances high-DoF dexterous manipulation. Researchers highlighted the elegance of the SoftVQ quantizer design and the stop-gradient operator applied to robot latents, which prevents noisy human data from distorting calibrated physical control spaces.
Commercial robotics developers have also followed the results closely, particularly in comparison to industry initiatives like NVIDIA Project GR00T. While commercial labs often invest heavily in physical teleoperation fleets and simulation farms, SALA indicates that academic research teams can achieve competitive generalist performance by combining modest robot teleoperation datasets with widely available egocentric human video corpora like Ego4D. The project team has released supplementary videos and demonstration materials on their official project website.
What it means for researchers, universities and robotics laboratories
The findings presented in the SALA preprint carry direct implications for university robotics departments, research computing facilities, and academic grant administration:
- Democratizing robot policy training: Academic robotics laboratories often lack the multimillion-dollar budgets required to maintain fleets of teleoperation rigs and human operators. By demonstrating that 2,678 hours of action-free human video combined with 888 hours of robot teleoperation can train a robust 63-DoF humanoid controller, SALA lowers the data collection barrier.
- Data management and cataloging standards: University research offices must adapt their AI and ML research outputs protocols to accommodate multimodal embodied datasets. Managing synchronized video, proprioceptive trajectories, and linguistic annotations requires structured metadata schemas.
- Experimental rigor and reproducibility: As embodied foundation models expand into complex bimanual manipulation, establishing a reproducible AI experiment workflow becomes essential. Preprints must clearly document seed variances, camera calibration angles, lighting conditions, and specific failure modes to support independent replication.
- Simulated vs physical benchmarking: Standardized benchmarks like the RoboTwin 2.0 benchmark provide critical initial validation, yet robotics researchers must balance virtual testing with physical validation. Implementing rigorous testing protocols helps avoid the pitfalls of a purely synthetic benchmark, where policies excel in simulation but fail under real-world friction.
- Governance and model accountability: Institutional governance bodies overseeing AI centers must ensure compliance with good machine learning practice, ensuring that embodied models operating high-torque robots incorporate hardware kill-switches and boundary verification during training. In addition, institutions tracking long-horizon autonomous capabilities can reference evaluation frameworks such as the METR time horizon framework to assess operational safety.
What happens next
Following the release of the preprint on November 12, 2026, the authors from Shanghai Jiao Tong University and Tsinghua University plan to release their model checkpoints, tokenization codebase, and pre-training recipes under an open-source research license. Subsequent phases of the project will focus on expanding the latent action vocabulary to handle additional dexterous tool-use tasks and evaluating cross-embodiment transfer across quadrupeds and mobile manipulators. Academic peer review is expected to proceed ahead of major robotics conferences in 2027, including ICRA and RSS, while research teams worldwide begin applying semantic latent action alignment to expand their laboratory datasets.
Sources
- Jin, J., Zhou, Z., Wang, H., Yang, X., Zhu, J., and Deng, Z. (2026). SALA: Semantically Aligned Latent Actions Enable Dual-Axis Scaling of Embodied Foundation Models. arXiv preprint. https://arxiv.org/abs/2611.17110 (Accessed 2026-11-16)
- SALA Project Team. (2026). SALA: Project Page and Demonstration Benchmarks. Shanghai Jiao Tong University, Tsinghua University, and Sharpa Robotics. https://sala-vla.github.io/ (Accessed 2026-11-16)
- Papers.cool. (2026). SALA: Semantically Aligned Latent Actions Research Summary and Citation Tracker. https://papers.cool/arxiv/2611.17110 (Accessed 2026-11-16)
- Chen, Y., et al. (2025). RoboTwin 2.0: A Scalable Benchmark for Dual-Arm Robot Manipulation. https://robotwin-benchmark.github.io/ (Accessed 2026-11-16)
- NVIDIA Robotics. (2025). Project GR00T: Generalist Robot Foundation Models for Humanoids. NVIDIA Developer Portal. https://developer.nvidia.com/project-gr00t (Accessed 2026-11-16)
- Open X-Embodiment Collaboration. (2023). Open X-Embodiment: Robotic Learning Datasets and RT-X Models. https://robotics-transformer-x.github.io/ (Accessed 2026-11-16)







