Written and maintained by CASRAI Editorial Board
Published
On this page
Open X-Embodiment is a collaboration that pooled robot-learning data from 60 existing datasets, covering 22 robot embodiments and more than one million real robot trajectories, into one common format, and released the RT-1-X and RT-2-X models trained on it. Researchers get the data as episodes in the RLDS episode format, loaded with TensorFlow Datasets or copied with gsutil, plus a public RT-1-X checkpoint with Colab and JAX examples. This guide explains what it is for, the access routes in the official repository, how to read the data and the model’s inputs and outputs, and what you must cite.
Ask CASRAI · free to tryQuestion about this guide? Ask CASRAI answers with linked sources. 2 free questions a day, no account.
What Open X-Embodiment is used for in research
The project page describes Open X-Embodiment as a dataset and model release for robotic learning, assembled by the Open X-Embodiment Collaboration, which the arXiv record lists as the collaboration plus 293 further authors (arXiv 2310.08864, first posted 13 October 2023, latest version v9 on 14 May 2025). According to the project page, the pooled data comes from 21 institutions and spans single arms, bimanual robots and quadrupeds. The abstract reports 527 skills and 160,266 tasks across the 22 embodiments.
The research question behind it is transfer: can a policy trained on data from many different robots perform better than one trained on a single robot’s data? The authors report positive transfer. On the project page, RT-1-X (the RT-1 architecture trained on the pooled data) is reported to outperform the original methods by about 50 percent in small-data domains, and RT-2-X (built on the RT-2 vision-language model, with robot actions represented as tokens) is reported to be about three times better than RT-2 on emergent-skill evaluations. Treat these as the authors’ own reported results and read the paper’s evaluation setup before comparing against them.
In practice researchers use the release in three ways: as pre-training or co-training data for robot policies, as a benchmark source with a shared episode format, and as a reference implementation for running an RT-1-X checkpoint offline on recorded episodes. For the wider method family see our guides to robot learning and imitation learning.
Access: the official sources
All access routes come from the official repository at github.com/google-deepmind/open_x_embodiment and the project page at robotics-transformer-x.github.io. The repository README states that the project aims to provide all open-sourced robotic data in the same unified format. Its README does not list a package-installation command or a dependency list, so use the environment your chosen notebook sets up, or the TensorFlow Datasets and Google Cloud gsutil tools the commands below rely on.
- Project page (robotics-transformer-x.github.io): links to the paper, the GitHub repository, the dataset spreadsheet, a Dataset Enrollment Form for contributors, and a contact address for the collaboration, [email protected].
- GitHub repository (google-deepmind/open_x_embodiment): the README, the Colab notebooks, and the model example code.
- Dataset spreadsheet: the repository links a Google Sheet that lists every dataset in the effort with its metadata and its citation.
The README states that this is not an official Google product. Licensing is split: all software is under the Apache License 2.0, and all other materials are under Creative Commons Attribution 4.0 International (CC-BY). For how attribution works under CC-BY, see our guide to Creative Commons licences for research data.
Step by step: load the data and run RT-1-X
The steps below follow the README. The commands are reproduced as the repository gives them; replace the placeholder with a dataset name from the dataset spreadsheet.
Route A: explore the data in the dataset Colab
- Open the repository’s
colabs/Open_X_Embodiment_Datasets.ipynbnotebook (the README links it as a self-contained Colab). - Run it to visualise a few episodes from each dataset and to see how batches of data are created for training and inference.
Route B: load a dataset with TensorFlow Datasets
- Choose the dataset name from the dataset spreadsheet.
- Load it with
tfds.load({dataset_name}), using the dataset name from the spreadsheet. - If loading fails with
DatasetNotFoundError, copy the dataset manually as described under troubleshooting, then repeat the load.
Route C: run the RT-1-X TensorFlow checkpoint on recorded episodes
- Open
colabs/Minimal_example_for_running_inference_using_RT_1_X_TF_using_tensorflow_datasets.ipynb. - Run it to load the checkpoint, run inference on offline episodes, and overlay the predicted actions on the ground-truth actions.
Route D: download the RT-1-X JAX checkpoint
- Copy the checkpoint with
gsutil -m cp -r gs://gdm-robotics-open-x-embodiment/open_x_embodiment_and_rt_x_oss/rt_1_x_jax . - Load it with the Flax checkpoint loader through the repository’s
models/rt1_inference_example.py.
The data table below summarises these four routes with the exact command or file each one uses.
Output and how to read it
How a dataset is organised
Each dataset is a sequence of episodes, and each episode is stored in the RLDS episode format that the README links to in the google-research/rlds repository. An episode is one recorded run of a robot attempting a task, made of the observations the robot saw and the actions it took. Because every contributed dataset uses this same structure, one data-loading and batching pipeline can read all of them, which is the practical benefit of the unified format.
What the RT-1-X checkpoint takes in and returns
The README gives the model’s interface exactly:
- Observation: one RGB image from the robot workspace camera, plus a task string describing what the robot should do. The task is communicated purely through that string.
- Rate: the README illustrates this with a model running at three hertz, so the latest image is fed in every 333 milliseconds to obtain the next action.
- Not used: the model currently does not take wrist-camera images, in-hand camera images or depth.
- Action: seven variables for the gripper movement (x, y, z, roll, pitch, yaw and gripper opening). Each variable represents the absolute value, the delta change to the dimension value, or the velocity of the dimension.
When you overlay predicted and ground-truth actions in the inference Colab, remember that the same seven slots can carry absolute, delta or velocity values depending on the dataset, so compare like with like before judging a prediction.
Reading the limits as a researcher
The interface above tells you what the released checkpoint cannot do. A study that depends on depth, wrist-camera views or multi-camera input cannot use the released RT-1-X checkpoint as it stands, and a claim about a robot-specific skill needs the underlying dataset’s own documentation, which the dataset spreadsheet points to. The reported gains in the paper are the authors’ evaluations; replicating them on your own robot or task is a separate piece of work. See our guide to making a dataset FAIR for what documentation reusable robot data should carry.
Troubleshooting
DatasetNotFoundError when calling tfds.load
If tfds.load({dataset_name}) fails with tensorflow_datasets.core.registered.DatasetNotFoundError: Dataset {dataset_name} not found., the README’s fix is to download the dataset manually with gsutil -m cp -r gs://gdm-robotics-open-x-embodiment/{dataset_name} ~/tensorflow_datasets/. After that, the regular tfds.load({dataset_name}) command works with the dataset.
Model results look wrong on your robot
Check first that your input matches the model’s interface: a single workspace RGB image and a task string, at roughly the rate the README describes, with the seven-variable action read as absolute, delta or velocity according to the dataset. The README does not describe adapting the checkpoint to other camera setups, so treat that as outside what the release documents.
Questions about a specific dataset
Use the dataset spreadsheet for metadata and citations. For collaboration questions or to enrol a dataset, the project page links the Dataset Enrollment Form and gives the contact address [email protected].
Free ways to work with it and related resources
Everything described above is open: the code is Apache 2.0, the other materials are CC-BY 4.0. If your aim is to explore the data first and train later, start with Route A. If you only need to confirm a loading pipeline, Route B on a single dataset from the spreadsheet is enough. To keep your own robot-learning data findable alongside it, see how Hugging Face Datasets hosts and documents machine-learning datasets.
Citing Open X-Embodiment correctly
The README asks for two things. If you use the Open X-Embodiment dataset and RT-X, cite the collaboration using the citation text on the project page. If you use datasets that were contributed to the joint effort, cite those as well; the dataset spreadsheet contains the citation for each. A paper that trains on, say, a handful of the 60 constituent datasets therefore cites the collaboration plus each of those datasets. See how to cite a dataset for the general pattern.
Frequently asked questions
What is Open X-Embodiment?
It is a collaborative dataset and model release for robot learning. The project page reports 60 pooled datasets, 22 robot embodiments and more than one million real robot trajectories, released with the RT-1-X and RT-2-X models.
What format is the data in?
Each dataset is a sequence of episodes in the RLDS episode format, per the repository README.
How do I load a dataset?
Use tfds.load({dataset_name}). If it raises DatasetNotFoundError, copy the dataset with the gsutil command in the troubleshooting section, then load it again.
Is it free to use?
The README states that software is licensed under Apache 2.0 and all other materials under CC-BY 4.0. Cite the collaboration and any individual datasets you use.
What inputs does the RT-1-X checkpoint accept?
One RGB workspace-camera image and a task string. It does not take wrist-camera, in-hand camera or depth input, and its output is seven gripper variables.
Sources
- Open X-Embodiment project page, robotics-transformer-x.github.io (read 2026-11-14).
- Open X-Embodiment Collaboration, “Open X-Embodiment: Robotic Learning Datasets and RT-X Models”, arXiv:2310.08864, https://arxiv.org/abs/2310.08864 (read 2026-11-14).
- google-deepmind/open_x_embodiment repository README, https://github.com/google-deepmind/open_x_embodiment (read 2026-11-14).
Data: Open X-Embodiment access routes and commands
Compilation of the access routes the Open X-Embodiment repository README documents for the dataset and the RT-1-X checkpoint, with the exact notebook path or command for each route and what it does. Compiled from the official repository README.
| Route | Command or file | What it does | Source |
|---|---|---|---|
| Dataset Colab | colabs/Open_X_Embodiment_Datasets.ipynb | Visualise episodes from each dataset; build batches for training and inference | README google-deepmind/open_x_embodiment |
| TensorFlow Datasets load | tfds.load({dataset_name}) | Load a downloaded dataset by name | README google-deepmind/open_x_embodiment |
| Manual dataset copy | gsutil -m cp -r gs://gdm-robotics-open-x-embodiment/{dataset_name} ~/tensorflow_datasets/ | Fix for DatasetNotFoundError; afterwards tfds.load works | README google-deepmind/open_x_embodiment |
| RT-1-X TensorFlow checkpoint | colabs/Minimal_example_for_running_inference_using_RT_1_X_TF_using_tensorflow_datasets.ipynb | Load checkpoint, run inference on offline episodes, overlay predicted and ground-truth actions | README google-deepmind/open_x_embodiment |
| RT-1-X JAX checkpoint | gsutil -m cp -r gs://gdm-robotics-open-x-embodiment/open_x_embodiment_and_rt_x_oss/rt_1_x_jax . | Download checkpoint; load with models/rt1_inference_example.py via the Flax checkpoint loader | README google-deepmind/open_x_embodiment |
Sources
- google-deepmind/open_x_embodiment README (accessed 2026-11-14)
Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.







