Skip to main content
v2026.11,986 entries · CC-BY 4.0

Open X-Embodiment: Dataset, RT-X Models and How to Access Them

Open X-Embodiment pools 60 robot datasets into one format. How to load the RLDS data, run RT-1-X, fix DatasetNotFoundError and cite it correctly.

Written and maintained by CASRAI Editorial Board

Published

On this page

Open X-Embodiment is a collaboration that pooled robot-learning data from 60 existing datasets, covering 22 robot embodiments and more than one million real robot trajectories, into one common format, and released the RT-1-X and RT-2-X models trained on it. Researchers get the data as episodes in the RLDS episode format, loaded with TensorFlow Datasets or copied with gsutil, plus a public RT-1-X checkpoint with Colab and JAX examples. This guide explains what it is for, the access routes in the official repository, how to read the data and the model’s inputs and outputs, and what you must cite.

Ask CASRAI · free to tryQuestion about this guide? Ask CASRAI answers with linked sources. 2 free questions a day, no account.

Ask your first question freeSubscribe

What Open X-Embodiment is used for in research

The project page describes Open X-Embodiment as a dataset and model release for robotic learning, assembled by the Open X-Embodiment Collaboration, which the arXiv record lists as the collaboration plus 293 further authors (arXiv 2310.08864, first posted 13 October 2023, latest version v9 on 14 May 2025). According to the project page, the pooled data comes from 21 institutions and spans single arms, bimanual robots and quadrupeds. The abstract reports 527 skills and 160,266 tasks across the 22 embodiments.

The research question behind it is transfer: can a policy trained on data from many different robots perform better than one trained on a single robot’s data? The authors report positive transfer. On the project page, RT-1-X (the RT-1 architecture trained on the pooled data) is reported to outperform the original methods by about 50 percent in small-data domains, and RT-2-X (built on the RT-2 vision-language model, with robot actions represented as tokens) is reported to be about three times better than RT-2 on emergent-skill evaluations. Treat these as the authors’ own reported results and read the paper’s evaluation setup before comparing against them.

In practice researchers use the release in three ways: as pre-training or co-training data for robot policies, as a benchmark source with a shared episode format, and as a reference implementation for running an RT-1-X checkpoint offline on recorded episodes. For the wider method family see our guides to robot learning and imitation learning.

Access: the official sources

All access routes come from the official repository at github.com/google-deepmind/open_x_embodiment and the project page at robotics-transformer-x.github.io. The repository README states that the project aims to provide all open-sourced robotic data in the same unified format. Its README does not list a package-installation command or a dependency list, so use the environment your chosen notebook sets up, or the TensorFlow Datasets and Google Cloud gsutil tools the commands below rely on.

  • Project page (robotics-transformer-x.github.io): links to the paper, the GitHub repository, the dataset spreadsheet, a Dataset Enrollment Form for contributors, and a contact address for the collaboration, [email protected].
  • GitHub repository (google-deepmind/open_x_embodiment): the README, the Colab notebooks, and the model example code.
  • Dataset spreadsheet: the repository links a Google Sheet that lists every dataset in the effort with its metadata and its citation.

The README states that this is not an official Google product. Licensing is split: all software is under the Apache License 2.0, and all other materials are under Creative Commons Attribution 4.0 International (CC-BY). For how attribution works under CC-BY, see our guide to Creative Commons licences for research data.

Step by step: load the data and run RT-1-X

The steps below follow the README. The commands are reproduced as the repository gives them; replace the placeholder with a dataset name from the dataset spreadsheet.

Route A: explore the data in the dataset Colab

  1. Open the repository’s colabs/Open_X_Embodiment_Datasets.ipynb notebook (the README links it as a self-contained Colab).
  2. Run it to visualise a few episodes from each dataset and to see how batches of data are created for training and inference.

Route B: load a dataset with TensorFlow Datasets

  1. Choose the dataset name from the dataset spreadsheet.
  2. Load it with tfds.load({dataset_name}), using the dataset name from the spreadsheet.
  3. If loading fails with DatasetNotFoundError, copy the dataset manually as described under troubleshooting, then repeat the load.

Route C: run the RT-1-X TensorFlow checkpoint on recorded episodes

  1. Open colabs/Minimal_example_for_running_inference_using_RT_1_X_TF_using_tensorflow_datasets.ipynb.
  2. Run it to load the checkpoint, run inference on offline episodes, and overlay the predicted actions on the ground-truth actions.

Route D: download the RT-1-X JAX checkpoint

  1. Copy the checkpoint with gsutil -m cp -r gs://gdm-robotics-open-x-embodiment/open_x_embodiment_and_rt_x_oss/rt_1_x_jax .
  2. Load it with the Flax checkpoint loader through the repository’s models/rt1_inference_example.py.

The data table below summarises these four routes with the exact command or file each one uses.

Output and how to read it

How a dataset is organised

Each dataset is a sequence of episodes, and each episode is stored in the RLDS episode format that the README links to in the google-research/rlds repository. An episode is one recorded run of a robot attempting a task, made of the observations the robot saw and the actions it took. Because every contributed dataset uses this same structure, one data-loading and batching pipeline can read all of them, which is the practical benefit of the unified format.

What the RT-1-X checkpoint takes in and returns

The README gives the model’s interface exactly:

  • Observation: one RGB image from the robot workspace camera, plus a task string describing what the robot should do. The task is communicated purely through that string.
  • Rate: the README illustrates this with a model running at three hertz, so the latest image is fed in every 333 milliseconds to obtain the next action.
  • Not used: the model currently does not take wrist-camera images, in-hand camera images or depth.
  • Action: seven variables for the gripper movement (x, y, z, roll, pitch, yaw and gripper opening). Each variable represents the absolute value, the delta change to the dimension value, or the velocity of the dimension.

When you overlay predicted and ground-truth actions in the inference Colab, remember that the same seven slots can carry absolute, delta or velocity values depending on the dataset, so compare like with like before judging a prediction.

Reading the limits as a researcher

The interface above tells you what the released checkpoint cannot do. A study that depends on depth, wrist-camera views or multi-camera input cannot use the released RT-1-X checkpoint as it stands, and a claim about a robot-specific skill needs the underlying dataset’s own documentation, which the dataset spreadsheet points to. The reported gains in the paper are the authors’ evaluations; replicating them on your own robot or task is a separate piece of work. See our guide to making a dataset FAIR for what documentation reusable robot data should carry.

Troubleshooting

DatasetNotFoundError when calling tfds.load

If tfds.load({dataset_name}) fails with tensorflow_datasets.core.registered.DatasetNotFoundError: Dataset {dataset_name} not found., the README’s fix is to download the dataset manually with gsutil -m cp -r gs://gdm-robotics-open-x-embodiment/{dataset_name} ~/tensorflow_datasets/. After that, the regular tfds.load({dataset_name}) command works with the dataset.

Model results look wrong on your robot

Check first that your input matches the model’s interface: a single workspace RGB image and a task string, at roughly the rate the README describes, with the seven-variable action read as absolute, delta or velocity according to the dataset. The README does not describe adapting the checkpoint to other camera setups, so treat that as outside what the release documents.

Questions about a specific dataset

Use the dataset spreadsheet for metadata and citations. For collaboration questions or to enrol a dataset, the project page links the Dataset Enrollment Form and gives the contact address [email protected].

Free ways to work with it and related resources

Everything described above is open: the code is Apache 2.0, the other materials are CC-BY 4.0. If your aim is to explore the data first and train later, start with Route A. If you only need to confirm a loading pipeline, Route B on a single dataset from the spreadsheet is enough. To keep your own robot-learning data findable alongside it, see how Hugging Face Datasets hosts and documents machine-learning datasets.

Citing Open X-Embodiment correctly

The README asks for two things. If you use the Open X-Embodiment dataset and RT-X, cite the collaboration using the citation text on the project page. If you use datasets that were contributed to the joint effort, cite those as well; the dataset spreadsheet contains the citation for each. A paper that trains on, say, a handful of the 60 constituent datasets therefore cites the collaboration plus each of those datasets. See how to cite a dataset for the general pattern.

Frequently asked questions

What is Open X-Embodiment?

It is a collaborative dataset and model release for robot learning. The project page reports 60 pooled datasets, 22 robot embodiments and more than one million real robot trajectories, released with the RT-1-X and RT-2-X models.

What format is the data in?

Each dataset is a sequence of episodes in the RLDS episode format, per the repository README.

How do I load a dataset?

Use tfds.load({dataset_name}). If it raises DatasetNotFoundError, copy the dataset with the gsutil command in the troubleshooting section, then load it again.

Is it free to use?

The README states that software is licensed under Apache 2.0 and all other materials under CC-BY 4.0. Cite the collaboration and any individual datasets you use.

What inputs does the RT-1-X checkpoint accept?

One RGB workspace-camera image and a task string. It does not take wrist-camera, in-hand camera or depth input, and its output is seven gripper variables.

Sources

  • Open X-Embodiment project page, robotics-transformer-x.github.io (read 2026-11-14).
  • Open X-Embodiment Collaboration, “Open X-Embodiment: Robotic Learning Datasets and RT-X Models”, arXiv:2310.08864, https://arxiv.org/abs/2310.08864 (read 2026-11-14).
  • google-deepmind/open_x_embodiment repository README, https://github.com/google-deepmind/open_x_embodiment (read 2026-11-14).

Data: Open X-Embodiment access routes and commands

Compilation of the access routes the Open X-Embodiment repository README documents for the dataset and the RT-1-X checkpoint, with the exact notebook path or command for each route and what it does. Compiled from the official repository README.

Open X-Embodiment access routes and commands
RouteCommand or fileWhat it doesSource
Dataset Colabcolabs/Open_X_Embodiment_Datasets.ipynbVisualise episodes from each dataset; build batches for training and inferenceREADME google-deepmind/open_x_embodiment
TensorFlow Datasets loadtfds.load({dataset_name})Load a downloaded dataset by nameREADME google-deepmind/open_x_embodiment
Manual dataset copygsutil -m cp -r gs://gdm-robotics-open-x-embodiment/{dataset_name} ~/tensorflow_datasets/Fix for DatasetNotFoundError; afterwards tfds.load worksREADME google-deepmind/open_x_embodiment
RT-1-X TensorFlow checkpointcolabs/Minimal_example_for_running_inference_using_RT_1_X_TF_using_tensorflow_datasets.ipynbLoad checkpoint, run inference on offline episodes, overlay predicted and ground-truth actionsREADME google-deepmind/open_x_embodiment
RT-1-X JAX checkpointgsutil -m cp -r gs://gdm-robotics-open-x-embodiment/open_x_embodiment_and_rt_x_oss/rt_1_x_jax .Download checkpoint; load with models/rt1_inference_example.py via the Flax checkpoint loaderREADME google-deepmind/open_x_embodiment

Sources

Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.

Follow CASRAI

Research-administration guidance, standards updates and independent tool reviews.

Ask CASRAI · free to try

Ask about Open X-Embodiment: Dataset, RT-X Models and How to Access Them

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →