Robot Learning: Software, Training, and Setup

Robot learning is the interdisciplinary domain within artificial intelligence and control engineering that develops algorithms enabling robots to acquire, adapt, and refine motor skills directly from empirical sensorimotor experience and demonstrations. In traditional robotics, behaviors are designed using hand-coded forward and inverse kinematics, analytic dynamic models, and trajectory optimization. Robot learning replaces or augments these rigid mathematical pipelines with data-driven models capable of handling high-dimensional visual feedback, complex contact dynamics, and unmodeled real-world friction.

What it is used for in research

Modern robot learning research operates at the confluence of machine learning, computer vision, and physical control. The primary objective is to train sensorimotor policies—mathematical functions mapping sensory inputs (RGB-D camera streams, joint encoders, force-torque sensors) into actuator commands (target joint angles, velocities, or end-effector poses).

Empirical research in this area spans several core methodologies:

Imitation Learning and Behavioral Cloning: Policy networks are trained directly on demonstrations collected through human teleoperation, kinesthetic teaching, or motion capture. State-of-the-art frameworks leverage Action Chunking with Transformers (ACT) and Diffusion Policies to predict trajectories of continuous action vectors, effectively eliminating temporal compounding error. For specialized algorithmic implementations in Gymnasium, see our guide on imitation learning.
Reinforcement Learning (RL): Autonomous policies optimize reward functions through iterative trial-and-error interaction inside simulated physics engines or directly on hardware. Popular policy optimization methods include Soft Actor-Critic (SAC), Proximal Policy Optimization (PPO), and model-based methods like TD-MPC. For baseline implementations and training loops, see our reference on Stable-Baselines3.
Vision-Language-Action (VLA) Foundation Models: Large multimodal neural networks integrate visual encoders and language backbones with robotics action decoders (such as Open-X Embodiment, Pi0, SmolVLA, and GR00T), allowing physical agents to execute natural language instructions across diverse environments.
Sim-to-Real Transfer: Bridging the reality gap where policies trained inside accelerated physics simulators fail when deployed to physical hardware. Researchers employ domain randomization, system identification, and tactile feedback to ensure robust transfer. For kinematic structure modeling of manipulators, researchers define joint and link topologies using URDF files.

Robot learning research is standardized and disseminated through academic bodies including the IEEE Robotics and Automation Society (IEEE RAS) Technical Committee on Robot Learning and the Robot Learning Foundation, which governs the annual Conference on Robot Learning (CoRL). Research methodologies also build upon foundational comparative frameworks between algorithmic modeling and statistical validation, as outlined in our overview of artificial intelligence and data science.

Install or access (official source only)

In contemporary robot learning research, Hugging Face LeRobot is a primary open-source library built natively in PyTorch (Apache-2.0 license) that standardizes datasets, policy architectures, and simulation-to-hardware deployment.

LeRobot requires Python 3.10 or newer, PyTorch with CUDA acceleration, and FFmpeg (required for encoding and streaming synchronized camera video feeds). The official recommended installation uses Conda to manage environment dependencies and system-level media libraries.

# Create and activate an isolated Python 3.10 environment
conda create -y -n lerobot python=3.10
conda activate lerobot

# Install FFmpeg from conda-forge for video decoding support
conda install -y ffmpeg=7.1.1 -c conda-forge

# Install core LeRobot package from PyPI
pip install lerobot

# Install training dependencies and standard simulation benchmarks (PushT and Aloha)
pip install "lerobot[training,pusht,aloha]"

To verify the installation and confirm that CUDA devices, FFmpeg libraries, and backend environments are correctly linked, run the official diagnostic utility:

lerobot-info

Step-by-step task

The following five-step task executes an end-to-end robot learning research workflow: downloading a standardized demonstration dataset, configuring a Diffusion Policy, training the model, evaluating policy rollout success rates in simulation, and inspecting policy weights.

Authenticate with the Hugging Face Hub:

Robot learning datasets and model checkpoints are distributed via the Hugging Face Hub. Authenticate your terminal to ensure seamless dataset streaming:
```
huggingface-cli login
```

Launch Policy Training:

Execute lerobot-train to train a Diffusion Policy on the standard lerobot/pusht benchmark environment. Specify the network architecture, batch size, action chunking horizon, and checkpoint intervals:

lerobot-train 
  --policy.type=diffusion 
  --dataset.repo_id=lerobot/pusht 
  --env.type=pusht 
  --env.task=pusht 
  --training.batch_size=64 
  --training.n_action_steps=8 
  --training.save_freq=5000 
  --training.offline_steps=50000 
  --device=cuda 
  --policy.push_to_hub=false

Monitor Training Checkpoints:

LeRobot saves periodic checkpoints, optimizer state dictionaries, and normalization statistics under outputs/train/. Verify that checkpoint artifacts have been generated in your workspace:
```
ls -la outputs/train/diffusion_pusht/checkpoints/last/pretrained_model/
```
Evaluate Policy Success Rate in Simulation:

Use the official evaluation utility lerobot-eval to execute quantitative rollouts inside the simulation environment across 10 distinct evaluation episodes:
```
lerobot-eval 
  --policy.path=outputs/train/diffusion_pusht/checkpoints/last/pretrained_model 
  --env.type=pusht 
  --env.task=pusht 
  --eval.n_episodes=10 
  --eval.batch_size=10
```

Deploy Policy to Hardware via the Python API:

To transfer the trained policy from simulation to a physical manipulator (such as an SO-100 or Aloha setup), instantiate the unified Robot class interface in Python:

from lerobot.robots.so100 import SO100Robot
from lerobot.policies.diffusion.modeling_diffusion import DiffusionPolicy

# Instantiate robot connection
robot = SO100Robot()
robot.connect()

# Load trained policy weights
policy = DiffusionPolicy.from_pretrained("outputs/train/diffusion_pusht/checkpoints/last/pretrained_model")

# Execute sensorimotor inference loop
obs = robot.get_observation()
action = policy.select_action(obs)
robot.send_action(action)

Output and how to read it

Robot learning workflows produce structured data artifacts across data ingestion, model optimization, and empirical evaluation.

1. Standardized LeRobotDataset Schema (v3.0): Unlike legacy frameworks that store high-dimensional image observations in monolithic HDF5 files, LeRobot structures datasets into two synchronized components:

data/chunk-*.parquet: Apache Parquet columnar files storing numerical timestamps, proprioceptive joint states, motor currents, and executed action vectors.
videos/observation.images.*.mp4: Synchronized MP4 video streams encoded with H.264, enabling hardware-accelerated video decoding and multithreaded dataloading without filesystem lock contention.
meta/info.json and meta/stats.json: Normalization statistics containing minimum, maximum, mean, and standard deviation values for observation and action spaces.

2. Training Telemetry and Loss Curves: During policy optimization, lerobot-train logs telemetry metrics:

loss/action_loss: Mean squared error or negative log-likelihood between predicted action trajectories and expert demonstrations. For Diffusion Policies, this represents the score-matching noise prediction loss.
loss/kld: Kullback-Leibler divergence when using conditional variational autoencoders (e.g., ACT), measuring latent space regularization against the prior Gaussian distribution.
grad_norm: The L2 norm of the network gradients, used to identify gradient explosion or saturation in deep transformer backbones.

3. Quantitative Evaluation Metrics: The output of lerobot-eval summarizes policy performance:

eval/success_rate: The proportion of evaluation episodes where the robot achieved the defined success criteria (e.g., placing a block within target tolerance boundaries). Reported as a floating-point value between 0.0 and 1.0 (or percentage).
eval/ep_len_mean: The average episode duration in steps before either task completion or timeout (typically 200–500 steps). Efficient policies exhibit shorter completion lengths.
eval/ep_reward_mean: The mean cumulative reward accumulated over the rollout horizon.

Troubleshooting

When running robot learning pipelines, research teams frequently encounter specific hardware and software bottlenecks. The following solutions address documented failure modes:

CUDA Out of Memory (OOM) During Diffusion Training: Diffusion policies and transformer-based action chunking architectures consume substantial VRAM due to denoising sampling steps and multi-camera image backbones. To resolve memory exhaustion, decrease the per-device batch size (e.g., set --training.batch_size=32 or 16), reduce observation image dimensions via configuration overrides, or activate gradient checkpointing.
FFmpeg Video Stream Decode Errors: If training crashes with errors like AV_ERROR_DECODER_NOT_FOUND or corrupt frame headers, your environment is linking against an incomplete system FFmpeg build. Resolve this by forcing the conda-forge build: conda install -y ffmpeg=7.1.1 -c conda-forge and verifying that PyAV and torchvision can read the video codecs.
Action-Observation Desynchronization: In real-world rollouts, communication latency over serial USB buses or network sockets can cause delays between camera acquisition and motor execution. Ensure that control loops operate at deterministic frequencies (e.g., 30 Hz or 50 Hz) and verify that n_action_steps matches the mechanical latency of your actuator hardware.
Distribution Drift and Covariate Shift: If the robot diverges into unrecoverable states when physical obstacles shift slightly, the policy suffers from classic behavioral cloning error accumulation. Mitigate this by collecting corrective demonstrations, employing data augmentation (random crops, brightness jitter), or incorporating closed-loop action chunking with receding horizon control.

Free alternatives

Several free and open-source frameworks support robot learning research. The table below compares the leading libraries across architecture, governance, and supported environments:

Framework	Maintainer / Institution	Primary Paradigm	Native Dataset Format	Supported Simulators	License
LeRobot	Hugging Face Community	Imitation Learning, RL, VLA	LeRobotDataset (Parquet + MP4)	MuJoCo, PushT, Aloha, LIBERO	Apache-2.0
robomimic	Stanford University / UT Austin (ARISE)	Imitation Learning, Offline RL	HDF5	MuJoCo, Robosuite	MIT License
Gymnasium-Robotics	Farama Foundation	Reinforcement Learning, Multi-Goal RL	NumPy Array / Farama Dict	MuJoCo, PyBullet	MIT License
Isaac Lab	NVIDIA / Open Source Community	Large-Scale RL, Sim-to-Real, Imitation	USD / HDF5 / PyTorch Tensor	NVIDIA Isaac Sim / PhysX	BSD-3-Clause

FAQ

What is the difference between classical robotics and robot learning?

Classical robotics relies on explicit analytical models of kinematics, dynamics, and environmental geometry to calculate deterministic trajectories. In contrast, robot learning trains statistical policies directly from sensorimotor observations and interaction data, enabling systems to adapt to unstructured environments, visual variations, and complex contact dynamics without explicit physics models.

What is action chunking and why is it used in imitation learning?

Action chunking groups multiple consecutive timesteps of action commands (e.g., 8–16 steps) into a single prediction chunk. By predicting whole trajectory segments instead of individual instantaneous actions, policy networks reduce compounding error, smooth motor execution, and overcome the non-Markovian nature of human demonstrations.

Why does LeRobot store observations as MP4 videos instead of HDF5 arrays?

Storing high-framerate multi-camera robotics demonstrations as uncompressed NumPy arrays inside HDF5 containers leads to enormous file sizes (hundreds of gigabytes per dataset) and severe I/O bottlenecks. LeRobot’s v3.0 format stores video frames compressed with H.264/MP4, reducing disk footprint by up to 90% while enabling hardware-accelerated video decoding during batch generation.

Can robot learning policies be fine-tuned on custom manipulators?

Yes. LeRobot and robomimic provide modular hardware and environment abstraction classes. By implementing a standardized Python interface containing connect(), get_observation(), and send_action() methods, researchers can map custom microcontrollers, ROS 2 topics, or serial servo buses into existing training and inference pipelines.

Sources (official docs)

Hugging Face LeRobot Documentation and Repository (Accessed 2026-11-09): https://github.com/huggingface/lerobot
robomimic: A Framework for Robot Learning from Demonstration (Mandlekar et al., Stanford / UT Austin / ARISE Initiative, Accessed 2026-11-09): https://robomimic.github.io/
Robot Learning Foundation and Conference on Robot Learning (CoRL) (Accessed 2026-11-09): https://robot-learning.org/
IEEE Robotics and Automation Society (IEEE RAS) Technical Committee on Robot Learning (Accessed 2026-11-09): https://www.ieee-ras.org/robot-learning