AI Papers Reader

Personalized digests of latest AI research

View on GitHub

RoboVerse: A Unified Platform for Scalable Robot Learning

Researchers at UC Berkeley, PKU, USC, UMich, UIUC, Stanford, CMU, UCLA, and BIGAI have introduced RoboVerse, a comprehensive framework designed to accelerate robot learning by unifying simulation, datasets, and benchmarks. The system addresses key challenges in the robotics field, such as the difficulty in collecting real-world data and the lack of standardized evaluation protocols.

The core of RoboVerse is METASIM, an infrastructure that abstracts diverse simulation environments into a universal interface. For example, a robotics researcher could use METASIM to easily transfer a task, like stacking blocks, from the Meta-World simulation environment to the more visually realistic Isaac Gym. This cross-simulator integration enables a unified benchmarking system and facilitates the transfer of environments and robot trajectories across different platforms. This is made possible by METASIM’s three core components.

One core concept of METASIM is “Hybrid Simulation.” It combines the strengths of multiple simulators—such as pairing advanced physics engines with superior renderers—to generate scalable and high-quality synthetic data. For instance, a user could utilize the accurate physics engine of MuJoCo with the photorealistic rendering of Isaac Sim for more realistic simulations.

Another aspect is “Cross-Embodiment Transfer,” which allows for the retargeting of trajectories across various robot arms with parallel grippers, maximizing dataset reuse from heterogeneous sources. If a robot learns how to open a drawer, a robot with a different arm can use that experience to guide its learning process with that information being translated between each robot’s form factor.

The researchers have constructed a large and diverse high-quality synthetic dataset using these workflows, including roughly 500,000 unique trajectories covering 276 task categories and over 5,500 assets. Additionally, they have generated over 50 million high-quality state transitions to support policy learning. An example of task data comes from existing datasets of “stack cube,” “close box,” and “move slider left.”

RoboVerse also features unified benchmarks for imitation learning and reinforcement learning. The benchmarks assess varying levels of generalization, like the performance of a robot arm that may be trained to act in one room but put in another.

The study validates the robustness of RoboVerse through experiments in imitation learning, reinforcement learning, and world model learning, demonstrating that it enhances the performance of learning algorithms and supports sim-to-real transfer with high-fidelity physics and rendering. Researchers use a 90%/10% training/validation split for this to test how a robot will respond in a variety of novel environments and lighting conditions.

In a demonstration of real-world applicability, the study fine-tuned OpenVLA (an open-source vision-language-action model) on the RoboVerse dataset and transferred the learned policy to real-world scenarios without additional fine-tuning, which resulted in the robot successfully manipulating unseen objects in new real-world settings.

Code and datasets can be found at https://roboverseorg.github.io/.