The future of robotics is becoming increasingly exciting with the introduction of groundbreaking technologies. AutoRT, SARA-RT, and RT-Trajectory are three innovative systems from Google Deepmind mark a significant leap forward in the realm of robotics. These advancements aim to enhance real-world robot data collection, speed up decision-making, and improve generalization capabilities.
AutoRT: Scaling Robotic Learning with Large Models
AutoRT is a revolutionary system designed to leverage the power of large foundation models, a crucial element in developing robots capable of understanding practical human goals. The system focuses on collecting diverse experiential training data to scale robotic learning for real-world applications.
By combining large foundation models, such as Large Language Models (LLM) or Visual Language Models (VLM), with robot control models (RT-1 or RT-2), AutoRT enables the deployment of robots for gathering training data in novel environments. The system can simultaneously direct multiple robots equipped with cameras and end effectors to perform diverse tasks in various settings. In real-world evaluations spanning seven months, AutoRT successfully orchestrated up to 20 robots simultaneously, accumulating a dataset of 77,000 robotic trials across 6,650 unique tasks. Safety is a paramount concern, and AutoRT incorporates layered safety protocols, including a Robot Constitution inspired by Isaac Asimov’s Three Laws of Robotics.
SARA-RT: Enhancing Efficiency with Self-Adaptive Robust Attention
Self-Adaptive Robust Attention for Robotics Transformers (SARA-RT) focuses on making Robotics Transformers (RT) models more efficient. The RT neural network architecture is prevalent in modern robotic control systems, and SARA-RT enhances the speed and accuracy of these models.
Transformers, while powerful, face limitations due to computational demands. SARA-RT introduces a novel method called “up-training” to convert quadratic complexity to linear complexity, significantly reducing computational requirements. The best SARA-RT-2 models exhibited a 10.6% increase in accuracy and a 14% improvement in speed compared to RT-2 models.
SARA-RT’s approach offers a scalable solution for speeding up Transformers, potentially expanding the use of this technology. With its theoretical grounding, SARA-RT can be applied to various Transformer models, showcasing its versatility.
RT-Trajectory: Facilitating Generalization in Robotic Tasks
RT-Trajectory addresses the challenge of generalizing robotic tasks by automatically adding visual outlines to training videos. This model overlays 2D trajectory sketches on robot arm movements during training, providing practical visual hints to enhance the model’s learning.
When tested on unseen tasks, RT-Trajectory demonstrated impressive results, achieving a task success rate of 63%, compared to 29% for RT-2. The model’s ability to interpret specific robot motions, as depicted in videos or sketches, contributes to improved task generalization.
RT-Trajectory’s versatility extends to creating trajectories from human demonstrations and hand-drawn sketches. The system is adaptable to different robot platforms, making it a valuable tool for diverse applications.
Building the Foundations for Next-Generation Robots
These advancements, building upon state-of-the-art RT-1 and RT-2 models, pave the way for more capable and helpful robots. The future envisions the integration of these systems, combining motion generalization from RT-Trajectory, efficiency from SARA-RT, and large-scale data collection from AutoRT. As robotics continues to evolve, these technologies will play a crucial role in shaping the capabilities of advanced robots. The journey involves ongoing adaptation to emerging challenges and the incorporation of new capabilities and technologies in robotics.
