Generative Artificial Intelligence Advancements in Robotics – RT-X Project
Generative artificial intelligence, which underlies instruments such as Chatgpt, Midjourney, and many others, is rapidly developing. These technologies are based on a simple formula: the use of huge neural networks trained on data from the Internet to perform a wide range of tasks.
However, the successful formula of generative AI is difficult to use in robotics. Teaching robots requires specialized data, often created slowly and tediously in laboratory conditions.
In 2023, Google laboratories and the University of California in Berkeley, in collaboration with 32 other laboratories in North America, Europe, and Asia, launched the RT-X project. The aim of the project is to collect data, resources, and code for creating universal robots.
In the first phase of the project, scientists are striving to create a universal robot using a single neural network capable of managing various types of robots. The RT-X database, called Open X-Embodiment Dataset, includes nearly a million tests for 22 types of robots, making it the largest open repository of real robot actions.
The model is designed to control the robot by understanding the given task and the type of robot.
Studies have indicated that data from multiple robots can be utilized with relatively simple machine learning techniques. A model trained on the RT-X dataset can easily identify which robot it controls based on data from the robot’s camera.
Moreover, experts have integrated image systems and texts from the Internet, enabling robots to perform more complex tasks that require an understanding of semantic relationships between objects. Test results from five laboratories have shown that this model surpasses existing management methods, successfully completing tasks an average of 50% more often.