We are opening a circle in which you can engage in modern robotics and artificial intelligence at the most advanced level and tasks. In the article, we will talk about how it will be and about the current results of teaching robots to walk.

To say that with the advent of ChatGPT and the Tesla bots, a grand future has arrived is an understatement. We stand on the threshold of great changes. What used to seem like fantasy is becoming a reality: robots can execute natural language commands, walk and manipulate objects in the physical world. All this is primarily due to the rapid development of deep learning in recent years. However, cutting-edge research in this field and robotics that will ultimately change the landscape of intelligent robots and their applications are yet to come. We will talk about these studies further.

Why is this important?

Sooner or later there will be a lot of walking robots. They will do the work for us. The race of companies to occupy the market of large walking robots has already begun.

The profession of developing these robots and teaching them various skills looks very promising.

For robots to be “intelligent” and able to do work for us in the same way as a human, their ability to learn and perform complex tasks must greatly increase. This requires new algorithms in the field of deep learning and reinforcement learning. As a matter of fact, in the group we will be engaged in cutting-edge research in the field of teaching robots to walk and solve other complex motor tasks. We will read foreign scientific articles, reproduce their results, generate new ideas, implement them, experiment on robots and in simulators, develop new algorithms, write program code, propose scientific innovations and write scientific articles.

Robots can already be trained on large data sets and in a simulator, but so far they do not learn well in real life and generally do not have the same self-learning abilities as humans. This creates problems because not everything can be reproduced in the simulator and there are not ready-made datasets for everything. I believe that the future lies in the ability of robots and AI to learn independently in the real world.

Training robots

For example, here is a four-legged robot that was trained at our Sbera Robotics Center first in a simulator and then run on a real hardware platform:

The PPO (Proximal Policy Optimization) algorithm was used with training in the Raisim simulator for several hours on a GeForce GTX 1080 Ti video card:

Rewards feature:

• Optimization of the current linear velocity of the body relative to the target;

• Minimized factors:

  • lateral movement and angular velocities;

  • deviation of body height from the nominal;

  • deviation of the position of the body from the nominal orientation;

  • work performed;

  • angular velocities of joints;

  • feet slipping on the surface;

  • smoothness of movements;

  • surface force;

• Maximized factors:

It would seem that what else is needed? But this approach works poorly or does not work at all for more complex robots such as Dijit, Tesla Bot, Figure. This is due to a large sim2real gap. Because of this, a crossroads appears — either to make the simulation even more realistic, or to teach robots in real life. Here is an example of training a four-legged robot from scratch in real life at our Sber Robotics Center:

The Actor-Critic algorithm was used with regularization of joint angle limits by a separate neural network. The robot was trained in real time on a laptop connected via ethernet.

Learning in real life now takes from two hours of pure work time when he is directly studying. So far, it is possible to learn only simple movements such as crawling forward, while, on average, in a couple of exercises, one motor burns out due to overloads in the hard mode of operation. However, this is a significant result that was previously unattainable. So far, there are few scientific articles in this direction and a lot of potential for research and new breakthroughs.

There are also hybrid and other approaches related to retraining, mixed approaches of reinforcement learning and optimal control. In recent years, hundreds of scientific articles with new approaches have been written in the field of walking robots, which indicates the rapid development of this topic. It is also worth noting that it was in stepping robots that reinforcement learning was first used in a really effective way and showed advantages compared to other methods.

Here is a presentation of our previous work on this project, where the robot runs on the classic optimal control c cMPC:

We laid out all source code open access for use by future researchers.

Our II circle in robotics

We see further prospects for research and have drawn up a plan of tasks for the near future. If you are interested in this topic, come to us for a free practice. For this purpose, we will organize sites in Moscow where you can train modern robots with us.

Currently, there is an agreement with the Institute of Artificial Intelligence of RTU MIREA, as well as a previous agreement with the Center for Youth Robotics of the Moscow State Technical University named after Bauman and the Laboratory for Humanoid Robots at Fiztech Lyceum for providing us with a place to work with robots. In general, any technical university in Moscow can express its desire to become a partner in this, we will provide the equipment and consultations. That is, we can say that we are opening something like a scientific research circle on robot training and advanced research in the field of AI in robotics. You can go to it from Monday to Saturday from 10:00 to 21:00 to our partners who provide training sites and support.

Our target audience is students, masters, postgraduates and adults who have technical experience, preferably specialized education in programming, robotics, machine learning. It will be too difficult for schoolchildren to do these tasks.

What are the requirements for candidates:

• reliable work in the Linux terminal;

• understanding of the operation of a local network, the ability to configure a static IP address in Linux;

• knowledge of Git and Python;

• knowledge of deep learning algorithms;

• knowledge of reinforcement learning algorithms;

• ability to read and analyze scientific articles in English;

• group classes at least three days a week;

• self-learning ability.

What will be an advantage:

• knowledge of ROS/ROS2, Docker, C++;

• own scientific articles;

• understanding of robotic algorithms such as PID, Kalman filter, etc.;

• Hot eyes on the topic of robotics and AI.

Definitely not suitable for those who:

• does not know how to program well;

• doubts whether he needs it;

• there is not enough time for this;

• does not understand what scientific articles are and why they are important.

How are the classes?

The Sber Robotics Center provides promising tasks in the field of training robots, which can be performed both individually and in a group. There is a selection of scientific articles and basic line algorithms to get you started. In the process of work, specialists of the Sber Robotics Center help with consultations. Currently, we do not offer additional training, that is, the knowledge you lack is acquired by yourself. Joint meetings are held once a week or two to plan further tasks. Writing joint scientific articles is welcome.

What are the prospects?

After two or three years of classes in the group, you will gain the necessary knowledge and experience, and the best of you will be able to get a job, for example, at the Sbera Robotics Center or another robotics company. I promise that with such a portfolio and competencies, you will definitely be in demand on the labor market. We also cooperate with various universities and research groups, in which you can get a job and continue doing research on this topic.

Thus, we are trying to develop a robotics community in the field of AI. The initiative of universities and scientific groups is welcomed.

If you want to work in a group, write to me on Habr or in Telegram. Classes are free, but there is a choice through test tasks.