Benchmarking the Full-Order Model Optimization Based Imitation in the Humanoid Robot Reinforcement Learning Walk

When a gait of a bipedal robot is developed using deep reinforcement learning, reference trajectories may or may not be used. Each approach has its advantages and disadvantages, and the choice of method is up to the control developer. This paper investigates the effect of reference trajectories on locomotion learning and the resulting gaits. We implemented three gaits of a full-order anthropomorphic robot model with different reward imitation ratios, provided sim-to-sim control policy transfer, and compared the gaits in terms of robustness and energy efficiency. In addition, we conducted a qualitative analysis of the gaits by interviewing people, since our task was to create an appealing and natural gait for a humanoid robot. According to the results of the experiments, the most successful approach was the one in which the average value of rewards for imitation and adherence to command velocity per episode remained balanced throughout the training. The gait obtained with this method retains naturalness (median of 3.6 according to the user study) compared to the gait trained with imitation only (median of 4.0), while remaining robust close to the gait trained without reference trajectories.

Combining Safe Interval Path Planning and Constrained Path Following Control: Preliminary Results

We study the navigation problem for a robot moving amidst static and dynamic obstacles and rely on a hierarchical approach to solve it. First, the reference trajectory is planned by the safe interval path planning algorithm that is capable of handling any-angle translations and rotations. Second, the path following problem is treated as the constrained control problem and the original flatness-based approach is proposed to generate control. We suggest a few enhancements for the path planning algorithm aimed at finding trajectories that are more likely to be followed by a robot without collisions. Results of the conducted experimental evaluation show that the number of successfully solved navigation instances significantly increases when using the suggested techniques.