Model-Based Reinforcement Learning Exploits Passive Body Dynamics for High-Performance Biped Robot Locomotion

Embodiment is a significant keyword in recent machine learning fields. This study focused on the passive nature of the body of a biped robot to generate walking and running locomotion using model-based deep reinforcement learning. We constructed two models in a simulator, one with passive elements (e.g., springs) and the other, which is similar to general humanoids, without passive elements. The training of the model with passive elements was highly affected by the attractor of the system. This lead that although the trajectories quickly converged to limit cycles, it took a long time to obtain large rewards. However, thanks to the attractor-driven learning, the acquired locomotion was robust and energy-efficient. The results revealed that robots with passive elements could efficiently acquire high-performance locomotion by utilizing stable limit cycles generated through dynamic interaction between the body and ground. This study demonstrates the importance of implementing passive properties in the body for future embodied AI.

Bipedal Robot Running: Human-like Actuation Timing Using Fast and Slow Adaptations

We have been developing human-sized biped robots based on passive dynamic mechanisms. In human locomotion, the muscles activate at the same rate relative to the gait cycle during running. To achieve adaptive running for robots, such characteristics should be reproduced to yield the desired effect. In this study, we designed a central pattern generator (CPG) involving fast and slow adaptation to achieve human-like running using a simple spring-mass model and our developed bipedal robot, which is equipped with actuators that imitate the human musculoskeletal system. Our results demonstrate that fast and slow adaptations can reproduce human-like running with a constant rate of muscle firing relative to the gait cycle. Furthermore, the results suggest that the CPG contributes to the adjustment of the muscle activation timing in human running.

Effect of the Dynamics of a Horizontally Wobbling Mass on Biped Walking Performance

We have developed biped robots with a passive dynamic walking mechanism. This study proposes a compass model with a wobbling mass connected to the upper body and oscillating in the horizontal direction to clarify the influence of the horizontal dynamics of the upper body on bipedal walking. The limit cycles of the model were numerically searched, and their stability and energy efficiency was investigated. Several qualitatively different limit cycles were obtained depending mainly on the spring constant that supports the wobbling mass. Specific types of solutions decreased the stability while reducing the risk of accidental falling and improving the energy efficiency. The obtained results were attributed to the wobbling mass moving in the opposite direction to the upper body, thereby preventing large changes in acceleration and deceleration while walking. The relationship between the locomotion of the proposed model and the actual biped robot and human gaits was investigated.