A Reinforcement Learning Based Controller to Minimize Forces on the Crutches of a Lower-Limb Exoskeleton

Metabolic energy consumption of a powered lower-limb exoskeleton user mainly comes from the upper body effort since the lower body is considered to be passive. However, the upper body effort of the users is largely ignored in the literature when designing motion controllers. In this work, we use deep reinforcement learning to develop a locomotion controller that minimizes ground reaction forces (GRF) on crutches. The rationale for minimizing GRF is to reduce the upper body effort of the user. Accordingly, we design a model and a learning framework for a human-exoskeleton system with crutches. We formulate a reward function to encourage the forward displacement of a human-exoskeleton system while satisfying the predetermined constraints of a physical robot. We evaluate our new framework using Proximal Policy Optimization, a state-of-the-art deep reinforcement learning (RL) method, on the MuJoCo physics simulator with different hyperparameters and network architectures over multiple trials. We empirically show that our learning model can generate joint torques based on the joint angle, velocities, and the GRF on the feet and crutch tips. The resulting exoskeleton model can directly generate joint torques from states in line with the RL framework. Finally, we empirically show that policy trained using our method can generate a gait with a 35% reduction in GRF with respect to the baseline.

Modular Safety-Critical Control of Legged Robots

Safety concerns during the operation of legged robots must be addressed to enable their widespread use. Machine learning-based control methods that use model-based constraints provide promising means to improve robot safety. This study presents a modular safety filter to improve the safety of a legged robot, i.e., reduce the chance of a fall. The prerequisite is the availability of a robot that is capable of locomotion, i.e., a nominal controller exists. During locomotion, terrain properties around the robot are estimated through machine learning which uses a minimal set of proprioceptive signals. A novel deep-learning model utilizing an efficient transformer architecture is used for the terrain estimation. A quadratic program combines the terrain estimations with inverse dynamics and a novel exponential control barrier function constraint to filter and certify nominal control signals. The result is an optimal controller that acts as a filter. The filtered control signal allows safe locomotion of the robot. The resulting approach is generalizable, and could be transferred with low effort to any other legged system.