Generating Realistic Arm Movements in Reinforcement Learning: A Quantitative Comparison of Reward Terms and Task Requirements

The mimicking of human-like arm movement characteristics involves the consideration of three factors during control policy synthesis: (a) chosen task requirements, (b) inclusion of noise during movement execution and (c) chosen optimality principles. Previous studies showed that when considering these factors (a-c) individually, it is possible to synthesize arm movements that either kinematically match the experimental data or reproduce the stereotypical triphasic muscle activation pattern. However, to date no quantitative comparison has been made on how realistic the arm movement generated by each factor is; as well as whether a partial or total combination of all factors results in arm movements with human-like kinematic characteristics and a triphasic muscle pattern. To investigate this, we used reinforcement learning to learn a control policy for a musculoskeletal arm model, aiming to discern which combination of factors (a-c) results in realistic arm movements according to four frequently reported stereotypical characteristics. Our findings indicate that incorporating velocity and acceleration requirements into the reaching task, employing reward terms that encourage minimization of mechanical work, hand jerk, and control effort, along with the inclusion of noise during movement, leads to the emergence of realistic human arm movements in reinforcement learning. We expect that the gained insights will help in the future to better predict desired arm movements and corrective forces in wearable assistive devices.

DimRad: A Radar-Based Perception System for Prosthetic Leg Barrier Traversing

Lower extremity amputees face challenges in natural locomotion, which is partially compensated using powered assistive systems, e.g., micro-processor controlled prosthetic leg. In this paper, a radar-based perception system is proposed to assist prosthetic legs for autonomous obstacle traversing, focusing on multiple-step staircases. The presented perception system is composed of a radar module operating with a multiple-input-multiple-output (MIMO) configuration to localize consecutive stair corners. An inertial measurement unit (IMU) is integrated for coordinates correction due to the angular dis-positioning that occurs because of the knee angular motion. The captured information from both sensors is used for staircase dimensioning (depth and height). A shallow neural network (NN) is proposed to model the error due to the hardware limitations and enhance the dimension estimation accuracy (1 cm). The algorithm is implemented on a microcontroller subsystem of the radar kit to qualify the perception system for embedded integration in powered prosthetic legs.