Learning of Balance Controller Considering Changes in Body State for Musculoskeletal Humanoids

The musculoskeletal humanoid is difficult to modelize due to the flexibility and redundancy of its body, whose state can change over time, and so balance control of its legs is challenging. There are some cases where ordinary PID controls may cause instability. In this study, to solve these problems, we propose a method of learning a correlation model among the joint angle, muscle tension, and muscle length of the ankle and the zero moment point to perform balance control. In addition, information on the changing body state is embedded in the model using parametric bias, and the model estimates and adapts to the current body state by learning this information online. This makes it possible to adapt to changes in upper body posture that are not directly taken into account in the model, since it is difficult to learn the complete dynamics of the whole body considering the amount of data and computation. The model can also adapt to changes in body state, such as the change in footwear and change in the joint origin due to recalibration. The effectiveness of this method is verified by a simulation and by using an actual musculoskeletal humanoid, Musashi.

Leveraging Pretrained Latent Representations for Few-Shot Imitation Learning on a Dexterous Robotic Hand

In the context of imitation learning applied to dexterous robotic hands, the high complexity of the systems makes learning complex manipulation tasks challenging. However, the numerous datasets depicting human hands in various different tasks could provide us with better knowledge regarding human hand motion. We propose a method to leverage multiple large-scale task-agnostic datasets to obtain latent representations that effectively encode motion subtrajectories that we included in a transformer-based behavior cloning method. Our results demonstrate that employing latent representations yields enhanced performance compared to conventional behavior cloning methods, particularly regarding resilience to errors and noise in perception and proprioception. Furthermore, the proposed approach solely relies on human demonstrations, eliminating the need for teleoperation and, therefore, accelerating the data acquisition process. Accurate inverse kinematics for fingertip retargeting ensures precise transfer from human hand data to the robot, facilitating effective learning and deployment of manipulation policies. Finally, the trained policies have been successfully transferred to a real-world 23Dof robotic system.

Development of Musculoskeletal Legs with Planar Interskeletal Structures to Realize Human Comparable Moving Function

Musculoskeletal humanoids have been developed by imitating humans and expected to perform natural and dynamic motions as well as humans. To achieve desired motions stably in current musculoskeletal humanoids is not easy because they cannot maintain the sufficient moment arm of muscles in various postures. In this research, we discuss planar structures that spread across joint structures such as ligament and planar muscles and the application of planar interskeletal structures to humanoid robots. Next, we develop MusashiOLegs, a musculoskeletal legs which has planar interskeletal structures and conducts several experiments to verify the importance of planar interskeletal structures.

Hardware Design and Learning-Based Software Architecture of Musculoskeletal Wheeled Robot Musashi-W for Real-World Applications

Various musculoskeletal humanoids have been developed so far. While these humanoids have the advantage of their flexible and redundant bodies that mimic the human body, they are still far from being applied to real-world tasks. One of the reasons for this is the difficulty of bipedal walking in a flexible body. Thus, we developed a musculoskeletal wheeled robot, Musashi-W, by combining a wheeled base and musculoskeletal upper limbs for real-world applications. Also, we constructed its software system by combining static and dynamic body schema learning, reflex control, and visual recognition. We show that the hardware and software of Musashi-W can make the most of the advantages of the musculoskeletal upper limbs, through several tasks of cleaning by human teaching, carrying a heavy object considering muscle addition, and setting a table through dynamic cloth manipulation with variable stiffness.

RAMIEL: A Parallel-Wire Driven Monopedal Robot for High and Continuous Jumping

Legged robots with high locomotive performance have been extensively studied, and various leg structures have been proposed. Especially, a leg structure that can achieve both continuous and high jumps is advantageous for moving around in a three-dimensional environment. In this study, we propose a parallel wire-driven leg structure, which has one DoF of linear motion and two DoFs of rotation and is controlled by six wires, as a structure that can achieve both continuous jumping and high jumping. The proposed structure can simultaneously achieve high controllability on each DoF, long acceleration distance and high power required for jumping. In order to verify the jumping performance of the parallel wire-driven leg structure, we have developed a parallel wire-driven monopedal robot, RAMIEL. RAMIEL is equipped with quasi-direct drive, high power wire winding mechanisms and a lightweight leg, and can achieve a maximum jumping height of 1.6 m and a maximum of seven continuous jumps.

Real Robot Challenge 2022: Learning Dexterous Manipulation from Offline Data in the Real World

Experimentation on real robots is demanding in terms of time and costs. For this reason, a large part of the reinforcement learning (RL) community uses simulators to develop and benchmark algorithms. However, insights gained in simulation do not necessarily translate to real robots, in particular for tasks involving complex interactions with the environment. The Real Robot Challenge 2022 therefore served as a bridge between the RL and robotics communities by allowing participants to experiment remotely with a real robot - as easily as in simulation. In the last years, offline reinforcement learning has matured into a promising paradigm for learning from pre-collected datasets, alleviating the reliance on expensive online interactions. We therefore asked the participants to learn two dexterous manipulation tasks involving pushing, grasping, and in-hand orientation from provided real-robot datasets. An extensive software documentation and an initial stage based on a simulation of the real set-up made the competition particularly accessible. By giving each team plenty of access budget to evaluate their offline-learned policies on a cluster of seven identical real TriFinger platforms, we organized an exciting competition for machine learners and roboticists alike. In this work we state the rules of the competition, present the methods used by the winning teams and compare their results with a benchmark of state-of-the-art offline RL algorithms on the challenge datasets.

Getting the Ball Rolling: Learning a Dexterous Policy for a Biomimetic Tendon-Driven Hand with Rolling Contact Joints

Biomimetic, dexterous robotic hands have the potential to replicate much of the tasks that a human can do, and to achieve status as a general manipulation platform. Recent advances in reinforcement learning (RL) frameworks have achieved remarkable performance in quadrupedal locomotion and dexterous manipulation tasks. Combined with GPU-based highly parallelized simulations capable of simulating thousands of robots in parallel, RL-based controllers have become more scalable and approachable. However, in order to bring RL-trained policies to the real world, we require training frameworks that output policies that can work with physical actuators and sensors as well as a hardware platform that can be manufactured with accessible materials yet is robust enough to run interactive policies. This work introduces the biomimetic tendon-driven Faive Hand and its system architecture, which uses tendon-driven rolling contact joints to achieve a 3D printable, robust high-DoF hand design. We model each element of the hand and integrate it into a GPU simulation environment to train a policy with RL, and achieve zero-shot transfer of a dexterous in-hand sphere rotation skill to the physical robot hand.

Dynamic Control of Soft Manipulators to Perform Real-World Tasks

Dynamic motions are a key feature of robotic arms, enabling them to perform tasks quickly and efficiently. Soft continuum manipulators do not currently consider dynamic parameters when operating in task space. This shortcoming makes existing soft robots slow and limits their ability to deal with external forces, especially during object manipulation. We address this issue by using dynamic operational space control. Our control approach takes into account the dynamic parameters of the 3D continuum arm and introduces new models that enable multi-segment soft manipulators to operate smoothly in task space. Advanced control methods, previously afforded only to rigid robots, are now adapted to soft robots; for example, potential field avoidance was previously only shown for rigid robots and is now extended to soft robots. Using our approach, a soft manipulator can now achieve a variety of tasks that were previously not possible: we evaluate the manipulator's performance in closed-loop controlled experiments such as pick-and-place, obstacle avoidance, throwing objects using an attached soft gripper, and deliberately applying forces to a surface by drawing with a grasped piece of chalk. Besides the newly enabled skills, our approach improves tracking accuracy by 59% and increases speed by a factor of 19.3 compared to state of the art for task space control. With these newfound abilities, soft robots can start to challenge rigid robots in the field of manipulation. Our inherently safe and compliant soft robot moves the future of robotic manipulation towards a cageless setup where humans and robots work in parallel.

A Robust Adaptive Approach to Dynamic Control of Soft Continuum Manipulators

Soft robots are made of compliant and deformable materials and can perform tasks challenging for conventional rigid robots. The inherent compliance of soft robots makes them more suitable and adaptable for interactions with humans and the environment. However, this preeminence comes at a cost: their continuum nature makes it challenging to develop robust model-based control strategies. Specifically, an adaptive control approach addressing this challenge has not yet been applied to physical soft robotic arms. This work presents a reformulation of dynamics for a soft continuum manipulator using the Euler-Lagrange method. The proposed model eliminates the simplifying assumption made in previous works and provides a more accurate description of the robot's inertia. Based on our model, we introduce a task-space adaptive control scheme. This controller is robust against model parameter uncertainties and unknown input disturbances. The controller is implemented on a physical soft continuum arm. A series of experiments were carried out to validate the effectiveness of the controller in task-space trajectory tracking under different payloads. The controller outperforms the state-of-the-art method both in terms of accuracy and robustness. Moreover, the proposed model-based control design is flexible and can be generalized to any continuum robotic arm with an arbitrary number of continuum segments.

SoPrA: Fabrication & Dynamical Modeling of a Scalable Soft Continuum Robotic Arm with Integrated Proprioceptive Sensing

Due to their inherent compliance, soft robots are more versatile than rigid linked robots when they interact with their environment, such as object manipulation or biomimetic motion, and considered the key element in introducing robots to everyday environments. Although various soft robotic actuators exist, past research has focused primarily on designing and analyzing single components. Limited effort has been made to combine each component to create an overall capable, integrated soft robot. Ideally, the behavior of such a robot can be accurately modeled, and its motion within an environment uses its proprioception, without requiring external sensors. This work presents a design and modeling process for a Soft continuum Proprioceptive Arm (SoPrA) actuated by pneumatics. The integrated design is suitable for an analytical model due to its internal capacitive flex sensor for proprioceptive measurements and its fiber-reinforced fluidic elastomer actuators. The proposed analytical dynamical model accounts for the inertial effects of the actuator's mass and the material properties, and predicts in real-time the soft robot's behavior. Our estimation method integrates the analytical model with proprioceptive sensors to calculate external forces, all without relying on an external motion capture system. SoPrA is validated in a series of experiments demonstrating the model's and sensor's accuracy in estimation. SoPrA will enable soft arm manipulation including force sensing while operating in obstructed environments that disallows exteroceptive measurements.