Simultaneously Learning Vision and Feature-based Control Policies for Real-world Ball-in-a-Cup

We present a method for fast training of vision based control policies on real robots. The key idea behind our method is to perform multi-task Reinforcement Learning with auxiliary tasks that differ not only in the reward to be optimized but also in the state-space in which they operate. In particular, we allow auxiliary task policies to utilize task features that are available only at training-time. This allows for fast learning of auxiliary policies, which subsequently generate good data for training the main, vision-based control policies. This method can be seen as an extension of the Scheduled Auxiliary Control (SAC-X) framework. We demonstrate the efficacy of our method by using both a simulated and real-world Ball-in-a-Cup game controlled by a robot arm. In simulation, our approach leads to significant learning speed-ups when compared to standard SAC-X. On the real robot we show that the task can be learned from-scratch, i.e., with no transfer from simulation and no imitation learning. Videos of our learned policies running on the real robot can be found at https://sites.google.com/view/rss-2019-sawyer-bic/.

Model Based In Situ Calibration with Temperature compensation of 6 axis Force Torque Sensors

It is well known that sensors using strain gauges have a potential dependency on temperature. This creates temperature drift in the measurements of six axis force torque sensors (F/T). The temperature drift can be considerable if an experiment is long or the environmental conditions are different from when the calibration of the sensor was performed. Other \textit{in situ} methods disregard the effect of temperature on the sensor measurements. Experiments performed using the humanoid robot platform iCub show that the effect of temperature is relevant. The model based \textit{in situ} calibration of six axis force torque sensors method is extended to perform temperature compensation.

A Control Architecture with Online Predictive Planning for Position and Torque Controlled Walking of Humanoid Robots

A common approach to the generation of walking patterns for humanoid robots consists in adopting a layered control architecture. This paper proposes an architecture composed of three nested control loops. The outer loop exploits a robot kinematic model to plan the footstep positions. In the mid layer, a predictive controller generates a Center of Mass trajectory according to the well-known table-cart model. Through a whole-body inverse kinematics algorithm, we can define joint references for position controlled walking. The outcomes of these two loops are then interpreted as inputs of a stack-of-task QP-based torque controller, which represents the inner loop of the presented control architecture. This resulting architecture allows the robot to walk also in torque control, guaranteeing higher level of compliance. Real world experiments have been carried on the humanoid robot iCub.

Modeling and Control of Humanoid Robots in Dynamic Environments: iCub Balancing on a Seesaw

Forthcoming applications concerning humanoid robots may involve physical interaction between the robot and a dynamic environment. In such scenario, classical balancing and walking controllers that neglect the environment dynamics may not be sufficient for achieving a stable robot behavior. This paper presents a modeling and control framework for balancing humanoid robots in contact with a dynamic environment. We first model the robot and environment dynamics, together with the contact constraints. Then, a control strategy for stabilizing the full system is proposed. Theoretical results are verified in simulation with robot iCub balancing on a seesaw.

Contact Force and Joint Torque Estimation Using Skin

In this paper, we present algorithms to estimate external contact forces and joint torques using only skin, i.e. distributed tactile sensors. To deal with gaps between the tactile sensors (taxels), we use interpolation techniques. The application of these interpolation techniques allows us to estimate contact forces and joint torques without the need for expensive force-torque sensors. Validation was performed using the iCub humanoid robot.

A Predictive Momentum-Based Whole-Body Torque Controller: Theory and Simulations for the iCub Stepping

When balancing, a humanoid robot can be easily subjected to unexpected disturbances like external pushes. In these circumstances, reactive movements as steps become a necessary requirement in order to avoid potentially harmful falling states. In this paper we conceive a Model Predictive Controller which determines a desired set of contact wrenches by predicting the future evolution of the robot, while taking into account constraints switching in case of steps. The control inputs computed by this strategy, namely the desired contact wrenches, are directly obtained on the robot through a modification of the momentum-based whole-body torque controller currently implemented on iCub. The proposed approach is validated through simulations in a stepping scenario, revealing high robustness and reliability when executing a recovery strategy.

On-line Joint Limit Avoidance for Torque Controlled Robots by Joint Space Parametrization

This paper proposes control laws ensuring the stabilization of a time-varying desired joint trajectory, as well as joint limit avoidance, in the case of fully-actuated manipulators. The key idea is to perform a parametrization of the feasible joint space in terms of exogenous states. It follows that the control of these states allows for joint limit avoidance. One of the main outcomes of this paper is that position terms in control laws are replaced by parametrized terms, where joint limits must be avoided. Stability and convergence of time-varying reference trajectories obtained with the proposed method are demonstrated to be in the sense of Lyapunov. The introduced control laws are verified by carrying out experiments on two degrees-of-freedom of the humanoid robot iCub.

An Optimization Based Control Framework for Balancing and Walking: Implementation on the iCub Robot

A whole-body torque control framework adapted for balancing and walking tasks is presented in this paper. In the proposed approach, centroidal momentum terms are excluded in favor of a hierarchy of high-priority position and orientation tasks and a low-priority postural task. More specifically, the controller stabilizes the position of the center of mass, the orientation of the pelvis frame, as well as the position and orientation of the feet frames. The low-priority postural task provides reference positions for each joint of the robot. Joint torques and contact forces to stabilize tasks are obtained through quadratic programming optimization. Besides the exclusion of centroidal momentum terms, part of the novelty of the approach lies in the definition of control laws in SE(3) which do not require the use of Euler parameterization. Validation of the framework was achieved in a scenario where the robot kept balance while walking in place. Experiments have been conducted with the iCub robot, in simulation and in real-world experiments.

Walking of the iCub humanoid robot in different scenarios: implementation and performance analysis

The humanoid robot iCub is a research platform of the Fondazione Istituto Italiano di Tecnologia (IIT), spread among different institutes around the world. In the most recent version of iCub, the robot is equipped with stronger legs and bigger feet, allowing it to perform balancing and walking motions that were not possible with the first generations. Despite the new legs hardware, walking has been rarely performed on the iCub robot. In this work the objective is to implement walking motions on the robot, from which we want to analyze its walking capabilities. We developed software modules based on extensions of classic techniques such as the ZMP based pattern generator and position control to identify which are the characteristics as well as limitations of the robot against different walking tasks in order to give the users a reference of the performance of the robot. Most of the experiments have been performed with HeiCub, a reduced version of iCub without arms and head.

Stability Analysis and Design of Momentum-based Controllers for Humanoid Robots

Envisioned applications for humanoid robots call for the design of balancing and walking controllers. While promising results have been recently achieved, robust and reliable controllers are still a challenge for the control community dealing with humanoid robotics. Momentum-based strategies have proven their effectiveness for controlling humanoids balancing, but the stability analysis of these controllers is still missing. The contribution of this paper is twofold. First, we numerically show that the application of state-of-the-art momentum-based control strategies may lead to unstable zero dynamics. Secondly, we propose simple modifications to the control architecture that avoid instabilities at the zero-dynamics level. Asymptotic stability of the closed loop system is shown by means of a Lyapunov analysis on the linearized system's joint space. The theoretical results are validated with both simulations and experiments on the iCub humanoid robot.