Alert button
Picture for Guido Herrmann

Guido Herrmann

Alert button

Sim-and-Real Reinforcement Learning for Manipulation: A Consensus-based Approach

Feb 26, 2023
Wenxing Liu, Hanlin Niu, Wei Pan, Guido Herrmann, Joaquin Carrasco

Figure 1 for Sim-and-Real Reinforcement Learning for Manipulation: A Consensus-based Approach
Figure 2 for Sim-and-Real Reinforcement Learning for Manipulation: A Consensus-based Approach
Figure 3 for Sim-and-Real Reinforcement Learning for Manipulation: A Consensus-based Approach
Figure 4 for Sim-and-Real Reinforcement Learning for Manipulation: A Consensus-based Approach

Sim-and-real training is a promising alternative to sim-to-real training for robot manipulations. However, the current sim-and-real training is neither efficient, i.e., slow convergence to the optimal policy, nor effective, i.e., sizeable real-world robot data. Given limited time and hardware budgets, the performance of sim-and-real training is not satisfactory. In this paper, we propose a Consensus-based Sim-And-Real deep reinforcement learning algorithm (CSAR) for manipulator pick-and-place tasks, which shows comparable performance in both sim-and-real worlds. In this algorithm, we train the agents in simulators and the real world to get the optimal policies for both sim-and-real worlds. We found two interesting phenomenons: (1) Best policy in simulation is not the best for sim-and-real training. (2) The more simulation agents, the better sim-and-real training. The experimental video is available at: https://youtu.be/mcHJtNIsTEQ.

* 7 pages, 8 figures, IEEE International Conference on Robotics and Automation (ICRA) 2023 
Viaarxiv icon

Drivers' Manoeuvre Modelling and Prediction for Safe HRI

Jun 03, 2021
Erwin Jose Lopez Pulgarin, Guido Herrmann, Ute Leonards

Figure 1 for Drivers' Manoeuvre Modelling and Prediction for Safe HRI
Figure 2 for Drivers' Manoeuvre Modelling and Prediction for Safe HRI
Figure 3 for Drivers' Manoeuvre Modelling and Prediction for Safe HRI
Figure 4 for Drivers' Manoeuvre Modelling and Prediction for Safe HRI

As autonomous machines such as robots and vehicles start performing tasks involving human users, ensuring a safe interaction between them becomes an important issue. Translating methods from human-robot interaction (HRI) studies to the interaction between humans and other highly complex machines (e.g. semi-autonomous vehicles) could help advance the use of those machines in scenarios requiring human interaction. One method involves understanding human intentions and decision-making to estimate the human's present and near-future actions whilst interacting with a robot. This idea originates from the psychological concept of Theory of Mind, which has been broadly explored for robotics and recently for autonomous and semi-autonomous vehicles. In this work, we explored how to predict human intentions before an action is performed by combining data from human-motion, vehicle-state and human inputs (e.g. steering wheel, pedals). A data-driven approach based on Recurrent Neural Network models was used to classify the current driving manoeuvre and to predict the future manoeuvre to be performed. A state-transition model was used with a fixed set of manoeuvres to label data recorded during the trials for real-time applications. Models were trained and tested using drivers of different seat preferences, driving expertise and arm-length; precision and recall metrics over 95% for manoeuvre identification and 86% for manoeuvre prediction were achieved, with prediction time-windows of up to 1 second for both known and unknown test subjects. Compared to our previous results, performance improved and manoeuvre prediction was possible for unknown test subjects without knowing the current manoeuvre.

* Submitted to IEEE Transactions on Human-Machine Systems on 17-Jul-2020 
Viaarxiv icon