Re4MPC: Reactive Nonlinear MPC for Multi-model Motion Planning via Deep Reinforcement Learning

Traditional motion planning methods for robots with many degrees-of-freedom, such as mobile manipulators, are often computationally prohibitive for real-world settings. In this paper, we propose a novel multi-model motion planning pipeline, termed Re4MPC, which computes trajectories using Nonlinear Model Predictive Control (NMPC). Re4MPC generates trajectories in a computationally efficient manner by reactively selecting the model, cost, and constraints of the NMPC problem depending on the complexity of the task and robot state. The policy for this reactive decision-making is learned via a Deep Reinforcement Learning (DRL) framework. We introduce a mathematical formulation to integrate NMPC into this DRL framework. To validate our methodology and design choices, we evaluate DRL training and test outcomes in a physics-based simulation involving a mobile manipulator. Experimental results demonstrate that Re4MPC is more computationally efficient and achieves higher success rates in reaching end-effector goals than the NMPC baseline, which computes whole-body trajectories without our learning mechanism.

Deep Reinforcement Learning based Robot Navigation in Dynamic Environments using Occupancy Values of Motion Primitives

This paper presents a Deep Reinforcement Learning based navigation approach in which we define the occupancy observations as heuristic evaluations of motion primitives, rather than using raw sensor data. Our method enables fast mapping of the occupancy data, generated by multi-sensor fusion, into trajectory values in 3D workspace. The computationally efficient trajectory evaluation allows dense sampling of the action space. We utilize our occupancy observations in different data structures to analyze their effects on both training process and navigation performance. We train and test our methodology on two different robots within challenging physics-based simulation environments including static and dynamic obstacles. We benchmark our occupancy representations with other conventional data structures from state-of-the-art methods. The trained navigation policies are also validated successfully with physical robots in dynamic environments. The results show that our method not only decreases the required training time but also improves the navigation performance as compared to other occupancy representations. The open-source implementation of our work and all related info are available at \url{https://github.com/RIVeR-Lab/tentabot}.

Reactive Navigation Framework for Mobile Robots by Heuristically Evaluated Pre-sampled Trajectories

This paper describes and analyzes a reactive navigation framework for mobile robots in unknown environments. The approach does not rely on a global map and only considers the local occupancy in its robot-centered 3D grid structure. The proposed algorithm enables fast navigation by heuristic evaluations of pre-sampled trajectories on-the-fly. At each cycle, these paths are evaluated by a weighted cost function, based on heuristic features such as closeness to the goal, previously selected trajectories, and nearby obstacles. This paper introduces a systematic method to calculate a feasible pose on the selected trajectory, before sending it to the controller for the motion execution. Defining the structures in the framework and providing the implementation details, the paper also explains how to adjust its offline and online parameters. To demonstrate the versatility and adaptability of the algorithm in unknown environments, physics-based simulations on various maps are presented. Benchmark tests show the superior performance of the proposed algorithm over its previous iteration and another state-of-art method. The open-source implementation of the algorithm and the benchmark data can be found at \url{https://github.com/RIVeR-Lab/tentabot}.

A 3D Reactive Navigation Algorithm for Mobile Robots by Using Tentacle-Based Sampling

This paper introduces a reactive navigation framework for mobile robots in 3-dimensional (3D) space. The proposed approach does not rely on the global map information and achieves fast navigation by employing a tentacle based sampling and their heuristic evaluations on-the-fly. This reactive nature of the approach comes from the prior arrangement of navigation points on tentacles (parametric contours) to sample the navigation space. These tentacles are evaluated at each time-step, based on heuristic features such as closeness to the goal, previous tentacle preferences and nearby obstacles in a robot-centered 3D grid. Then, the navigable sampling point on the selected tentacle is passed to a controller for the motion execution. The proposed framework does not only extend its 2D tentacle-based counterparts into 3D, but also introduces offline and online parameters, whose tuning provides versatility and adaptability of the algorithm to work in unknown environments. To demonstrate the superior performance of the proposed algorithm over a state-of-art method, the statistical results from physics-based simulations on various maps are presented. The video of the work is available at https://youtu.be/rrF7wHCz-0M.