Retro-RL: Reinforcing Nominal Controller With Deep Reinforcement Learning for Tilting-Rotor Drones

Studies that broaden drone applications into complex tasks require a stable control framework. Recently, deep reinforcement learning (RL) algorithms have been exploited in many studies for robot control to accomplish complex tasks. Unfortunately, deep RL algorithms might not be suitable for being deployed directly into a real-world robot platform due to the difficulty in interpreting the learned policy and lack of stability guarantee, especially for a complex task such as a wall-climbing drone. This paper proposes a novel hybrid architecture that reinforces a nominal controller with a robust policy learned using a model-free deep RL algorithm. The proposed architecture employs an uncertainty-aware control mixer to preserve guaranteed stability of a nominal controller while using the extended robust performance of the learned policy. The policy is trained in a simulated environment with thousands of domain randomizations to achieve robust performance over diverse uncertainties. The performance of the proposed method was verified through real-world experiments and then compared with a conventional controller and the state-of-the-art learning-based controller trained with a vanilla deep RL algorithm.

Development and Analysis of Digging and Soil Removing Mechanisms for Mole-Bot: Bio-Inspired Mole-Like Drilling Robot

Interests in exploration of new energy resources are increasing due to the exhaustion of existing resources. To explore new energy sources, various studies have been conducted to improve the drilling performance of drilling equipment for deep and strong ground. However, with better performance, the modern drilling equipment is bulky and, furthermore, has become inconvenient in both installation and operation, for it takes complex procedures for complex terrains. Moreover, environmental issues are also a concern because of the excessive use of mud and slurry to remove excavated soil. To overcome these limitations, a mechanism that combines an expandable drill bit and link structure to simulate the function of the teeth and forelimbs of a mole is proposed. In this paper, the proposed expandable drill bit simplifies the complexity and high number of degrees of freedom of the animal head. In addition, a debris removal mechanism mimicking a shoulder structure and forefoot movement is proposed. For efficient debris removal, the proposed mechanism enables the simultaneous rotation and expanding/folding motions of the drill bit by using a single actuator. The performance of the proposed system is evaluated by dynamic simulations and experiments.