Geometric Fault-Tolerant Control of Quadrotors in Case of Rotor Failures: An Attitude Based Comparative Study

The ability of aerial robots to operate in the presence of failures is crucial in various applications that demand continuous operations, such as surveillance, monitoring, and inspection. In this paper, we propose a fault-tolerant control strategy for quadrotors that can adapt to single and dual complete rotor failures. Our approach augments a classic geometric tracking controller on $SO(3)\times\mathbb{R}^3$ to accommodate the effects of rotor failures. We provide an in-depth analysis of several attitude error metrics to identify the most appropriate design choice for fault-tolerant control strategies. To assess the effectiveness of these metrics, we evaluate trajectory tracking accuracies. Simulation results demonstrate the performance of the proposed approach.

AutoCharge: Autonomous Charging for Perpetual Quadrotor Missions

Battery endurance represents a key challenge for long-term autonomy and long-range operations, especially in the case of aerial robots. In this paper, we propose AutoCharge, an autonomous charging solution for quadrotors that combines a portable ground station with a flexible, lightweight charging tether and is capable of universal, highly efficient, and robust charging. We design and manufacture a pair of circular magnetic connectors to ensure a precise orientation-agnostic electrical connection between the ground station and the charging tether. Moreover, we supply the ground station with an electromagnet that largely increases the tolerance to localization and control errors during the docking maneuver, while still guaranteeing smooth un-docking once the charging process is completed. We demonstrate AutoCharge on a perpetual 10 hours quadrotor flight experiment and show that the docking and un-docking performance is solidly repeatable, enabling perpetual quadrotor flight missions.

Long-range UAV Thermal Geo-localization with Satellite Imagery

Onboard sensors, such as cameras and thermal sensors, have emerged as effective alternatives to Global Positioning System (GPS) for geo-localization in Unmanned Aerial Vehicle (UAV) navigation. Since GPS can suffer from signal loss and spoofing problems, researchers have explored camera-based techniques such as Visual Geo-localization (VG) using satellite imagery. Additionally, thermal geo-localization (TG) has become crucial for long-range UAV flights in low-illumination environments. This paper proposes a novel thermal geo-localization framework using satellite imagery, which includes multiple domain adaptation methods to address the limited availability of paired thermal and satellite images. The experimental results demonstrate the effectiveness of the proposed approach in achieving reliable thermal geo-localization performance, even in thermal images with indistinct self-similar features. We evaluate our approach on real data collected onboard a UAV. We also release the code and \textit{Boson-nighttime}, a dataset of paired satellite-thermal and unpaired satellite images for thermal geo-localization with satellite imagery. To the best of our knowledge, this work is the first to propose a thermal geo-localization method using satellite imagery in long-range flights.

BackpropTools: A Fast, Portable Deep Reinforcement Learning Library for Continuous Control

Deep Reinforcement Learning (RL) has been demonstrated to yield capable agents and control policies in several domains but is commonly plagued by prohibitively long training times. Additionally, in the case of continuous control problems, the applicability of learned policies on real-world embedded devices is limited due to the lack of real-time guarantees and portability of existing deep learning libraries. To address these challenges, we present BackpropTools, a dependency-free, header-only, pure C++ library for deep supervised and reinforcement learning. Leveraging the template meta-programming capabilities of recent C++ standards, we provide composable components that can be tightly integrated by the compiler. Its novel architecture allows BackpropTools to be used seamlessly on a heterogeneous set of platforms, from HPC clusters over workstations and laptops to smartphones, smartwatches, and microcontrollers. Specifically, due to the tight integration of the RL algorithms with simulation environments, BackpropTools can solve popular RL problems like the Pendulum-v1 swing-up about 7 to 15 times faster in terms of wall-clock training time compared to other popular RL frameworks when using TD3. We also provide a low-overhead and parallelized interface to the MuJoCo simulator, showing that our PPO implementation achieves state of the art returns in the Ant-v4 environment while achieving a 25 to 30 percent faster wall-clock training time. Finally, we also benchmark the policy inference on a diverse set of microcontrollers and show that in most cases our optimized inference implementation is much faster than even the manufacturer's DSP libraries. To the best of our knowledge, BackpropTools enables the first-ever demonstration of training a deep RL algorithm directly on a microcontroller, giving rise to the field of Tiny Reinforcement Learning (TinyRL). Project page: https://backprop.tools

PENet: A Joint Panoptic Edge Detection Network

In recent years, compact and efficient scene understanding representations have gained popularity in increasing situational awareness and autonomy of robotic systems. In this work, we illustrate the concept of a panoptic edge segmentation and propose PENet, a novel detection network called that combines semantic edge detection and instance-level perception into a compact panoptic edge representation. This is obtained through a joint network by multi-task learning that concurrently predicts semantic edges, instance centers and offset flow map without bounding box predictions exploiting the cross-task correlations among the tasks. The proposed approach allows extending semantic edge detection to panoptic edge detection which encapsulates both category-aware and instance-aware segmentation. We validate the proposed panoptic edge segmentation method and demonstrate its effectiveness on the real-world Cityscapes dataset.

GaPT: Gaussian Process Toolkit for Online Regression with Application to Learning Quadrotor Dynamics

Gaussian Processes (GPs) are expressive models for capturing signal statistics and expressing prediction uncertainty. As a result, the robotics community has gathered interest in leveraging these methods for inference, planning, and control. Unfortunately, despite providing a closed-form inference solution, GPs are non-parametric models that typically scale cubically with the dataset size, hence making them difficult to be used especially on onboard Size, Weight, and Power (SWaP) constrained aerial robots. In addition, the integration of popular libraries with GPs for different kernels is not trivial. In this paper, we propose GaPT, a novel toolkit that converts GPs to their state space form and performs regression in linear time. GaPT is designed to be highly compatible with several optimizers popular in robotics. We thoroughly validate the proposed approach for learning quadrotor dynamics on both single and multiple input GP settings. GaPT accurately captures the system behavior in multiple flight regimes and operating conditions, including those producing highly nonlinear effects such as aerodynamic forces and rotor interactions. Moreover, the results demonstrate the superior computational performance of GaPT compared to a classical GP inference approach on both single and multi-input settings especially when considering large number of data points, enabling real-time regression speed on embedded platforms used on SWaP-constrained aerial robots.

Nonlinear Model Predictive Control for Cooperative Transportation and Manipulation of Cable Suspended Payloads with Multiple Quadrotors

Autonomous Micro Aerial Vehicles (MAVs) such as quadrotors equipped with manipulation mechanisms have the potential to assist humans in tasks such as construction and package delivery. Cables are a promising option for manipulation mechanisms due to their low weight, low cost, and simple design. However, designing control and planning strategies for cable mechanisms presents challenges due to indirect load actuation, nonlinear configuration space, and highly coupled system dynamics. In this paper, we propose a novel Nonlinear Model Predictive Control (NMPC) method that enables a team of quadrotors to manipulate a rigid-body payload in all 6 degrees of freedom via suspended cables. Our approach can concurrently exploit, as part of the receding horizon optimization, the available mechanical system redundancies to perform additional tasks such as inter-robot separation and obstacle avoidance while respecting payload dynamics and actuator constraints. To address real-time computational requirements and scalability, we employ a lightweight state vector parametrization that includes only payload states in all six degrees of freedom. This also enables the planning of trajectories on the $SE(3)$ manifold load configuration space, thereby also reducing planning complexity. We validate the proposed approach through simulation and real-world experiments.

Active Learning of Discrete-Time Dynamics for Uncertainty-Aware Model Predictive Control

Model-based control requires an accurate model of the system dynamics for precisely and safely controlling the robot in complex and dynamic environments. Moreover, in presence of variations in the operating conditions, the model should be continuously refined to compensate for dynamics changes. In this paper, we propose a self-supervised learning approach to actively model robot discrete-time dynamics. We combine offline learning from past experience and online learning from present robot interaction with the unknown environment. These two ingredients enable highly sample-efficient and adaptive learning for accurate inference of the model dynamics in real-time even in operating regimes significantly different from the training distribution. Moreover, we design an uncertainty-aware model predictive controller that is conditioned to the aleatoric (data) uncertainty of the learned dynamics. The controller actively selects the optimal control actions that (i) optimize the control performance and (ii) boost the online learning sample efficiency. We apply the proposed method to a quadrotor system in multiple challenging real-world experiments. Our approach exhibits high flexibility and generalization capabilities by consistently adapting to unseen flight conditions, while it significantly outperforms classical and adaptive control baselines.

Safety-Aware Human-Robot Collaborative Transportation and Manipulation with Multiple MAVs

Human-robot interaction will play an essential role in many future industries and daily life tasks, enabling robots to collaborate with humans and reduce their workload effectively. Most existing approaches for human-robot physical collaboration focus on collaboration between humans and grounded robots. In recent years, very little progress has been made in this area when considering aerial robots, which present increased versatility and mobility compared to their grounded counterparts. This paper proposes a novel approach for safe human-robot collaborative transportation and manipulation of a cable-suspended payload with multiple aerial robots. We leverage the proposed method to enable seamless and transparent interaction between the transported objects and a human worker while considering safety constraints during operations by exploiting the redundancy of the internal transportation system. The critical system components are (a) a distributed force-sensor-free payload external wrench estimator; (b) a 6D admittance controller for human-aerial-robot collaborative transportation and manipulation; (c) a safety-aware controller that exploits the internal system redundancy to guarantee the execution of additional tasks devoted to preserving the human or robot safety without affecting the payload trajectory tracking or quality of interaction. We validate the approach through extensive real-world experiments, including the robot team assisting the human in transporting and manipulating a load or the human helping the robot team navigates the environment. To our best knowledge, this work is the first to create an interactive and safety-aware pipeline for quadrotor teams to collaborate physically with a human operator to transport and manipulate a payload.

Vision-based Perimeter Defense via Multiview Pose Estimation

Previous studies in the perimeter defense game have largely focused on the fully observable setting where the true player states are known to all players. However, this is unrealistic for practical implementation since defenders may have to perceive the intruders and estimate their states. In this work, we study the perimeter defense game in a photo-realistic simulator and the real world, requiring defenders to estimate intruder states from vision. We train a deep machine learning-based system for intruder pose detection with domain randomization that aggregates multiple views to reduce state estimation errors and adapt the defensive strategy to account for this. We newly introduce performance metrics to evaluate the vision-based perimeter defense. Through extensive experiments, we show that our approach improves state estimation, and eventually, perimeter defense performance in both 1-defender-vs-1-intruder games, and 2-defenders-vs-1-intruder games.