A CBF-Adaptive Control Architecture for Visual Navigation for UAV in the Presence of Uncertainties

In this article, we propose a control solution for the safe transfer of a quadrotor UAV between two surface robots positioning itself only using the visual features on the surface robots, which enforces safety constraints for precise landing and visual locking, in the presence of modeling uncertainties and external disturbances. The controller handles the ascending and descending phases of the navigation using a visual locking control barrier function (VCBF) and a parametrizable switching descending CBF (DCBF) respectively, eliminating the need for an external planner. The control scheme has a backstepping approach for the position controller with the CBF filter acting on the position kinematics to produce a filtered virtual velocity control input, which is tracked by an adaptive controller to overcome modeling uncertainties and external disturbances. The experimental validation is carried out with a UAV that navigates from the base to the target using an RGB camera.

Environmental Awareness Dynamic 5G QoS for Retaining Real Time Constraints in Robotic Applications

The fifth generation (5G) cellular network technology is mature and increasingly utilized in many industrial and robotics applications, while an important functionality is the advanced Quality of Service (QoS) features. Despite the prevalence of 5G QoS discussions in the related literature, there is a notable absence of real-life implementations and studies concerning their application in time-critical robotics scenarios. This article considers the operation of time-critical applications for 5G-enabled unmanned aerial vehicles (UAVs) and how their operation can be improved by the possibility to dynamically switch between QoS data flows with different priorities. As such, we introduce a robotics oriented analysis on the impact of the 5G QoS functionality on the performance of 5G-enabled UAVs. Furthermore, we introduce a novel framework for the dynamic selection of distinct 5G QoS data flows that is autonomously managed by the 5G-enabled UAV. This problem is addressed in a novel feedback loop fashion utilizing a probabilistic finite state machine (PFSM). Finally, the efficacy of the proposed scheme is experimentally validated with a 5G-enabled UAV in a real-world 5G stand-alone (SA) network.

Belief Scene Graphs: Expanding Partial Scenes with Objects through Computation of Expectation

In this article, we propose the novel concept of Belief Scene Graphs, which are utility-driven extensions of partial 3D scene graphs, that enable efficient high-level task planning with partial information. We propose a graph-based learning methodology for the computation of belief (also referred to as expectation) on any given 3D scene graph, which is then used to strategically add new nodes (referred to as blind nodes) that are relevant for a robotic mission. We propose the method of Computation of Expectation based on Correlation Information (CECI), to reasonably approximate real Belief/Expectation, by learning histograms from available training data. A novel Graph Convolutional Neural Network (GCN) model is developed, to learn CECI from a repository of 3D scene graphs. As no database of 3D scene graphs exists for the training of the novel CECI model, we present a novel methodology for generating a 3D scene graph dataset based on semantically annotated real-life 3D spaces. The generated dataset is then utilized to train the proposed CECI model and for extensive validation of the proposed method. We establish the novel concept of \textit{Belief Scene Graphs} (BSG), as a core component to integrate expectations into abstract representations. This new concept is an evolution of the classical 3D scene graph concept and aims to enable high-level reasoning for the task planning and optimization of a variety of robotics missions. The efficacy of the overall framework has been evaluated in an object search scenario, and has also been tested on a real-life experiment to emulate human common sense of unseen-objects.

Semantic and Topological Mapping using Intersection Identification

This article presents a novel approach to identifying and classifying intersections for semantic and topological mapping. More specifically, the proposed novel approach has the merit of generating a semantically meaningful map containing intersections, pathways, dead ends, and pathways leading to unexplored frontiers. Furthermore, the resulting semantic map can be used to generate a sparse topological map representation, that can be utilized by robots for global navigation. The proposed solution also introduces a built-in filtering to handle noises in the environment, to remove openings in the map that the robot cannot pass, and to remove small objects to optimize and simplify the overall mapping results. The efficacy of the proposed semantic and topological mapping method is demonstrated over a map of an indoor structured environment that is built from experimental data. The proposed framework, when compared with similar state-of-the-art topological mapping solutions, is able to produce a map with up to 89% fewer nodes than the next best solution.

Event Camera and LiDAR based Human Tracking for Adverse Lighting Conditions in Subterranean Environments

In this article, we propose a novel LiDAR and event camera fusion modality for subterranean (SubT) environments for fast and precise object and human detection in a wide variety of adverse lighting conditions, such as low or no light, high-contrast zones and in the presence of blinding light sources. In the proposed approach, information from the event camera and LiDAR are fused to localize a human or an object-of-interest in a robot's local frame. The local detection is then transformed into the inertial frame and used to set references for a Nonlinear Model Predictive Controller (NMPC) for reactive tracking of humans or objects in SubT environments. The proposed novel fusion uses intensity filtering and K-means clustering on the LiDAR point cloud and frequency filtering and connectivity clustering on the events induced in an event camera by the returning LiDAR beams. The centroids of the clusters in the event camera and LiDAR streams are then paired to localize reflective markers present on safety vests and signs in SubT environments. The efficacy of the proposed scheme has been experimentally validated in a real SubT environment (a mine) with a Pioneer 3AT mobile robot. The experimental results show real-time performance for human detection and the NMPC-based controller allows for reactive tracking of a human or object of interest, even in complete darkness.

Reactive Task Allocation for Balanced Servicing of Multiple Task Queues

In this article, we propose a reactive task allocation architecture for a multi-agent system for scenarios where the tasks arrive at random times and are grouped into multiple queues. Two stage tasks are considered where every task has a beginning, an intermediate and a final part, typical in pick-and-drop and inspect-and-report scenarios. A centralized auction-based task allocation system is proposed, where an auction system takes into consideration bids submitted by the agents for individual tasks, current length of the queues and the waiting times of the tasks in the queues to decide on a task allocation strategy. The costs associated with these considerations, along with the constraints of having unique mappings between tasks and agents and constraints on the maximum number of agents that can be assigned to a queue, results in a Linear Integer Program (LIP) that is solved using the SCIP solver. For the scenario where the queue lengths are penalized but not the waiting times, we demonstrate that the auction system allocates tasks in a manner that all the queue lengths become constant, which is termed balancing. For the scenarios where both the costs are considered, we qualitatively analyse the effect of the choice of the relative weights on the resulting task allocation and provide guidelines for the choice of the weights. We present simulation results that illustrate the balanced allocation of tasks and validate the analysis for the trade-off between the costs related to queue lengths and task waiting times.

Reactive Multi-agent Coordination using Auction-based Task Allocation and Behavior Trees

This article presents an architecture for multi-agent task allocation and task execution, through the unification of a market-inspired task-auctioning system with Behavior Trees for managing and executing lower level behaviors. We consider the scenario with multi-stage tasks, such as 'pick and place', whose arrival times are not known a priori. In such a scenario, a coordinating architecture is expected to be reactive to newly arrived tasks and the resulting rerouting of agents should be dependent on the stage of completion of their current multi-stage tasks. In the novel architecture proposed in this article, a central auctioning system gathers bids (cost-estimates for completing currently available tasks) from all agents, and solves a combinatorial problem to optimally assign tasks to agents. For every agent, it's participation in the auctioning system and execution of an assigned multi-stage task is managed using behavior trees, which switch among several well-defined behaviors in response to changing scenarios. The auctioning system is run at a fixed rate, allowing for newly added tasks to be incorporated into the auctioning system, which makes the solution reactive and allows for the rerouting of some agents (subject to the states of the behavior trees). We demonstrate that the proposed architecture is especially well-suited for multi-stage tasks, where high costs are incurred when rerouting agents who have completed one or more stages of their current tasks. The scalability analysis of the proposed architecture reveals that it scales well with the number of agents and number of tasks. The proposed framework is experimentally validated in multiple scenarios in a lab environment. A video of a demonstration can be viewed at: https://youtu.be/ZdEkoOOlB2g}.

Safe Autonomous Docking Maneuvers for a Floating Platform based on Input Sharing Control Barrier Functions

In this article, we present a control strategy for the problem of safe autonomous docking for a planar floating platform (Slider) that emulates the movement of a satellite. Employing the proposed strategy, Slider approaches a docking port with the right orientation, maintaining a safe distance, while always keeping a visual lock on the docking port throughout the docking maneuver. Control barrier functions are designed to impose the safety, direction of approach and visual locking constraints. Three control inputs of the Slider are shared among three barrier functions in enforcing the constraints. It is proved that the control inputs are shared in a conflict-free manner in rendering the sets defining safety and visual locking constraints forward invariant and in establishing finite-time convergence to the visual locking mode. The conflict-free input-sharing ensures the feasibility of a quadratic program that generates minimally-invasive corrections for a nominal controller, that is designed to track the docking port, so that the barrier constraints are respected throughout the docking maneuver. The efficacy of the proposed control design approach is validated through various simulations.