Safe Model-Based Reinforcement Learning Using Robust Control Barrier Functions

Reinforcement Learning (RL) is effective in many scenarios. However, it typically requires the exploration of a sufficiently large number of state-action pairs, some of which may be unsafe. Consequently, its application to safety-critical systems remains a challenge. Towards this end, an increasingly common approach to address safety involves the addition of a safety layer that projects the RL actions onto a safe set of actions. In turn, a challenge for such frameworks is how to effectively couple RL with the safety layer to improve the learning performance. In the context of leveraging control barrier functions for safe RL training, prior work focuses on a restricted class of barrier functions and utilizes an auxiliary neural net to account for the effects of the safety layer which inherently results in an approximation. In this paper, we frame safety as a differentiable robust-control-barrier-function layer in a model-based RL framework. As such, this approach both ensures safety and effectively guides exploration during training resulting in increased sample efficiency as demonstrated in the experiments.

Data-Driven Robust Barrier Functions for Safe, Long-Term Operation

Applications that require multi-robot systems to operate independently for extended periods of time in unknown or unstructured environments face a broad set of challenges, such as hardware degradation, changing weather patterns, or unfamiliar terrain. To operate effectively under these changing conditions, algorithms developed for long-term autonomy applications require a stronger focus on robustness. Consequently, this work considers the ability to satisfy the operation-critical constraints of a disturbed system in a modular fashion, which means compatibility with different system objectives and disturbance representations. Toward this end, this paper introduces a controller-synthesis approach to constraint satisfaction for disturbed control-affine dynamical systems by utilizing Control Barrier Functions (CBFs). The aforementioned framework is constructed by modelling the disturbance as a union of convex hulls and leveraging previous work on CBFs for differential inclusions. This method of disturbance modeling grants compatibility with different disturbance-estimation methods. For example, this work demonstrates how a disturbance learned via a Gaussian process may be utilized in the proposed framework. These estimated disturbances are incorporated into the proposed controller-synthesis framework which is then tested on a fleet of robots in different scenarios.

Data-Driven Adaptive Task Allocation for Heterogeneous Multi-Robot Teams Using Robust Control Barrier Functions

Multi-robot task allocation is a ubiquitous problem in robotics due to its applicability in a variety of scenarios. Adaptive task-allocation algorithms account for unknown disturbances and unpredicted phenomena in the environment where robots are deployed to execute tasks. However, this adaptivity typically comes at the cost of requiring precise knowledge of robot models in order to evaluate the allocation effectiveness and to adjust the task assignment online. As such, environmental disturbances can significantly degrade the accuracy of the models which in turn negatively affects the quality of the task allocation. In this paper, we leverage Gaussian processes, differential inclusions, and robust control barrier functions to learn environmental disturbances in order to guarantee robust task execution. We show the implementation and the effectiveness of the proposed framework on a real multi-robot system.

Robust Barrier Functions for a Fully Autonomous, Remotely Accessible Swarm-Robotics Testbed

The Robotarium, a remotely accessible swarm-robotics testbed, has provided free, open access to robotics and controls research for hundreds of users in thousands of experiments. This high level of usage requires autonomy in the system, which mainly corresponds to constraint satisfaction in the context of users' submissions. In other words, in case that the users' inputs to the robots may lead to collisions, these inputs must be altered to avoid these collisions automatically. However, these alterations must be minimal so as to preserve the users' objective in the experiment. Toward this end, the system has utilized barrier functions, which admit a minimally invasive controller-synthesis procedure. However, barrier functions are yet to be robustified with respect to unmodeled disturbances (e.g., wheel slip or packet loss) in a manner conducive to real-time synthesis. As such, this paper formulates robust barrier functions for a general class of disturbed control-affine systems that, in turn, is key for the Robotarium to operate fully autonomously (i.e., without human supervision). Experimental results showcase the effectiveness of this robust formulation in a long-term experiment in the Robotarium.

A Parametric MPC Approach to Balancing the Cost of Abstraction for Differential-Drive Mobile Robots

When designing control strategies for differential-drive mobile robots, one standard tool is the consideration of a point at a fixed distance along a line orthogonal to the wheel axis instead of the full pose of the vehicle. This abstraction supports replacing the non-holonomic, three-state unicycle model with a much simpler two-state single-integrator model (i.e., a velocity-controlled point). Yet this transformation comes at a performance cost, through the robot's precision and maneuverability. This work contains derivations for expressions of these precision and maneuverability costs in terms of the transformation's parameters. Furthermore, these costs show that only selecting the parameter once over the course of an application may cause an undue loss of precision. Model Predictive Control (MPC) represents one such method to ameliorate this condition. However, MPC typically realizes a control signal, rather than a parameter, so this work also proposes a Parametric Model Predictive Control (PMPC) method for parameter and sampling horizon optimization. Experimental results are presented that demonstrate the effects of the parameterization on the deployment of algorithms developed for the single-integrator model on actual differential-drive mobile robots.

The Robotarium: A remotely accessible swarm robotics research testbed

This paper describes the Robotarium -- a remotely accessible, multi-robot research facility. The impetus behind the Robotarium is that multi-robot testbeds constitute an integral and essential part of the multi-robot research cycle, yet they are expensive, complex, and time-consuming to develop, operate, and maintain. These resource constraints, in turn, limit access for large groups of researchers and students, which is what the Robotarium is remedying by providing users with remote access to a state-of-the-art multi-robot test facility. This paper details the design and operation of the Robotarium and discusses the considerations one must take when making complex hardware remotely accessible. In particular, safety must be built into the system already at the design phase without overly constraining what coordinated control programs users can upload and execute, which calls for minimally invasive safety routines with provable performance guarantees.

Safe, Remote-Access Swarm Robotics Research on the Robotarium

This paper describes the development of the Robotarium -- a remotely accessible, multi-robot research facility. The impetus behind the Robotarium is that multi-robot testbeds constitute an integral and essential part of the multi-agent research cycle, yet they are expensive, complex, and time-consuming to develop, operate, and maintain. These resource constraints, in turn, limit access for large groups of researchers and students, which is what the Robotarium is remedying by providing users with remote access to a state-of-the-art multi-robot test facility. This paper details the design and operation of the Robotarium as well as connects these to the particular considerations one must take when making complex hardware remotely accessible. In particular, safety must be built in already at the design phase without overly constraining which coordinated control programs the users can upload and execute, which calls for minimally invasive safety routines with provable performance guarantees.