Barrier Functions for Multiagent-POMDPs with DTL Specifications

Multi-agent partially observable Markov decision processes (MPOMDPs) provide a framework to represent heterogeneous autonomous agents subject to uncertainty and partial observation. In this paper, given a nominal policy provided by a human operator or a conventional planning method, we propose a technique based on barrier functions to design a minimally interfering safety-shield ensuring satisfaction of high-level specifications in terms of linear distribution temporal logic (LDTL). To this end, we use sufficient and necessary conditions for the invariance of a given set based on discrete-time barrier functions (DTBFs) and formulate sufficient conditions for finite time DTBF to study finite time convergence to a set. We then show that different LDTL mission/safety specifications can be cast as a set of invariance or finite time reachability problems. We demonstrate that the proposed method for safety-shield synthesis can be implemented online by a sequence of one-step greedy algorithms. We demonstrate the efficacy of the proposed method using experiments involving a team of robots.

Data-driven Characterization of Human Interaction for Model-based Control of Powered Prostheses

This paper proposes a data-driven method for powered prosthesis control that achieves stable walking without the need for additional sensors on the human. The key idea is to extract the nominal gait and the human interaction information from motion capture data, and reconstruct the walking behavior with a dynamic model of the human-prosthesis system. The walking behavior of a human wearing a powered prosthesis is obtained through motion capture, which yields the limb and joint trajectories. Then a nominal trajectory is obtained by solving a gait optimization problem designed to reconstruct the walking behavior observed by motion capture. Moreover, the interaction force profiles between the human and the prosthesis are recovered by simulating the model following the recorded gaits, which are then used to construct a force tube that covers all the interaction force profiles. Finally, a robust Control Lyapunov Function (CLF) Quadratic Programming (QP) controller is designed to guarantee the convergence to the nominal trajectory under all possible interaction forces within the tube. Simulation results show this controller's improved tracking performance with a perturbed force profile compared to other control methods with less model information.

Human Preference-Based Learning for High-dimensional Optimization of Exoskeleton Walking Gaits

Understanding users' gait preferences of a lower-body exoskeleton requires optimizing over the high-dimensional gait parameter space. However, existing preference-based learning methods have only explored low-dimensional domains due to computational limitations. To learn user preferences in high dimensions, this work presents LineCoSpar, a human-in-the-loop preference-based framework that enables optimization over many parameters by iteratively exploring one-dimensional subspaces. Additionally, this work identifies gait attributes that characterize broader preferences across users. In simulations and human trials, we empirically verify that LineCoSpar is a sample-efficient approach for high-dimensional preference optimization. Our analysis of the experimental data reveals a correspondence between human preferences and objective measures of dynamic stability, while also highlighting inconsistencies in the utility functions underlying different users' gait preferences. This has implications for exoskeleton gait synthesis, an active field with applications to clinical use and patient rehabilitation.

Recurrent Neural Network Control of a Hybrid Dynamic Transfemoral Prosthesis with EdgeDRNN Accelerator

Lower leg prostheses could improve the life quality of amputees by increasing comfort and reducing energy to locomote, but currently control methods are limited in modulating behaviors based upon the human's experience. This paper describes the first steps toward learning complex controllers for dynamical robotic assistive devices. We provide the first example of behavioral cloning to control a powered transfemoral prostheses using a Gated Recurrent Unit (GRU) based recurrent neural network (RNN) running on a custom hardware accelerator that exploits temporal sparsity. The RNN is trained on data collected from the original prosthesis controller. The RNN inference is realized by a novel EdgeDRNN accelerator in real-time. Experimental results show that the RNN can replace the nominal PD controller to realize end-to-end control of the AMPRO3 prosthetic leg walking on flat ground and unforeseen slopes with comparable tracking accuracy. EdgeDRNN computes the RNN about 240 times faster than real time, opening the possibility of running larger networks for more complex tasks in the future. Implementing an RNN on this real-time dynamical system with impacts sets the ground work to incorporate other learned elements of the human-prosthesis system into prosthesis control.

Partially Observable Games for Secure Autonomy

Technology development efforts in autonomy and cyber-defense have been evolving independently of each other, over the past decade. In this paper, we report our ongoing effort to integrate these two presently distinct areas into a single framework. To this end, we propose the two-player partially observable stochastic game formalism to capture both high-level autonomous mission planning under uncertainty and adversarial decision making subject to imperfect information. We show that synthesizing sub-optimal strategies for such games is possible under finite-memory assumptions for both the autonomous decision maker and the cyber-adversary. We then describe an experimental testbed to evaluate the efficacy of the proposed framework.

Intermittent Connectivity for Exploration in Communication-Constrained Multi-Agent Systems

Motivated by exploration of communication-constrained underground environments using robot teams, we study the problem of planning for intermittent connectivity in multi-agent systems. We propose a novel concept of information-consistency to handle situations where the plan is not initially known by all agents, and suggest an integer linear program for synthesizing information-consistent plans that also achieve auxiliary goals. Furthermore, inspired by network flow problems we propose a novel way to pose connectivity constraints that scales much better than previous methods. In the second part of the paper we apply these results in an exploration setting, and propose a clustering method that separates a large exploration problem into smaller problems that can be solved independently. We demonstrate how the resulting exploration algorithm is able to coordinate a team of ten agents to explore a large environment.

An Inverse Dynamics Approach to Control Lyapunov Functions

With the goal of moving towards implementation of increasingly dynamic behaviors on underactuated systems, this paper presents an optimization-based approach for solving full-body dynamics based controllers on underactuated bipedal robots. The primary focus of this paper is on the development of an alternative approach to the implementation of controllers utilizing control Lyapunov function based quadratic programs. This approach utilizes many of the desirable aspects from successful inverse dynamics based controllers in the literature, while also incorporating a variant of control Lyapunov functions that renders better convergence in the context of tracking outputs. The principal benefits of this formulation include a greater ability to add costs which regulate the resulting behavior of the robot, in addition, the model error-prone inertia matrix is used only once, in a non-inverted form. The result is a successful demonstration of the controller for walking in simulation, and applied on hardware in real-time for crouching.

Improved Performance on Moving-Mass Hopping Robots with Parallel Elasticity

Robotic Hopping is challenging from the perspective of both modeling the dynamics as well as the mechanical design due to the short period of ground contact in which to actuate on the world. Previous work has demonstrated stable hopping on a moving-mass robot, wherein a single spring was utilized below the body of the robot. This paper finds that the addition of a spring in parallel to the actuator greatly improves the performance of moving mass hopping robots. This is demonstrated through the design of a novel one-dimensional hopping robot. For this robot, a rigorous trajectory optimization method is developed using hybrid systems models with experimentally tuned parameters. Simulation results are used to study the effects of a parallel spring on energetic efficiency, stability and hopping effort. We find that the double-spring model had 2.5x better energy efficiency than the single-spring model, and was able to hop using 40% less peak force from the actuator. Furthermore, the double-spring model produces stable hopping without the need for stabilizing controllers. These concepts are demonstrated experimentally on a novel hopping robot, wherein hop heights up to 40cm were achieved.

Risk-Averse Planning Under Uncertainty

We consider the problem of designing policies for partially observable Markov decision processes (POMDPs) with dynamic coherent risk objectives. Synthesizing risk-averse optimal policies for POMDPs requires infinite memory and thus undecidable. To overcome this difficulty, we propose a method based on bounded policy iteration for designing stochastic but finite state (memory) controllers, which takes advantage of standard convex optimization methods. Given a memory budget and optimality criterion, the proposed method modifies the stochastic finite state controller leading to sub-optimal solutions with lower coherent risk.