Synthesizing Robust Walking Gaits via Discrete-Time Barrier Functions with Application to Multi-Contact Exoskeleton Locomotion

Successfully achieving bipedal locomotion remains challenging due to real-world factors such as model uncertainty, random disturbances, and imperfect state estimation. In this work, we propose the use of discrete-time barrier functions to certify hybrid forward invariance of reduced step-to-step dynamics. The size of these invariant sets can then be used as a metric for locomotive robustness. We demonstrate an application of this metric towards synthesizing robust nominal walking gaits using a simulation-in-the-loop approach. This procedure produces reference motions with step-to-step dynamics that are maximally forward-invariant with respect to the reduced representation of choice. The results demonstrate robust locomotion for both flat-foot walking and multi-contact walking on the Atalante lower-body exoskeleton.

Humanoid Robot Co-Design: Coupling Hardware Design with Gait Generation via Hybrid Zero Dynamics

Selecting robot design parameters can be challenging since these parameters are often coupled with the performance of the controller and, therefore, the resulting capabilities of the robot. This leads to a time-consuming and often expensive process whereby one iterates between designing the robot and manually evaluating its capabilities. This is particularly challenging for bipedal robots, where it can be difficult to evaluate the behavior of the system due to the underlying nonlinear and hybrid dynamics. Thus, in an effort to streamline the design process of bipedal robots, and maximize their performance, this paper presents a systematic framework for the co-design of humanoid robots and their associated walking gaits. To this end, we leverage the framework of hybrid zero dynamic (HZD) gait generation, which gives a formal approach to the generation of dynamic walking gaits. The key novelty of this paper is to consider both virtual constraints associated with the actuators of the robot, coupled with design virtual constraints that encode the associated parameters of the robot to be designed. These virtual constraints are combined in an HZD optimization problem which simultaneously determines the design parameters while finding a stable walking gait that minimizes a given cost function. The proposed approach is demonstrated through the design of a novel humanoid robot, ADAM, wherein its thigh and shin are co-designed so as to yield energy efficient bipedal locomotion.

Input-to-State Stability in Probability

Input-to-State Stability (ISS) is fundamental in mathematically quantifying how stability degrades in the presence of bounded disturbances. If a system is ISS, its trajectories will remain bounded, and will converge to a neighborhood of an equilibrium of the undisturbed system. This graceful degradation of stability in the presence of disturbances describes a variety of real-world control implementations. Despite its utility, this property requires the disturbance to be bounded and provides invariance and stability guarantees only with respect to this worst-case bound. In this work, we introduce the concept of ``ISS in probability (ISSp)'' which generalizes ISS to discrete-time systems subject to unbounded stochastic disturbances. Using tools from martingale theory, we provide Lyapunov conditions for a system to be exponentially ISSp, and connect ISSp to stochastic stability conditions found in literature. We exemplify the utility of this method through its application to a bipedal robot confronted with step heights sampled from a truncated Gaussian distribution.

An Input-to-State Stability Perspective on Robust Locomotion

Uneven terrain necessarily transforms periodic walking into a non-periodic motion. As such, traditional stability analysis tools no longer adequately capture the ability of a bipedal robot to locomote in the presence of such disturbances. This motivates the need for analytical tools aimed at generalized notions of stability -- robustness. Towards this, we propose a novel definition of robustness, termed \emph{$\delta$-robustness}, to characterize the domain on which a nominal periodic orbit remains stable despite uncertain terrain. This definition is derived by treating perturbations in ground height as disturbances in the context of the input-to-state-stability (ISS) of the extended Poincar\'{e} map associated with a periodic orbit. The main theoretic result is the formulation of robust Lyapunov functions that certify $\delta$-robustness of periodic orbits. This yields an optimization framework for verifying $\delta$-robustness, which is demonstrated in simulation with a bipedal robot walking on uneven terrain.

Robust Bipedal Locomotion: Leveraging Saltation Matrices for Gait Optimization

The ability to generate robust walking gaits on bipedal robots is key to their successful realization on hardware. To this end, this work extends the method of Hybrid Zero Dynamics (HZD) -- which traditionally only accounts for locomotive stability via periodicity constraints under perfect impact events -- through the inclusion of the saltation matrix with a view toward synthesizing robust walking gaits. By jointly minimizing the norm of the extended saltation matrix and the torque of the robot directly in the gait generation process, we show that the synthesized gaits are more robust than gaits generated with either term alone; these results are shown in simulation and on hardware for both the AMBER-3M planar biped and the Atalante lower-body exoskeleton (both with and without a human subject). The end result is experimental validation that combining saltation matrices with HZD methods produces more robust bipedal walking in practice.

POLAR: Preference Optimization and Learning Algorithms for Robotics

Parameter tuning for robotic systems is a time-consuming and challenging task that often relies on domain expertise of the human operator. Moreover, existing learning methods are not well suited for parameter tuning for many reasons including: the absence of a clear numerical metric for `good robotic behavior'; limited data due to the reliance on real-world experimental data; and the large search space of parameter combinations. In this work, we present an open-source MATLAB Preference Optimization and Learning Algorithms for Robotics toolbox (POLAR) for systematically exploring high-dimensional parameter spaces using human-in-the-loop preference-based learning. This aim of this toolbox is to systematically and efficiently accomplish one of two objectives: 1) to optimize robotic behaviors for human operator preference; 2) to learn the operator's underlying preference landscape to better understand the relationship between adjustable parameters and operator preference. The POLAR toolbox achieves these objectives using only subjective feedback mechanisms (pairwise preferences, coactive feedback, and ordinal labels) to infer a Bayesian posterior over the underlying reward function dictating the user's preferences. We demonstrate the performance of the toolbox in simulation and present various applications of human-in-the-loop preference-based learning.

Safety-Aware Preference-Based Learning for Safety-Critical Control

Bringing dynamic robots into the wild requires a tenuous balance between performance and safety. Yet controllers designed to provide robust safety guarantees often result in conservative behavior, and tuning these controllers to find the ideal trade-off between performance and safety typically requires domain expertise or a carefully constructed reward function. This work presents a design paradigm for systematically achieving behaviors that balance performance and robust safety by integrating safety-aware Preference-Based Learning (PBL) with Control Barrier Functions (CBFs). Fusing these concepts -- safety-aware learning and safety-critical control -- gives a robust means to achieve safe behaviors on complex robotic systems in practice. We demonstrate the capability of this design paradigm to achieve safe and performant perception-based autonomous operation of a quadrupedal robot both in simulation and experimentally on hardware.

Natural Multicontact Walking for Robotic Assistive Devices via Musculoskeletal Models and Hybrid Zero Dynamics

Generating provably stable walking gaits that yield natural locomotion when executed on robotic-assistive devices is a challenging task that often requires hand-tuning by domain experts. This paper presents an alternative methodology, where we propose the addition of musculoskeletal models directly into the gait generation process to intuitively shape the resulting behavior. In particular, we construct a multi-domain hybrid system model that combines the system dynamics with muscle models to represent natural multicontact walking. Stable walking gaits can then be formally generated for this model via the hybrid zero dynamics method. We experimentally apply our framework towards achieving multicontact locomotion on a dual-actuated transfemoral prosthesis, AMPRO3. The results demonstrate that enforcing feasible muscle dynamics produces gaits that yield natural locomotion (as analyzed via electromyography), without the need for extensive manual tuning. Moreover, these gaits yield similar behavior to expert-tuned gaits. We conclude that the novel approach of combining robotic walking methods (specifically HZD) with muscle models successfully generates anthropomorphic robotic-assisted locomotion.

Learning Controller Gains on Bipedal Walking Robots via User Preferences

Experimental demonstration of complex robotic behaviors relies heavily on finding the correct controller gains. This painstaking process is often completed by a domain expert, requiring deep knowledge of the relationship between parameter values and the resulting behavior of the system. Even when such knowledge is possessed, it can take significant effort to navigate the nonintuitive landscape of possible parameter combinations. In this work, we explore the extent to which preference-based learning can be used to optimize controller gains online by repeatedly querying the user for their preferences. This general methodology is applied to two variants of control Lyapunov function based nonlinear controllers framed as quadratic programs, which have nice theoretic properties but are challenging to realize in practice. These controllers are successfully demonstrated both on the planar underactuated biped, AMBER, and on the 3D underactuated biped, Cassie. We experimentally evaluate the performance of the learned controllers and show that the proposed method is repeatably able to learn gains that yield stable and robust locomotion.

Preference-Based Learning for User-Guided HZD Gait Generation on Bipedal Walking Robots

This paper presents a framework that unifies control theory and machine learning in the setting of bipedal locomotion. Traditionally, gaits are generated through trajectory optimization methods and then realized experimentally -- a process that often requires extensive tuning due to differences between the models and hardware. In this work, the process of gait realization via hybrid zero dynamics (HZD) based optimization problems is formally combined with preference-based learning to systematically realize dynamically stable walking. Importantly, this learning approach does not require a carefully constructed reward function, but instead utilizes human pairwise preferences. The power of the proposed approach is demonstrated through two experiments on a planar biped AMBER-3M: the first with rigid point feet, and the second with induced model uncertainty through the addition of springs where the added compliance was not accounted for in the gait generation or in the controller. In both experiments, the framework achieves stable, robust, efficient, and natural walking in fewer than 50 iterations with no reliance on a simulation environment. These results demonstrate a promising step in the unification of control theory and learning.