SPIRIT: Perceptive Shared Autonomy for Robust Robotic Manipulation under Deep Learning Uncertainty

Deep learning (DL) has enabled impressive advances in robotic perception, yet its limited robustness and lack of interpretability hinder reliable deployment in safety critical applications. We propose a concept termed perceptive shared autonomy, in which uncertainty estimates from DL based perception are used to regulate the level of autonomy. Specifically, when the robot's perception is confident, semi-autonomous manipulation is enabled to improve performance; when uncertainty increases, control transitions to haptic teleoperation for maintaining robustness. In this way, high-performing but uninterpretable DL methods can be integrated safely into robotic systems. A key technical enabler is an uncertainty aware DL based point cloud registration approach based on the so called Neural Tangent Kernels (NTK). We evaluate perceptive shared autonomy on challenging aerial manipulation tasks through a user study of 15 participants and realization of mock-up industrial scenarios, demonstrating reliable robotic manipulation despite failures in DL based perception. The resulting system, named SPIRIT, improves both manipulation performance and system reliability. SPIRIT was selected as a finalist of a major industrial innovation award.

EigenMPC: An Eigenmanifold-Inspired Model-Predictive Control Framework for Exciting Efficient Oscillations in Mechanical Systems

This paper proposes a Nonlinear Model-Predictive Control (NMPC) method capable of finding and converging to energy-efficient regular oscillations, which require no control action to be sustained. The approach builds up on the recently developed Eigenmanifold theory, which defines the sets of line-shaped oscillations of a robot as an invariant two-dimensional submanifold of its state space. By defining the control problem as a nonlinear program (NLP), the controller is able to deal with constraints in the state and control variables and be energy-efficient not only in its final trajectory but also during the convergence phase. An initial implementation of this approach is proposed, analyzed, and tested in simulation.

A Nonlinear Observer for Free-Floating Target Motion using only Pose Measurements

In this paper, we design a nonlinear observer to estimate the inertial pose and the velocity of a free-floating non-cooperative satellite (Target) using only relative pose measurements. In the context of control design for orbital robotic capture of such a non-cooperative Target, due to lack of navigational aids, only a relative pose estimate may be obtained from slow-sampled and noisy exteroceptive sensors. The velocity, however, cannot be measured directly. To address this problem, we develop a model-based observer which acts as an internal model for Target kinematics/dynamics and therefore, may act as a predictor during periods of no measurement. To this end, firstly, we formalize the estimation problem on the SE(3) Lie group with different state and measurement spaces. Secondly, we develop the kinematics and dynamics observer such that the overall observer error dynamics possesses a stability property. Finally, the proposed observer is validated through robust Monte-Carlo simulations and experiments on a robotic facility.