Enhancing Hand Palm Motion Gesture Recognition by Eliminating Reference Frame Bias via Frame-Invariant Similarity Measures

The ability of robots to recognize human gestures facilitates a natural and accessible human-robot collaboration. However, most work in gesture recognition remains rooted in reference frame-dependent representations. This poses a challenge when reference frames vary due to different work cell layouts, imprecise frame calibrations, or other environmental changes. This paper investigated the use of invariant trajectory descriptors for robust hand palm motion gesture recognition under reference frame changes. First, a novel dataset of recorded Hand Palm Motion (HPM) gestures is introduced. The motion gestures in this dataset were specifically designed to be distinguishable without dependence on specific reference frames or directional cues. Afterwards, multiple invariant trajectory descriptor approaches were benchmarked to assess how their performances generalize to this novel HPM dataset. After this offline benchmarking, the best scoring approach is validated for online recognition by developing a real-time Proof of Concept (PoC). In this PoC, hand palm motion gestures were used to control the real-time movement of a manipulator arm. The PoC demonstrated a high recognition reliability in real-time operation, achieving an $F_1$-score of 92.3%. This work demonstrates the effectiveness of the invariant descriptor approach as a standalone solution. Moreover, we believe that the invariant descriptor approach can also be utilized within other state-of-the-art pattern recognition and learning systems to improve their robustness against reference frame variations.

From Instantaneous to Predictive Control: A More Intuitive and Tunable MPC Formulation for Robot Manipulators

Model predictive control (MPC) has become increasingly popular for the control of robot manipulators due to its improved performance compared to instantaneous control approaches. However, tuning these controllers remains a considerable hurdle. To address this hurdle, we propose a practical MPC formulation which retains the more interpretable tuning parameters of the instantaneous control approach while enhancing the performance through a prediction horizon. The formulation is motivated at hand of a simple example, highlighting the practical tuning challenges associated with typical MPC approaches and showing how the proposed formulation alleviates these challenges. Furthermore, the formulation is validated on a surface-following task, illustrating its applicability to industrially relevant scenarios. Although the research is presented in the context of robot manipulator control, we anticipate that the formulation is more broadly applicable.

BILTS: A novel bi-invariant local trajectory-shape descriptor for rigid-body motion

Measuring the similarity between motions and established motion models is crucial for motion analysis, recognition, generation, and adaptation. To enhance similarity measurement across diverse contexts, invariant motion descriptors have been proposed. However, for rigid-body motion, few invariant descriptors exist that are bi-invariant, meaning invariant to both the body and world reference frames used to describe the motion. Moreover, their robustness to singularities is limited. This paper introduces a novel Bi-Invariant Local Trajectory-Shape descriptor (BILTS) and a corresponding dissimilarity measure. Mathematical relationships between BILTS and existing descriptors are derived, providing new insights into their properties. The paper also includes an algorithm to reproduce the motion from the BILTS descriptor, demonstrating its bidirectionality and usefulness for trajectory generation. Experimental validation using datasets of daily-life activities shows the higher robustness of the BILTS descriptor compared to the bi-invariant ISA descriptor. This higher robustness supports the further application of bi-invariant descriptors for motion recognition and generalization.

Automatic Derivation of an Optimal Task Frame for Learning and Controlling Contact-Rich Tasks

This study investigates learning from demonstration (LfD) for contact-rich tasks. The procedure for choosing a task frame to express the learned signals for the motion and interaction wrench is often omitted or using expert insight. This article presents a procedure to derive the optimal task frame from motion and wrench data recorded during the demonstration. The procedure is based on two principles that are hypothesized to underpin the control configuration targeted by an expert, and assumes task frame origins and orientations that are fixed to either the world or the robot tool. It is rooted in screw theory, is entirely probabilistic and does not involve any hyperparameters. The procedure was validated by demonstrating several tasks, including surface following and manipulation of articulated objects, showing good agreement between the obtained and the assumed expert task frames. To validate the performance of the learned tasks by a UR10e robot, a constraint-based controller was designed based on the derived task frames and the learned data expressed therein. These experiments showed the effectiveness and versatility of the proposed approach. The task frame derivation approach fills a gap in the state of the art of LfD, bringing LfD for contact-rich tasks closer to practical application.

Enhancing motion trajectory segmentation of rigid bodies using a novel screw-based trajectory-shape representation

Trajectory segmentation refers to dividing a trajectory into meaningful consecutive sub-trajectories. This paper focuses on trajectory segmentation for 3D rigid-body motions. Most segmentation approaches in the literature represent the body's trajectory as a point trajectory, considering only its translation and neglecting its rotation. We propose a novel trajectory representation for rigid-body motions that incorporates both translation and rotation, and additionally exhibits several invariant properties. This representation consists of a geometric progress rate and a third-order trajectory-shape descriptor. Concepts from screw theory were used to make this representation time-invariant and also invariant to the choice of body reference point. This new representation is validated for a self-supervised segmentation approach, both in simulation and using real recordings of human-demonstrated pouring motions. The results show a more robust detection of consecutive submotions with distinct features and a more consistent segmentation compared to conventional representations. We believe that other existing segmentation methods may benefit from using this trajectory representation to improve their invariance.

Invariant Descriptors of Motion and Force Trajectories for Interpreting Object Manipulation Tasks in Contact

Invariant descriptors of point and rigid-body motion trajectories have been proposed in the past as representative task models for motion recognition and generalization. Currently, no invariant descriptor exists for representing force trajectories which appear in contact tasks. This paper introduces invariant descriptors for force trajectories by exploiting the duality between motion and force. Two types of invariant descriptors are presented depending on whether the trajectories consist of screw coordinates or vector coordinates. Methods and software are provided for robustly calculating the invariant descriptors from noisy measurements using optimal control. Using experimental human demonstrations of a 3D contour following task, invariant descriptors are shown to result in task representations that do not depend on the calibration of reference frames or sensor locations. Tuning of the optimal control problems is shown to be fast and intuitive. Similarly as for motions in free space, the proposed invariant descriptors for motion and force trajectories may prove useful for the recognition and generalization of constrained motions such as during object manipulation in contact.

Using Intent Estimation and Decision Theory to Support Lifting Motions with a Quasi-Passive Hip Exoskeleton

This paper compares three controllers for quasi-passive exoskeletons. The Utility Maximizing Controller (UMC) uses intent estimation to recognize user motions and decision theory to activate the support mechanism. The intent estimation algorithm requires demonstrations for each motion to be recognized. Depending on what motion is recognized, different control signals are sent to the exoskeleton. The Extended UMC (E-UMC) adds a calibration step and a velocity module to trigger the UMC. As a benchmark, and to compare the behavior of the controllers irrespective of the hardware, a Passive Exoskeleton Controller (PEC) is developed as well. The controllers were implemented on a hip exoskeleton and evaluated in a user study consisting of two phases. First, demonstrations of three motions were recorded: squat, stoop left and stoop right. Afterwards, the controllers were evaluated. The E-UMC combines benefits from the UMC and the PEC, confirming the need for the two extensions. The E-UMC discriminates between the three motions and does not generate false positives for previously unseen motions such as stair walking. The proposed methods can also be applied to support other motions.

FATROP : A Fast Constrained Optimal Control Problem Solver for Robot Trajectory Optimization and Control

Trajectory optimization is a powerful tool for robot motion planning and control. State-of-the-art general-purpose nonlinear programming solvers are versatile, handle constraints in an effective way and provide a high numerical robustness, but they are slow because they do not fully exploit the optimal control problem structure at hand. Existing structure-exploiting solvers are fast but they often lack techniques to deal with nonlinearity or rely on penalty methods to enforce (equality or inequality) path constraints. This works presents FATROP: a trajectory optimization solver that is fast and benefits from the salient features of general-purpose nonlinear optimization solvers. The speed-up is mainly achieved through the use of a specialized linear solver, based on a Riccati recursion that is generalized to also support stagewise equality constraints. To demonstrate the algorithm's potential, it is benchmarked on a set of robot problems that are challenging from a numerical perspective, including problems with a minimum-time objective and no-collision constraints. The solver is shown to solve problems for trajectory generation of a quadrotor, a robot manipulator and a truck-trailer problem in a few tens of milliseconds. The algorithm's C++-code implementation accompanies this work as open source software, released under the GNU Lesser General Public License (LGPL). This software framework may encourage and enable the robotics community to use trajectory optimization in more challenging applications.

A Rigorously Bayesian Beam Model and an Adaptive Full Scan Model for Range Finders in Dynamic Environments

This paper proposes and experimentally validates a Bayesian network model of a range finder adapted to dynamic environments. All modeling assumptions are rigorously explained, and all model parameters have a physical interpretation. This approach results in a transparent and intuitive model. With respect to the state of the art beam model this paper: (i) proposes a different functional form for the probability of range measurements caused by unmodeled objects, (ii) intuitively explains the discontinuity encountered in te state of the art beam model, and (iii) reduces the number of model parameters, while maintaining the same representational power for experimental data. The proposed beam model is called RBBM, short for Rigorously Bayesian Beam Model. A maximum likelihood and a variational Bayesian estimator (both based on expectation-maximization) are proposed to learn the model parameters. Furthermore, the RBBM is extended to a full scan model in two steps: first, to a full scan model for static environments and next, to a full scan model for general, dynamic environments. The full scan model accounts for the dependency between beams and adapts to the local sample density when using a particle filter. In contrast to Gaussian-based state of the art models, the proposed full scan model uses a sample-based approximation. This sample-based approximation enables handling dynamic environments and capturing multi-modality, which occurs even in simple static environments.