Optimization over Trained (and Sparse) Neural Networks: A Surrogate within a Surrogate

We can approximate a constraint or an objective function that is uncertain or nonlinear with a neural network that we embed in the optimization model. This approach, which is known as constraint learning, faces the challenge that optimization models with neural network surrogates are harder to solve. Such difficulties have motivated studies on model reformulation, specialized optimization algorithms, and - to a lesser extent - pruning of the embedded networks. In this work, we double down on the use of surrogates by applying network pruning to produce a surrogate of the neural network itself. In the context of using a Mixed-Integer Linear Programming (MILP) solver to verify neural networks, we obtained faster adversarial perturbations for dense neural networks by using sparse surrogates, especially - and surprisingly - if not taking the time to finetune the sparse network to make up for the loss in accuracy. In other words, we show that a pruned network with bad classification performance can still be a good - and more efficient - surrogate.

Ensemble Learning of Myocardial Displacements for Myocardial Infarction Detection in Echocardiography

Early detection and localization of myocardial infarction (MI) can reduce the severity of cardiac damage through timely treatment interventions. In recent years, deep learning techniques have shown promise for detecting MI in echocardiographic images. However, there has been no examination of how segmentation accuracy affects MI classification performance and the potential benefits of using ensemble learning approaches. Our study investigates this relationship and introduces a robust method that combines features from multiple segmentation models to improve MI classification performance by leveraging ensemble learning. Our method combines myocardial segment displacement features from multiple segmentation models, which are then input into a typical classifier to estimate the risk of MI. We validated the proposed approach on two datasets: the public HMC-QU dataset (109 echocardiograms) for training and validation, and an E-Hospital dataset (60 echocardiograms) from a local clinical site in Vietnam for independent testing. Model performance was evaluated based on accuracy, sensitivity, and specificity. The proposed approach demonstrated excellent performance in detecting MI. The results showed that the proposed approach outperformed the state-of-the-art feature-based method. Further research is necessary to determine its potential use in clinical settings as a tool to assist cardiologists and technicians with objective assessments and reduce dependence on operator subjectivity. Our research codes are available on GitHub at https://github.com/vinuni-vishc/mi-detection-echo.

Learning Sequences of Manipulation Primitives for Robotic Assembly

This paper explores the idea that skillful assembly is best represented as dynamic sequences of Manipulation Primitives, and that such sequences can be automatically discovered by Reinforcement Learning. Manipulation Primitives, such as ``Move down until contact'', ``Slide along x while maintaining contact with the surface'', have enough complexity to keep the search tree shallow, yet are generic enough to generalize across a wide range of assembly tasks. Policies are learned in simulation, and then transferred onto the physical platform. Direct sim2real transfer (without retraining in real) achieves excellent success rates on challenging assembly tasks, such as round peg insertion with 100 micron clearance or square peg insertion with large hole position/orientation estimation errors.

Convex Controller Synthesis for Contact

Controlling contacts is truly challenging, and this has been a major hurdle to deploying industrial robots into unstructured/human-centric environments. More specifically, the main challenges are: (i) how to ensure stability at all times; (ii) how to satisfy task-specific performance requirements; (iii) how to achieve (i) and (ii) under environment uncertainty, robot parameters uncertainty, sensor and actuator time delays, external perturbations, etc. Here, we propose a new approach -- Convex Controller Synthesis (CCS) -- to tackle the above challenges based on robust control theory and convex optimization. In two physical interaction tasks -- robot hand guiding and sliding on surfaces with different and unknown stiffnesses -- we show that CCS controllers outperform their classical counterparts in an essential way.

Siamese Convolutional Neural Network for Sub-millimeter-accurate Camera Pose Estimation and Visual Servoing

Visual Servoing (VS), where images taken from a camera typically attached to the robot end-effector are used to guide the robot motions, is an important technique to tackle robotic tasks that require a high level of accuracy. We propose a new neural network, based on a Siamese architecture, for highly accurate camera pose estimation. This, in turn, can be used as a final refinement step following a coarse VS or, if applied in an iterative manner, as a standalone VS on its own. The key feature of our neural network is that it outputs the relative pose between any pair of images, and does so with sub-millimeter accuracy. We show that our network can reduce pose estimation errors to 0.6 mm in translation and 0.4 degrees in rotation, from initial errors of 10 mm / 5 degrees if applied once, or of several cm / tens of degrees if applied iteratively. The network can generalize to similar objects, is robust against changing lighting conditions, and to partial occlusions (when used iteratively). The high accuracy achieved enables tackling low-tolerance assembly tasks downstream: using our network, an industrial robot can achieve 97.5% success rate on a VGA-connector insertion task without any force sensing mechanism.

Critically fast pick-and-place with suction cups

Fast robotics pick-and-place with suction cups is a crucial component in the current development of automation in logistics (factory lines, e-commerce, etc.). By "critically fast" we mean the fastest possible movement for transporting an object such that it does not slip or fall from the suction cup. The main difficulties are: (i) handling the contact between the suction cup and the object, which fundamentally involves kinodynamic constraints; and (ii) doing so at a low computational cost, typically a few hundreds of milliseconds. To address these difficulties, we propose (a) a model for suction cup contacts, (b) a procedure to identify the contact stability constraint based on that model, and (c) a pipeline to parameterize, in a time-optimal manner, arbitrary geometric paths under the identified contact stability constraint. We experimentally validate the proposed pipeline on a physical robot system: the cycle time for a typical pick-and-place task was less than 5 seconds, planning and execution times included. The full pipeline is released as open-source for the robotics community.

Time-Optimal Path Tracking via Reachability Analysis

Given a geometric path, the Time-Optimal Path Tracking problem consists in finding the control strategy to traverse the path time-optimally while regulating tracking errors. A simple yet effective approach to this problem is to decompose the controller into two components: (i)~a path controller, which modulates the parameterization of the desired path in an online manner, yielding a reference trajectory; and (ii)~a tracking controller, which takes the reference trajectory and outputs joint torques for tracking. However, there is one major difficulty: the path controller might not find any feasible reference trajectory that can be tracked by the tracking controller because of torque bounds. In turn, this results in degraded tracking performances. Here, we propose a new path controller that is guaranteed to find feasible reference trajectories by accounting for possible future perturbations. The main technical tool underlying the proposed controller is Reachability Analysis, a new method for analyzing path parameterization problems. Simulations show that the proposed controller outperforms existing methods.

A New Approach to Time-Optimal Path Parameterization based on Reachability Analysis

Time-Optimal Path Parameterization (TOPP) is a well-studied problem in robotics and has a wide range of applications. There are two main families of methods to address TOPP: Numerical Integration (NI) and Convex Optimization (CO). NI-based methods are fast but difficult to implement and suffer from robustness issues, while CO-based approaches are more robust but at the same time significantly slower. Here we propose a new approach to TOPP based on Reachability Analysis (RA). The key insight is to recursively compute reachable and controllable sets at discretized positions on the path by solving small Linear Programs (LPs). The resulting algorithm is faster than NI-based methods and as robust as CO-based ones (100% success rate), as confirmed by extensive numerical evaluations. Moreover, the proposed approach offers unique additional benefits: Admissible Velocity Propagation and robustness to parametric uncertainty can be derived from it in a simple and natural way.

On the Structure of the Time-Optimal Path Parameterization Problem with Third-Order Constraints

Finding the Time-Optimal Parameterization of a Path (TOPP) subject to second-order constraints (e.g. acceleration, torque, contact stability, etc.) is an important and well-studied problem in robotics. In comparison, TOPP subject to third-order constraints (e.g. jerk, torque rate, etc.) has received far less attention and remains largely open. In this paper, we investigate the structure of the TOPP problem with third-order constraints. In particular, we identify two major difficulties: (i) how to smoothly connect optimal profiles, and (ii) how to address singularities, which stop profile integration prematurely. We propose a new algorithm, TOPP3, which addresses these two difficulties and thereby constitutes an important milestone towards an efficient computational solution to TOPP with third-order constraints.

Robotic manipulation of a rotating chain

This paper considers the problem of manipulating a uniformly rotating chain: the chain is rotated at a constant angular speed around a fixed axis using a robotic manipulator. Manipulation is quasi-static in the sense that transitions are slow enough for the chain to be always in "rotational" equilibrium. The curve traced by the chain in a rotating plane -- its shape function -- can be determined by a simple force analysis, yet it possesses complex multi-solutions behavior typical of non-linear systems. We prove that the configuration space of the uniformly rotating chain is homeomorphic to a two-dimensional surface embedded in $\mathbb{R}^3$. Using that representation, we devise a manipulation strategy for transiting between different rotation modes in a stable and controlled manner. We demonstrate the strategy on a physical robotic arm manipulating a rotating chain. Finally, we discuss how the ideas developed here might find fruitful applications in the study of other flexible objects, such as elastic rods or concentric tubes.