Control of a pendulum system: From simulation to reality

Control theory deals with the study of controlling dynamical systems. Robots today are growing increasingly complex and moving out of factory floors to real world environment. These robots have to interact with real world environment factors such as disturbances and this requires the robot to have a control system that is robust. Testing control algorithms on robots in real world environment can pose critical safety issues and can be financially expensive. This has resulted in a heavy emphasis on using simulation to test control algorithms before deploying them in real world environments. Designing control algorithms is an iterative process that starts with modelling the target system in simulation, designing a controller, testing the controller in simulation and then changing the controller parameters to design a better controller. This report explores how an approximated system model of a target hardware system can be developed, which can then be used to design a LQR controller for the target system. The controller is then tested under a disturbance, on hardware and in simulation, and the system response is recorded. The system response from hardware and simulation are then compared to validate the use of approximated system models in simulation for designing and testing control algorithms.

Learning multi-modal generative models with permutation-invariant encoders and tighter variational bounds

Devising deep latent variable models for multi-modal data has been a long-standing theme in machine learning research. Multi-modal Variational Autoencoders (VAEs) have been a popular generative model class that learns latent representations which jointly explain multiple modalities. Various objective functions for such models have been suggested, often motivated as lower bounds on the multi-modal data log-likelihood or from information-theoretic considerations. In order to encode latent variables from different modality subsets, Product-of-Experts (PoE) or Mixture-of-Experts (MoE) aggregation schemes have been routinely used and shown to yield different trade-offs, for instance, regarding their generative quality or consistency across multiple modalities. In this work, we consider a variational bound that can tightly lower bound the data log-likelihood. We develop more flexible aggregation schemes that generalise PoE or MoE approaches by combining encoded features from different modalities based on permutation-invariant neural networks. Our numerical experiments illustrate trade-offs for multi-modal variational bounds and various aggregation schemes. We show that tighter variational bounds and more flexible aggregation models can become beneficial when one wants to approximate the true joint distribution over observed modalities and latent variables in identifiable models.

A Basic Geometric Framework for Quasi-Static Mechanical Manipulation

In this work, we propose a geometric framework for analyzing mechanical manipulation, for example, by a robotic agent. Under the assumption of conservative forces and quasi-static manipulation, we use energy methods to derive a metric. We first review and show that the natural geometric setting is represented by the cotangent bundle and its Lagrangian submanifolds. These are standard concepts in geometric mechanics but usually presented within dynamical frameworks. We review the basic definitions from a static mechanics perspective and show how Lagrangian submanifolds are naturally derived from a first order analysis. Then, via a second order analysis, we derive the Hessian of total energy. As this is not necessarily positive-definite from a control perspective, we propose the use of the squared-Hessian for optimality measures, motivated by insights {derived from both mechanics (Gauss's Principle) and biology (Separation Principle)}. We conclude by showing how such methods can be applied, for example, to the simple case of an elastically driven pendulum. The example is simple enough to allow for analytical solution. However, an extension is further derived and numerically solved, which is more realistically connected with actual robotic manipulation problems.

GloCAL: Glocalized Curriculum-Aided Learning of Multiple Tasks with Application to Robotic Grasping

The domain of robotics is challenging to apply deep reinforcement learning due to the need for large amounts of data and for ensuring safety during learning. Curriculum learning has shown good performance in terms of sample- efficient deep learning. In this paper, we propose an algorithm (named GloCAL) that creates a curriculum for an agent to learn multiple discrete tasks, based on clustering tasks according to their evaluation scores. From the highest-performing cluster, a global task representative of the cluster is identified for learning a global policy that transfers to subsequently formed new clusters, while the remaining tasks in the cluster are learned as local policies. The efficacy and efficiency of our GloCAL algorithm are compared with other approaches in the domain of grasp learning for 49 objects with varied object complexity and grasp difficulty from the EGAD! dataset. The results show that GloCAL is able to learn to grasp 100% of the objects, whereas other approaches achieve at most 86% despite being given 1.5 times longer training time.