SMEMO: Social Memory for Trajectory Forecasting

Effective modeling of human interactions is of utmost importance when forecasting behaviors such as future trajectories. Each individual, with its motion, influences surrounding agents since everyone obeys to social non-written rules such as collision avoidance or group following. In this paper we model such interactions, which constantly evolve through time, by looking at the problem from an algorithmic point of view, i.e. as a data manipulation task. We present a neural network based on an end-to-end trainable working memory, which acts as an external storage where information about each agent can be continuously written, updated and recalled. We show that our method is capable of learning explainable cause-effect relationships between motions of different agents, obtaining state-of-the-art results on multiple trajectory forecasting datasets.

Data Augmentation for Electrocardiograms

Neural network models have demonstrated impressive performance in predicting pathologies and outcomes from the 12-lead electrocardiogram (ECG). However, these models often need to be trained with large, labelled datasets, which are not available for many predictive tasks of interest. In this work, we perform an empirical study examining whether training time data augmentation methods can be used to improve performance on such data-scarce ECG prediction problems. We investigate how data augmentation strategies impact model performance when detecting cardiac abnormalities from the ECG. Motivated by our finding that the effectiveness of existing augmentation strategies is highly task-dependent, we introduce a new method, TaskAug, which defines a flexible augmentation policy that is optimized on a per-task basis. We outline an efficient learning algorithm to do so that leverages recent work in nested optimization and implicit differentiation. In experiments, considering three datasets and eight predictive tasks, we find that TaskAug is competitive with or improves on prior work, and the learned policies shed light on what transformations are most effective for different tasks. We distill key insights from our experimental evaluation, generating a set of best practices for applying data augmentation to ECG prediction problems.

Quality of service based radar resource management for interference mitigation

An intelligent radar resource management is an essential building block of any modern radar system. The quality of service based resource allocation model (Q-RAM) provides a framework for profound and quantifiable decision making but lacks the flexibility necessary for optimal mitigation strategies in the presence of interference. We define an extension of the Q-RAM based radar resource management framework with an intelligent interference handling capability using various mitigation methods. The approach incorporates virtual time resources and alternative task configurations to compute near-optimal solutions in the presence of interference. The provided experimental results demonstrate a significant improvement over traditional strategies.

Learned Turbulence Modelling with Differentiable Fluid Solvers

In this paper, we train turbulence models based on convolutional neural networks. These learned turbulence models improve under-resolved low resolution solutions to the incompressible Navier-Stokes equations at simulation time. Our method involves the development of a differentiable numerical solver that supports the propagation of optimisation gradients through multiple solver steps. We showcase the significance of this property by demonstrating the superior stability and accuracy of those models that featured a higher number of unrolled steps during training. This approach is applied to three two-dimensional turbulence flow scenarios, a homogeneous decaying turbulence case, a temporally evolving mixing layer and a spatially evolving mixing layer. Our method achieves significant improvements of long-term \textit{a-posteriori} statistics when compared to no-model simulations, without requiring these statistics to be directly included in the learning targets. At inference time, our proposed method also gains substantial performance improvements over similarly accurate, purely numerical methods.

CenterSnap: Single-Shot Multi-Object 3D Shape Reconstruction and Categorical 6D Pose and Size Estimation

This paper studies the complex task of simultaneous multi-object 3D reconstruction, 6D pose and size estimation from a single-view RGB-D observation. In contrast to instance-level pose estimation, we focus on a more challenging problem where CAD models are not available at inference time. Existing approaches mainly follow a complex multi-stage pipeline which first localizes and detects each object instance in the image and then regresses to either their 3D meshes or 6D poses. These approaches suffer from high-computational cost and low performance in complex multi-object scenarios, where occlusions can be present. Hence, we present a simple one-stage approach to predict both the 3D shape and estimate the 6D pose and size jointly in a bounding-box free manner. In particular, our method treats object instances as spatial centers where each center denotes the complete shape of an object along with its 6D pose and size. Through this per-pixel representation, our approach can reconstruct in real-time (40 FPS) multiple novel object instances and predict their 6D pose and sizes in a single-forward pass. Through extensive experiments, we demonstrate that our approach significantly outperforms all shape completion and categorical 6D pose and size estimation baselines on multi-object ShapeNet and NOCS datasets respectively with a 12.6% absolute improvement in mAP for 6D pose for novel real-world object instances.

Leaders or Followers? A Temporal Analysis of Tweets from IRA Trolls

The Internet Research Agency (IRA) influences online political conversations in the United States, exacerbating existing partisan divides and sowing discord. In this paper we investigate the IRA's communication strategies by analyzing trending terms on Twitter to identify cases in which the IRA leads or follows other users. Our analysis focuses on over 38M tweets posted between 2016 and 2017 from IRA users (n=3,613), journalists (n=976), members of Congress (n=526), and politically engaged users from the general public (n=71,128). We find that the IRA tends to lead on topics related to the 2016 election, race, and entertainment, suggesting that these are areas both of strategic importance as well having the highest potential impact. Furthermore, we identify topics where the IRA has been relatively ineffective, such as tweets on military, political scandals, and violent attacks. Despite many tweets on these topics, the IRA rarely leads the conversation and thus has little opportunity to influence it. We offer our proposed methodology as a way to track the strategic choices of future influence operations in real-time.

LightDefectNet: A Highly Compact Deep Anti-Aliased Attention Condenser Neural Network Architecture for Light Guide Plate Surface Defect Detection

Light guide plates are essential optical components widely used in a diverse range of applications ranging from medical lighting fixtures to back-lit TV displays. An essential step in the manufacturing of light guide plates is the quality inspection of defects such as scratches, bright/dark spots, and impurities. This is mainly done in industry through manual visual inspection for plate pattern irregularities, which is time-consuming and prone to human error and thus act as a significant barrier to high-throughput production. Advances in deep learning-driven computer vision has led to the exploration of automated visual quality inspection of light guide plates to improve inspection consistency, accuracy, and efficiency. However, given the cost constraints in visual inspection scenarios, the widespread adoption of deep learning-driven computer vision methods for inspecting light guide plates has been greatly limited due to high computational requirements. In this study, we explore the utilization of machine-driven design exploration with computational and "best-practices" constraints as well as L$_1$ paired classification discrepancy loss to create LightDefectNet, a highly compact deep anti-aliased attention condenser neural network architecture tailored specifically for light guide plate surface defect detection in resource-constrained scenarios. Experiments show that LightDetectNet achieves a detection accuracy of $\sim$98.2% on the LGPSDD benchmark while having just 770K parameters ($\sim$33$\times$ and $\sim$6.9$\times$ lower than ResNet-50 and EfficientNet-B0, respectively) and $\sim$93M FLOPs ($\sim$88$\times$ and $\sim$8.4$\times$ lower than ResNet-50 and EfficientNet-B0, respectively) and $\sim$8.8$\times$ faster inference speed than EfficientNet-B0 on an embedded ARM processor.

An Integrated Programmable CPG with Bounded Output

Cyclic motions are fundamental patterns in robotic applications including industrial manipulation and legged robot locomotion. This paper proposes an approach for the online modulation of cyclic motions in robotic applications. For this purpose, we present an integrated programmable Central Pattern Generator (CPG) for the online generation of the reference joint trajectory of a robotic system out of a library of desired periodic motions. The reference trajectory is then followed by the lower-level controller of the robot. The proposed CPG generates a smooth reference joint trajectory convergence to the desired one while preserving the position and velocity joint limits of the robot. The integrated programmable CPG consists of one novel bounded output programmable oscillator. We design the programmable oscillator for encoding the desired multidimensional periodic trajectory as a stable limit cycle. We also use the state transformation method to ensure that the oscillator's output and its first-time derivative preserve the joint position and velocity limits of the robot. With the help of Lyapunov-based arguments, We prove that the proposed CPG provides the global stability and convergence of the desired trajectory. The effectiveness of the proposed integrated CPG for trajectory generation is shown in a passive rehabilitation scenario on the Kuka iiwa robot arm, and also in a walking simulation on a seven-link bipedal robot.

Methodology for forecasting and optimization in IEEE-CIS 3rd Technical Challenge

This report provides a description of the methodology I used in the IEEE-CIS 3rd Technical Challenge. For the forecast, I used a quantile regression forest approach using the solar variables provided by the Bureau of Meterology of Australia (BOM) and many of the weather variables from the European Centre for Medium-Range Weather Forecasting (ECMWF). Groups of buildings and all of the solar instances were trained together as they were observed to be closely correlated over time. Other variables used included Fourier values based on hour of day and day of year, and binary variables for combinations of days of the week. The start dates for the time series were carefully tuned based on phase 1 and cleaning and thresholding was used to reduce the observed error rate for each time series. For the optimization, a four-step approach was used using the forecast developed. First, a mixed-integer program (MIP) was solved for the recurring and recurring plus once-off activities, then each of these was extended using a mixed-integer quadratic program (MIQP). The general strategy was chosen from one of two ("array" from the "array" and "tuples" approaches) while the specific step improvement strategy was chosen from one of five ("no forced discharge").

Real-Time Trajectory Optimization in Robot-Assisted Exercise and Rehabilitation

This work focuses on the optimization of the training trajectory orientation using a robot as an advanced exercise machine (AEM) and muscle activations as biofeedback. Muscle recruitment patterns depend on trajectory parameters of the AEMs and correlate with the efficiency of exercise. Thus, improvements to training efficiency may be achieved by optimizing these parameters. The optimal regulation of these parameters is challenging because of the complexity of the physiological dynamics from person to person as a result of the unique physical features such as musculoskeletal distribution. Furthermore, these effects can vary due to fatigue, body temperature, and other physiological factors. In this paper, a model-free optimization method using Extremum Seeking Control (ESC) as a real-time optimizer is proposed. After selecting a muscle objective, this method seeks for the optimal combination of parameters using the muscle activations as biofeedback. The muscle objective can be selected by a therapist to emphasize or de-emphasize certain muscle groups. The feasibility of this method has been proven for the automatic regulation of an ellipsoidal curve orientation, suggesting the existence of two local optimal orientations. This methodology can also be applied to other parameter regulations using a different physiological effects such as oxygen consumption and heart rate as biofeedback.