A Non-linear Function-on-Function Model for Regression with Time Series Data

In the last few decades, building regression models for non-scalar variables, including time series, text, image, and video, has attracted increasing interests of researchers from the data analytic community. In this paper, we focus on a multivariate time series regression problem. Specifically, we aim to learn mathematical mappings from multiple chronologically measured numerical variables within a certain time interval S to multiple numerical variables of interest over time interval T. Prior arts, including the multivariate regression model, the Seq2Seq model, and the functional linear models, suffer from several limitations. The first two types of models can only handle regularly observed time series. Besides, the conventional multivariate regression models tend to be biased and inefficient, as they are incapable of encoding the temporal dependencies among observations from the same time series. The sequential learning models explicitly use the same set of parameters along time, which has negative impacts on accuracy. The function-on-function linear model in functional data analysis (a branch of statistics) is insufficient to capture complex correlations among the considered time series and suffer from underfitting easily. In this paper, we propose a general functional mapping that embraces the function-on-function linear model as a special case. We then propose a non-linear function-on-function model using the fully connected neural network to learn the mapping from data, which addresses the aforementioned concerns in the existing approaches. For the proposed model, we describe in detail the corresponding numerical implementation procedures. The effectiveness of the proposed model is demonstrated through the application to two real-world problems.

Decentralized Event-Triggered Federated Learning with Heterogeneous Communication Thresholds

A recent emphasis of distributed learning research has been on federated learning (FL), in which model training is conducted by the data-collecting devices. Existing research on FL has mostly focused on a star topology learning architecture with synchronized (time-triggered) model training rounds, where the local models of the devices are periodically aggregated by a centralized coordinating node. However, in many settings, such a coordinating node may not exist, motivating efforts to fully decentralize FL. In this work, we propose a novel methodology for distributed model aggregations via asynchronous, event-triggered consensus iterations over the network graph topology. We consider heterogeneous communication event thresholds at each device that weigh the change in local model parameters against the available local resources in deciding the benefit of aggregations at each iteration. Through theoretical analysis, we demonstrate that our methodology achieves asymptotic convergence to the globally optimal learning model under standard assumptions in distributed learning and graph consensus literature, and without restrictive connectivity requirements on the underlying topology. Subsequent numerical results demonstrate that our methodology obtains substantial improvements in communication requirements compared with FL baselines.

Neural Lagrangian Schrödinger Bridge

Population dynamics is the study of temporal and spatial variation in the size of populations of organisms and is a major part of population ecology. One of the main difficulties in analyzing population dynamics is that we can only obtain observation data with coarse time intervals from fixed-point observations due to experimental costs or other constraints. Recently, modeling population dynamics by using continuous normalizing flows (CNFs) and dynamic optimal transport has been proposed to infer the expected trajectory of samples from a fixed-point observed population. While the sample behavior in CNF is deterministic, the actual sample in biological systems moves in an essentially random yet directional manner. Moreover, when a sample moves from point A to point B in dynamical systems, its trajectory is such that the corresponding action has the smallest possible value, known as the principle of least action. To satisfy these requirements of the sample trajectories, we formulate the Lagrangian Schr\"odinger bridge (LSB) problem and propose to solve it approximately using neural SDE with regularization. We also develop a model architecture that enables faster computation. Our experiments show that our solution to the LSB problem can approximate the dynamics at the population level and that using the prior knowledge introduced by the Lagrangian enables us to estimate the trajectories of individual samples with stochastic behavior.

Immersive Text Game and Personality Classification

We designed and built a game called \textit{Immersive Text Game}, which allows the player to choose a story and a character, and interact with other characters in the story in an immersive manner of dialogues. The game is based on several latest models, including text generation language model, information extraction model, commonsense reasoning model, and psychology evaluation model. In the past, similar text games usually let players choose from limited actions instead of answering on their own, and not every time what characters said are determined by the player. Through the combination of these models and elaborate game mechanics and modes, the player will find some novel experiences as driven through the storyline.

Edge Detection and Deep Learning Based SETI Signal Classification Method

Scientists at the Berkeley SETI Research Center are Searching for Extraterrestrial Intelligence (SETI) by a new signal detection method that converts radio signals into spectrograms through Fourier transforms and classifies signals represented by two-dimensional time-frequency spectrums, which successfully converts a signal classification problem into an image classification task. In view of the negative impact of background noises on the accuracy of spectrograms classification, a new method is introduced in this paper. After Gaussian convolution smoothing the signals, edge detection functions are applied to detect the edge of the signals and enhance the outline of the signals, then the processed spectrograms are used to train the deep neural network to compare the classification accuracy of various image classification networks. The results show that the proposed method can effectively improve the classification accuracy of SETI spectrums.

Decentralized Connectivity Maintenance with Time Delays using Control Barrier Functions

Connectivity maintenance is crucial for the real world deployment of multi-robot systems, as it ultimately allows the robots to communicate, coordinate and perform tasks in a collaborative way. A connectivity maintenance controller must keep the multi-robot system connected independently from the system's mission and in the presence of undesired real world effects such as communication delays, model errors, and computational time delays, among others. In this paper we present the implementation, on a real robotic setup, of a connectivity maintenance control strategy based on Control Barrier Functions. During experimentation, we found that the presence of communication delays has a significant impact on the performance of the controlled system, with respect to the ideal case. We propose a heuristic to counteract the effects of communication delays, and we verify its efficacy both in simulation and with physical robot experiments.

Blue Data Computation Maximization in 6G Space-Air-Sea Non-Terrestrial Networks

Non-terrestrial networks (NTN), encompassing space and air platforms, are a key component of the upcoming sixth-generation (6G) cellular network. Meanwhile, maritime network traffic has grown significantly in recent years due to sea transportation used for national defense, research, recreational activities, domestic and international trade. In this paper, the seamless and reliable demand for communication and computation in maritime wireless networks is investigated. Two types of marine user equipment (UEs), i.e., low-antenna gain and high-antenna gain UEs, are considered. A joint task computation and time allocation problem for weighted sum-rate maximization is formulated as mixed-integer linear programming (MILP). The goal is to design an algorithm that enables the network to efficiently provide backhaul resources to an unmanned aerial vehicle (UAV) and offload HUEs tasks to LEO satellite for blue data (i.e., marine user's data). To solve this MILP, a solution based on the Bender and primal decomposition is proposed. The Bender decomposes MILP into the master problem for binary task decision and subproblem for continuous-time resource allocation. Moreover, primal decomposition deals with a coupling constraint in the subproblem. Finally, numerical results demonstrate that the proposed algorithm provides the maritime UEs coverage demand in polynomial time computational complexity and achieves a near-optimal solution.

Tag-Based Attention Guided Bottom-Up Approach for Video Instance Segmentation

Video Instance Segmentation is a fundamental computer vision task that deals with segmenting and tracking object instances across a video sequence. Most existing methods typically accomplish this task by employing a multi-stage top-down approach that usually involves separate networks to detect and segment objects in each frame, followed by associating these detections in consecutive frames using a learned tracking head. In this work, however, we introduce a simple end-to-end trainable bottom-up approach to achieve instance mask predictions at the pixel-level granularity, instead of the typical region-proposals-based approach. Unlike contemporary frame-based models, our network pipeline processes an input video clip as a single 3D volume to incorporate temporal information. The central idea of our formulation is to solve the video instance segmentation task as a tag assignment problem, such that generating distinct tag values essentially separates individual object instances across the video sequence (here each tag could be any arbitrary value between 0 and 1). To this end, we propose a novel spatio-temporal tagging loss that allows for sufficient separation of different objects as well as necessary identification of different instances of the same object. Furthermore, we present a tag-based attention module that improves instance tags, while concurrently learning instance propagation within a video. Evaluations demonstrate that our method provides competitive results on YouTube-VIS and DAVIS-19 datasets, and has minimum run-time compared to other state-of-the-art performance methods.

Territory Design for Dynamic Multi-Period Vehicle Routing Problem with Time Windows

This study introduces the Territory Design for Dynamic Multi-Period Vehicle Routing Problem with Time Windows (TD-DMPVRPTW), motivated by a real-world application at a food company's distribution center. This problem deals with the design of contiguous and compact territories for delivery of orders from a depot to a set of customers, with time windows, over a multi-period planning horizon. Customers and their demands vary dynamically over time. The problem is modeled as a mixed-integer linear program (MILP) and solved by a proposed heuristic. The heuristic solutions are compared with the proposed MILP solutions on a set of small artificial instances and the food company's solutions on a set of real-world instances. Computational results show that the proposed algorithm can yield high-quality solutions within moderate running times.

Colonoscopy polyp detection with massive endoscopic images

We improved an existing end-to-end polyp detection model with better average precision validated by different data sets with trivial cost on detection speed. Previous work on detecting polyps within colonoscopy \cite{Chen2018} provided an efficient end-to-end solution to alleviate doctor's examination overhead. However, our later experiments found this framework is not as robust as before as the condition of polyp capturing varies. In this work, we conducted several studies on data set, identifying main issues that causes low precision rate in the task of polyp detection. We used an optimized anchor generation methods to get better anchor box shape and more boxes are used for detection as we believe this is necessary for small object detection. A alternative backbone is used to compensate the heavy time cost introduced by dense anchor box regression. With use of the attention gate module, our model can achieve state-of-the-art polyp detection performance while still maintain real-time detection speed.