Learning Robot Swarm Tactics over Complex Adversarial Environments

To accomplish complex swarm robotic missions in the real world, one needs to plan and execute a combination of single robot behaviors, group primitives such as task allocation, path planning, and formation control, and mission-specific objectives such as target search and group coverage. Most such missions are designed manually by teams of robotics experts. Recent work in automated approaches to learning swarm behavior has been limited to individual primitives with sparse work on learning complete missions. This paper presents a systematic approach to learn tactical mission-specific policies that compose primitives in a swarm to accomplish the mission efficiently using neural networks with special input and output encoding. To learn swarm tactics in an adversarial environment, we employ a combination of 1) map-to-graph abstraction, 2) input/output encoding via Pareto filtering of points of interest and clustering of robots, and 3) learning via neuroevolution and policy gradient approaches. We illustrate this combination as critical to providing tractable learning, especially given the computational cost of simulating swarm missions of this scale and complexity. Successful mission completion outcomes are demonstrated with up to 60 robots. In addition, a close match in the performance statistics in training and testing scenarios shows the potential generalizability of the proposed framework.

Metamodel Based Forward and Inverse Design for Passive Vibration Suppression

Aperiodic metamaterials represent a class of structural systems that are composed of different building blocks (cells), instead of a self-repeating chain of the same unit cells. Optimizing aperiodic cellular structural systems thus presents high-dimensional problems that are challenging to solve using purely high-fidelity structural optimization approaches. Specialized analytical modeling along with metamodel based optimization can provide a more tractable alternative solution approach. To this end, this paper presents a design automation framework applied to a 1D metamaterial system, namely a drill string, where vibration suppression is of utmost importance. The drill string comprises a set of nonuniform rings attached to the outer surface of a longitudinal rod. As such, the resultant system can now be perceived as an aperiodic 1D metamaterial with each ring/gap representing a cell. Despite being a 1D system, the simultaneous consideration of multiple DoF (i.e., torsional, axial, and lateral motions) poses significant computational challenges. Therefore, a transfer matrix method (TMM) is employed to analytically determine the frequency response of the drill string. A suite of neural networks (ANN) is trained on TMM samples (which present minute-scale computing costs per evaluation), to model the frequency response. ANN-based optimization is then performed to minimize mass subject to constraints on the gap between consecutive resonance peaks in one case, and minimizing this gap in the second case, leading to crucial improvements over baselines. Further novel contribution occurs through the development of an inverse modeling approach that can instantaneously produce the 1D metamaterial design with minimum mass for a given desired non-resonant frequency range. This is accomplished by using invertible neural networks, and results show promising alignment with forward solutions.

Training Detection-Range-Frugal Cooperative Collision Avoidance Models for Quadcopters via Neuroevolution

Cooperative autonomous approaches to avoiding collisions among small Unmanned Aerial Vehicles (UAVs) is central to safe integration of UAVs within the civilian airspace. One potential online cooperative approach is the concept of reciprocal actions, where both UAVs take pre-trained mutually coherent actions that do not require active online coordination (thereby avoiding the computational burden and risk associated with it). This paper presents a learning based approach to train such reciprocal maneuvers. Neuroevolution, which uses evolutionary algorithms to simultaneously optimize the topology and weights of neural networks, is used as the learning method -- which operates over a set of sample approach scenarios. Unlike most existing work (that minimize travel distance, energy or risk), the training process here focuses on the objective of minimizing the required detection range; this has important practical implications w.r.t. alleviating the dependency on sophisticated sensing and their reliability under various environments. A specialized design of experiments and line search is used to identify the minimum detection range for each sample scenarios. In order to allow an efficient training process, a classifier is used to discard actions (without simulating them) where the controller would fail. The model obtained via neuroevolution is observed to generalize well to (i.e., successful collision avoidance over) unseen approach scenarios.

Adaptive Genomic Evolution of Neural Network Topologies (AGENT) for State-to-Action Mapping in Autonomous Agents

Neuroevolution is a process of training neural networks (NN) through an evolutionary algorithm, usually to serve as a state-to-action mapping model in control or reinforcement learning-type problems. This paper builds on the Neuro Evolution of Augmented Topologies (NEAT) formalism that allows designing topology and weight evolving NNs. Fundamental advancements are made to the neuroevolution process to address premature stagnation and convergence issues, central among which is the incorporation of automated mechanisms to control the population diversity and average fitness improvement within the neuroevolution process. Insights into the performance and efficiency of the new algorithm is obtained by evaluating it on three benchmark problems from the Open AI platform and an Unmanned Aerial Vehicle (UAV) collision avoidance problem.

Multi-criteria Evolution of Neural Network Topologies: Balancing Experience and Performance in Autonomous Systems

Majority of Artificial Neural Network (ANN) implementations in autonomous systems use a fixed/user-prescribed network topology, leading to sub-optimal performance and low portability. The existing neuro-evolution of augmenting topology or NEAT paradigm offers a powerful alternative by allowing the network topology and the connection weights to be simultaneously optimized through an evolutionary process. However, most NEAT implementations allow the consideration of only a single objective. There also persists the question of how to tractably introduce topological diversification that mitigates overfitting to training scenarios. To address these gaps, this paper develops a multi-objective neuro-evolution algorithm. While adopting the basic elements of NEAT, important modifications are made to the selection, speciation, and mutation processes. With the backdrop of small-robot path-planning applications, an experience-gain criterion is derived to encapsulate the amount of diverse local environment encountered by the system. This criterion facilitates the evolution of genes that support exploration, thereby seeking to generalize from a smaller set of mission scenarios than possible with performance maximization alone. The effectiveness of the single-objective (optimizing performance) and the multi-objective (optimizing performance and experience-gain) neuro-evolution approaches are evaluated on two different small-robot cases, with ANNs obtained by the multi-objective optimization observed to provide superior performance in unseen scenarios.