Alert button

Architecture Agnostic Neural Networks

Nov 05, 2020
Sabera Talukder, Guruprasad Raghavan, Yisong Yue

Figure 1 for Architecture Agnostic Neural Networks
Figure 2 for Architecture Agnostic Neural Networks
Figure 3 for Architecture Agnostic Neural Networks
Figure 4 for Architecture Agnostic Neural Networks

Share this with someone who'll enjoy it:

In this paper, we explore an alternate method for synthesizing neural network architectures, inspired by the brain's stochastic synaptic pruning. During a person's lifetime, numerous distinct neuronal architectures are responsible for performing the same tasks. This indicates that biological neural networks are, to some degree, architecture agnostic. However, artificial networks rely on their fine-tuned weights and hand-crafted architectures for their remarkable performance. This contrast begs the question: Can we build artificial architecture agnostic neural networks? To ground this study we utilize sparse, binary neural networks that parallel the brain's circuits. Within this sparse, binary paradigm we sample many binary architectures to create families of architecture agnostic neural networks not trained via backpropagation. These high-performing network families share the same sparsity, distribution of binary weights, and succeed in both static and dynamic tasks. In summation, we create an architecture manifold search procedure to discover families or architecture agnostic neural networks.

View paper onarxiv iconopen_review iconOpenReview

Share this with someone who'll enjoy it: