Developing Convolutional Neural Networks using a Novel Lamarckian Co-Evolutionary Algorithm

Neural Architecture Search (NAS) methods autonomously discover high-accuracy neural network architectures, outperforming manually crafted ones. However, The NAS methods require high computational costs due to the high dimension search space and the need to train multiple candidate solutions. This paper introduces LCoDeepNEAT, an instantiation of Lamarckian genetic algorithms, which extends the foundational principles of the CoDeepNEAT framework. LCoDeepNEAT co-evolves CNN architectures and their respective final layer weights. The evaluation process of LCoDeepNEAT entails a single epoch of SGD, followed by the transference of the acquired final layer weights to the genetic representation of the network. In addition, it expedites the process of evolving by imposing restrictions on the architecture search space, specifically targeting architectures comprising just two fully connected layers for classification. Our method yields a notable improvement in the classification accuracy of candidate solutions throughout the evolutionary process, ranging from 2% to 5.6%. This outcome underscores the efficacy and effectiveness of integrating gradient information and evolving the last layer of candidate solutions within LCoDeepNEAT. LCoDeepNEAT is assessed across six standard image classification datasets and benchmarked against eight leading NAS methods. Results demonstrate LCoDeepNEAT's ability to swiftly discover competitive CNN architectures with fewer parameters, conserving computational resources, and achieving superior classification accuracy compared to other approaches.

Modular Grammatical Evolution for the Generation of Artificial Neural Networks

This paper presents a novel method, called Modular Grammatical Evolution (MGE), towards validating the hypothesis that restricting the solution space of NeuroEvolution to modular and simple neural networks enables the efficient generation of smaller and more structured neural networks while providing acceptable (and in some cases superior) accuracy on large data sets. MGE also enhances the state-of-the-art Grammatical Evolution (GE) methods in two directions. First, MGE's representation is modular in that each individual has a set of genes, and each gene is mapped to a neuron by grammatical rules. Second, the proposed representation mitigates two important drawbacks of GE, namely the low scalability and weak locality of representation, towards generating modular and multi-layer networks with a high number of neurons. We define and evaluate five different forms of structures with and without modularity using MGE and find single-layer modules with no coupling more productive. Our experiments demonstrate that modularity helps in finding better neural networks faster. We have validated the proposed method using ten well-known classification benchmarks with different sizes, feature counts, and output class count. Our experimental results indicate that MGE provides superior accuracy with respect to existing NeuroEvolution methods and returns classifiers that are significantly simpler than other machine learning generated classifiers. Finally, we empirically demonstrate that MGE outperforms other GE methods in terms of locality and scalability properties.