Generating stable molecules using imitation and reinforcement learning

Chemical space is routinely explored by machine learning methods to discover interesting molecules, before time-consuming experimental synthesizing is attempted. However, these methods often rely on a graph representation, ignoring 3D information necessary for determining the stability of the molecules. We propose a reinforcement learning approach for generating molecules in cartesian coordinates allowing for quantum chemical prediction of the stability. To improve sample-efficiency we learn basic chemical rules from imitation learning on the GDB-11 database to create an initial model applicable for all stoichiometries. We then deploy multiple copies of the model conditioned on a specific stoichiometry in a reinforcement learning setting. The models correctly identify low energy molecules in the database and produce novel isomers not found in the training set. Finally, we apply the model to larger molecules to show how reinforcement learning further refines the imitation learning model in domains far from the training data.

Atomistic Structure Learning Algorithm with surrogate energy model relaxation

The recently proposed Atomistic Structure Learning Algorithm (ASLA) builds on neural network enabled image recognition and reinforcement learning. It enables fully autonomous structure determination when used in combination with a first-principles total energy calculator, e.g. a density functional theory (DFT) program. To save on the computational requirements, ASLA utilizes the DFT program in a single-point mode, i.e. without allowing for relaxation of the structural candidates according to the force information at the DFT level. In this work, we augment ASLA to establish a surrogate energy model concurrently with its structure search. This enables approximative but computationally cheap relaxation of the structural candidates before the single-point energy evaluation with the computationally expensive DFT program. We demonstrate a significantly increased performance of ASLA for building benzene while utilizing a surrogate energy landscape. Further we apply this model-enhanced ASLA in a thorough investigation of the c(4x8) phase of the Ag(111) surface oxide. ASLA successfully identifies a surface reconstruction which has previously only been guessed on the basis of scanning tunnelling microscopy images.