Cutting Through the Noise: On-the-fly Outlier Detection for Robust Training of Machine Learning Interatomic Potentials

The accuracy of machine learning interatomic potentials suffers from reference data that contains numerical noise. Often originating from unconverged or inconsistent electronic-structure calculations, this noise is challenging to identify. Existing mitigation strategies such as manual filtering or iterative refinement of outliers, require either substantial expert effort or multiple expensive retraining cycles, making them difficult to scale to large datasets. Here, we introduce an on-the-fly outlier detection scheme that automatically down-weights noisy samples, without requiring additional reference calculations. By tracking the loss distribution via an exponential moving average, this unsupervised method identifies outliers throughout a single training run. We show that this approach prevents overfitting and matches the performance of iterative refinement baselines with significantly reduced overhead. The method's effectiveness is demonstrated by recovering accurate physical observables for liquid water from unconverged reference data, including diffusion coefficients. Furthermore, we validate its scalability by training a foundation model for organic chemistry on the SPICE dataset, where it reduces energy errors by a factor of three. This framework provides a simple, automated solution for training robust models on imperfect datasets across dataset sizes.

Committee neural network potentials control generalization errors and enable active learning

It is well known in the field of machine learning that committee models improve accuracy, provide generalization error estimates, and enable active learning strategies. In this work, we adapt these concepts to interatomic potentials based on artificial neural networks. Instead of a single model, multiple models that share the same atomic environment descriptors yield an average that outperforms its individual members as well as a measure of the generalization error in the form of the committee disagreement. We not only use this disagreement to identify the most relevant configurations to build up the model's training set in an active learning procedure, but also monitor and bias it during simulations to control the generalization error. This facilitates the adaptive development of committee neural network potentials and their training sets, while keeping the number of ab initio calculations to a minimum. To illustrate the benefits of this methodology, we apply it to the development of a committee model for water in the condensed phase. Starting from a single reference ab initio simulation, we use active learning to expand into new state points and to describe the quantum nature of the nuclei. The final model, trained on 814 reference calculations, yields excellent results under a range of conditions, from liquid water at ambient and elevated temperatures and pressures to different phases of ice, and the air-water interface - all including nuclear quantum effects. The methods introduced here will enable the systematic development of robust machine learning models for a broad range of systems.

Automated Fitting of Neural Network Potentials at Coupled Cluster Accuracy: Protonated Water Clusters as Testing Ground

Highly accurate potential energy surfaces are of key interest for the detailed understanding and predictive modeling of chemical systems. In recent years, several new types of force fields, which are based on machine learning algorithms and fitted to ab initio reference calculations, have been introduced to meet this requirement. Here we show how high-dimensional neural network potentials can be employed to automatically generate the potential energy surface of finite sized clusters at coupled cluster accuracy, namely CCSD(T*)-F12a/aug-cc-pVTZ. The developed automated procedure utilizes the established intrinsic properties of the model such that the configurations for the training set are selected in an unbiased and efficient way to minimize the computational effort of expensive reference calculations. These ideas are applied to protonated water clusters from the hydronium cation, H$_3$O$^+$, up to the tetramer, H$_9$O$_{4}^{+}$, and lead to a common potential energy surface that describes all these systems at essentially converged coupled cluster accuracy with a fitting error of 0.06 kJ/mol per atom. The fit is validated in detail and separately for all clusters up to the tetramer and yields reliable results not only for stationary points, but also for reaction pathways, intermediate configurations, as well as different sampling techniques. Per design the NNPs constructed in this fashion can handle very different conditions including the quantum nature of the nuclei and enhanced sampling techniques covering very low as well as high temperatures. This enables fast and exhaustive exploration of the targeted protonated water clusters at essentially converged interactions. In addition, the automated process will allow one to tackle finite systems much beyond the present case.