The Role of Model Architecture and Scale in Predicting Molecular Properties: Insights from Fine-Tuning RoBERTa, BART, and LLaMA

This study introduces a systematic framework to compare the efficacy of Large Language Models (LLMs) for fine-tuning across various cheminformatics tasks. Employing a uniform training methodology, we assessed three well-known models-RoBERTa, BART, and LLaMA-on their ability to predict molecular properties using the Simplified Molecular Input Line Entry System (SMILES) as a universal molecular representation format. Our comparative analysis involved pre-training 18 configurations of these models, with varying parameter sizes and dataset scales, followed by fine-tuning them on six benchmarking tasks from DeepChem. We maintained consistent training environments across models to ensure reliable comparisons. This approach allowed us to assess the influence of model type, size, and training dataset size on model performance. Specifically, we found that LLaMA-based models generally offered the lowest validation loss, suggesting their superior adaptability across tasks and scales. However, we observed that absolute validation loss is not a definitive indicator of model performance - contradicts previous research - at least for fine-tuning tasks: instead, model size plays a crucial role. Through rigorous replication and validation, involving multiple training and fine-tuning cycles, our study not only delineates the strengths and limitations of each model type but also provides a robust methodology for selecting the most suitable LLM for specific cheminformatics applications. This research underscores the importance of considering model architecture and dataset characteristics in deploying AI for molecular property prediction, paving the way for more informed and effective utilization of AI in drug discovery and related fields.

Deep Learning based Modeling of Wireless Communication Channel with Fading

In the realm of wireless communication, stochastic modeling of channels is instrumental for the assessment and design of operational systems. Deep learning neural networks (DLNN), including generative adversarial networks (GANs), are being used to approximate wireless Orthogonal frequency-division multiplexing (OFDM) channels with fading and noise, using real measurement data. These models primarily focus on channel output (y) distribution given input x: p(y|x), limiting their application scope. DLNN channel models have been tested predominantly on simple simulated channels. In this paper, we build both GANs and feedforward neural networks (FNN) to approximate a more general channel model, which is represented by a conditional probability density function (PDF) of receiving signal or power of node receiving power Prx: f_p_rx|d(()), where is communication distance. The stochastic models are trained and tested for the impact of fading channels on transmissions of OFDM QAM modulated signal and transmissions of general signal regardless of modulations. New metrics are proposed for evaluation of modeling accuracy and comparisons of the GAN-based model with the FNN-based model. Extensive experiments on Nakagami fading channel show accuracy and the effectiveness of the approaches.