Prot2Token: A Unified Framework for Protein Modeling via Next-Token Prediction

The diverse nature of protein prediction tasks has traditionally necessitated specialized models, hindering the development of broadly applicable and computationally efficient Protein Language Models (PLMs). In this work, we introduce Prot2Token, a unified framework that overcomes these challenges by converting a wide spectrum of protein-related predictions, from sequence-level properties and residue-specific attributes to complex inter-protein interactions, into a standardized next-token prediction format. At its core, Prot2Token employs an autoregressive decoder, conditioned on embeddings from pre-trained protein encoders and guided by learnable task tokens, to perform diverse predictions. This architecture uniquely facilitates multi-task learning, enabling a single model to master numerous tasks with improved efficiency. We present extensive experimental validation across a variety of benchmarks, demonstrating Prot2Tokens strong predictive power in different types of protein-prediction tasks. Key results include significant speedups (e.g., near 1000x over AlphaFold2 with MSA) and performance often matching or exceeding specialized approaches. Beyond that, we introduce an auxiliary self-supervised decoder pre-training approach to improve spatially sensitive task performance. Prot2Token thus offers a significant step towards a versatile, high-throughput paradigm for protein modeling, promising to accelerate biological discovery and the development of novel therapeutics. The code is available at https://github.com/mahdip72/prot2token .

Diffusion Models in Bioinformatics: A New Wave of Deep Learning Revolution in Action

Denoising diffusion models have emerged as one of the most powerful generative models in recent years. They have achieved remarkable success in many fields, such as computer vision, natural language processing (NLP), and bioinformatics. Although there are a few excellent reviews on diffusion models and their applications in computer vision and NLP, there is a lack of an overview of their applications in bioinformatics. This review aims to provide a rather thorough overview of the applications of diffusion models in bioinformatics to aid their further development in bioinformatics and computational biology. We start with an introduction of the key concepts and theoretical foundations of three cornerstone diffusion modeling frameworks (denoising diffusion probabilistic models, noise-conditioned scoring networks, and stochastic differential equations), followed by a comprehensive description of diffusion models employed in the different domains of bioinformatics, including cryo-EM data enhancement, single-cell data analysis, protein design and generation, drug and small molecule design, and protein-ligand interaction. The review is concluded with a summary of the potential new development and applications of diffusion models in bioinformatics.

Detection of Malfunctioning Smart Electricity Meter

In this paper, a method for malfunctioning smart meter detection, based on Long Short-Term Memory (LSTM) and Temporal Phase Convolutional Neural Network (TPCNN), is proposed originally. This method is very useful for some developing countries where smart meters have not been popularized but in high demand. In addition, it is a new topic that people try to increase the service life span of smart meters to prevent unnecessary waste by detecting malfunctioning meters. We are the first people complete a combination of malfunctioning meters detection and prediction model based on deep learning methods. To the best our knowledge, our approach is the first method that achieves the malfunctioning meter detection of specific residential areas with their residents' data in practice. The procedure proposed creatively in this paper mainly consists of four components: data collecting and cleaning, prediction about electricity consumption based on LSTM, sliding window detection, and single user classification based on CNN. To make better classifying of malfunctioned user meters, we combine recurrence plots as image-input and combine them with sequence-input, which is the first work that applies one and two dimensions as two paths CNN's input for sequence data classification. Finally, many classical methods are compared with the method proposed in this paper. After comparison with classical methods, Elastic Net and Gradient Boosting Regression, the result shows that our method has higher accuracy. The average area under the Receiver Operating Characteristic (ROC) curve is 0.80 and the standard deviation is 0.04. The average area under the Precision-Recall Curve (PRC) is 0.84.