Effectiveness of LLM-based Software Diversity for Reliability Improvement -- an Empirical Study

Software diversity has been extensively studied as a means of reducing the risk of common-mode failures. Classic work showed that the central issue is whether failures of diversely redundant components overlap in ways that limit the reliability gains. Traditional software diversity is costly to obtain, since it requires multiple implementations as well as the corresponding validation, maintenance, and deployment effort. Recent advances in Large Language Models (LLMs) may change this. LLMs enable inexpensive code generation: they produce many candidate implementations of the same specification quickly, across different models, decoding settings, and programming languages. This raises a natural question: can LLMs serve as practical generators of software diversity, and how much reliability improvement can that diversity actually provide? In this paper, we extend classical empirical studies of software diversity in human-written programs to LLM-generated code. We study three specifications using both historical human-written programs and large pools of LLM-generated ones evaluated under a common compilation, sandboxing, and exhaustive test suite. We explore LLM diversity along multiple axes, including model family, generation temperature, and programming language. Reliability improvement is evaluated in a 1-out-of-2 configuration across both homogeneous and heterogeneous program populations, including within-LLM pairings and pairings across programming languages and across LLM-generated and human-written programs. The results show that combining LLM-generated programs, especially in heterogeneous settings, can yield reliability gains, although this is partly conditioned by the programming language and generation setting. Taken together, these findings suggest that LLMs provide a scalable source of comparatively low-cost programs whose diversity can be leveraged for reliability improvement.

Variational autoencoders stabilise TCN performance when classifying weakly labelled bioacoustics data

Passive acoustic monitoring (PAM) data is often weakly labelled, audited at the scale of detection presence or absence on timescales of minutes to hours. Moreover, this data exhibits great variability from one deployment to the next, due to differences in ambient noise and the signals across sources and geographies. This study proposes a two-step solution to leverage weakly annotated data for training Deep Learning (DL) detection models. Our case study involves binary classification of the presence/absence of sperm whale (\textit{Physeter macrocephalus}) click trains in 4-minute-long recordings from a dataset comprising diverse sources and deployment conditions to maximise generalisability. We tested methods for extracting acoustic features from lengthy audio segments and integrated Temporal Convolutional Networks (TCNs) trained on the extracted features for sequence classification. For feature extraction, we introduced a new approach using Variational AutoEncoders (VAEs) to extract information from both waveforms and spectrograms, which eliminates the necessity for manual threshold setting or time-consuming strong labelling. For classification, TCNs were trained separately on sequences of either VAE embeddings or handpicked acoustic features extracted from the waveform and spectrogram representations using classical methods, to compare the efficacy of the two approaches. The TCN demonstrated robust classification capabilities on a validation set, achieving accuracies exceeding 85\% when applied to 4-minute acoustic recordings. Notably, TCNs trained on handpicked acoustic features exhibited greater variability in performance across recordings from diverse deployment conditions, whereas those trained on VAEs showed a more consistent performance, highlighting the robust transferability of VAEs for feature extraction across different deployment conditions.

How likely is a random graph shift-enabled?

The shift-enabled property of an underlying graph is essential in designing distributed filters. This article discusses when a random graph is shift-enabled. In particular, popular graph models ER, WS, BA random graph are used, weighted and unweighted, as well as signed graphs. Our results show that the considered unweighted connected random graphs are shift-enabled with high probability when the number of edges is moderately high. However, very dense graphs, as well as fully connected graphs, are not shift-enabled. Interestingly, this behaviour is not observed for weighted connected graphs, which are always shift-enabled unless the number of edges in the graph is very low.

Robust Deep Graph Based Learning for Binary Classification

Convolutional neural network (CNN)-based feature learning has become state of the art, since given sufficient training data, CNN can significantly outperform traditional methods for various classification tasks. However, feature learning becomes more difficult if some training labels are noisy. With traditional regularization techniques, CNN often overfits to the noisy training labels, resulting in sub-par classification performance. In this paper, we propose a robust binary classifier, based on CNNs, to learn deep metric functions, which are then used to construct an optimal underlying graph structure used to clean noisy labels via graph Laplacian regularization (GLR). GLR is posed as a convex maximum a posteriori (MAP) problem solved via convex quadratic programming (QP). To penalize samples around the decision boundary, we propose two regularized loss functions for semi-supervised learning. The binary classification experiments on three datasets, varying in number and type of features, demonstrate that given a noisy training dataset, our proposed networks outperform several state-of-the-art classifiers, including label-noise robust support vector machine, CNNs with three different robust loss functions, model-based GLR, and dynamic graph CNN classifiers.