Effectiveness of LLM-based Software Diversity for Reliability Improvement -- an Empirical Study

Software diversity has been extensively studied as a means of reducing the risk of common-mode failures. Classic work showed that the central issue is whether failures of diversely redundant components overlap in ways that limit the reliability gains. Traditional software diversity is costly to obtain, since it requires multiple implementations as well as the corresponding validation, maintenance, and deployment effort. Recent advances in Large Language Models (LLMs) may change this. LLMs enable inexpensive code generation: they produce many candidate implementations of the same specification quickly, across different models, decoding settings, and programming languages. This raises a natural question: can LLMs serve as practical generators of software diversity, and how much reliability improvement can that diversity actually provide? In this paper, we extend classical empirical studies of software diversity in human-written programs to LLM-generated code. We study three specifications using both historical human-written programs and large pools of LLM-generated ones evaluated under a common compilation, sandboxing, and exhaustive test suite. We explore LLM diversity along multiple axes, including model family, generation temperature, and programming language. Reliability improvement is evaluated in a 1-out-of-2 configuration across both homogeneous and heterogeneous program populations, including within-LLM pairings and pairings across programming languages and across LLM-generated and human-written programs. The results show that combining LLM-generated programs, especially in heterogeneous settings, can yield reliability gains, although this is partly conditioned by the programming language and generation setting. Taken together, these findings suggest that LLMs provide a scalable source of comparatively low-cost programs whose diversity can be leveraged for reliability improvement.

Rethinking Reinforcement Fine-Tuning in LVLM: Convergence, Reward Decomposition, and Generalization

Reinforcement fine-tuning with verifiable rewards (RLVR) has emerged as a powerful paradigm for equipping large vision-language models (LVLMs) with agentic capabilities such as tool use and multi-step reasoning. Despite striking empirical successes, most notably Visual Agentic Reinforcement Fine-Tuning (Visual-ARFT), the theoretical underpinnings of this paradigm remain poorly understood. In particular, two critical questions lack rigorous answers: (i)~how does the composite structure of verifiable rewards (format compliance, answer accuracy, tool executability) affect the convergence of Group Relative Policy Optimization (GRPO), and (ii)~why does training on a small set of tool-augmented tasks transfer to out-of-distribution domains? We address these gaps by introducing the \emph{Tool-Augmented Markov Decision Process} (TA-MDP), a formal framework that models multimodal agentic decision-making with bounded-depth tool calls. Within this framework, we establish three main results. First, we prove that GRPO under composite verifiable rewards converges to a first-order stationary point at rate $O(1/\sqrt{T})$ with explicit dependence on the number of reward components and group size (\textbf{Theorem~1}). Second, we derive a \emph{Reward Decomposition Theorem} that bounds the sub-optimality gap between decomposed per-component optimization and joint optimization, providing a precise characterization of when reward decomposition is beneficial (\textbf{Theorem~2}). Third, we establish a PAC-Bayes generalization bound for tool-augmented policies that explains the strong out-of-distribution transfer observed in Visual-ARFT (\textbf{Theorem~3}).