Agents' Last Exam

Recent AI systems have achieved strong results on a wide range of benchmarks, yet these gains have not translated into economically meaningful deployment across many professional domains. We argue that this gap is largely an evaluation problem: widely used benchmarks lack sustained performance measurement on real and economically valuable workflows. This paper introduces Agents' Last Exam (ALE), a benchmark designed to evaluate AI agents on long-horizon, economically valuable, real-world tasks with verifiable outcomes. Developed in collaboration with 250+ industry experts, ALE covers non-physical industries defined with reference to O*NET / SOC 2018 (the U.S. federal occupational taxonomy). It is organized around a task taxonomy with 55 subfields grouped into 13 industry clusters covering 1K+ tasks. Current results show that the hardest tier remains far from saturated: across mainstream harness and backbone configurations, the average full pass rate is 2.6%. ALE is designed as a living benchmark: its task pool grows continuously as new workflows and industries are onboarded. More broadly, ALE is intended not merely as another leaderboard, but as an instrument for closing the gap between benchmark success and GDP-relevant impact.

AVP-AP: Self-supervised Automatic View Positioning in 3D cardiac CT via Atlas Prompting

Automatic view positioning is crucial for cardiac computed tomography (CT) examinations, including disease diagnosis and surgical planning. However, it is highly challenging due to individual variability and large 3D search space. Existing work needs labor-intensive and time-consuming manual annotations to train view-specific models, which are limited to predicting only a fixed set of planes. However, in real clinical scenarios, the challenge of positioning semantic 2D slices with any orientation into varying coordinate space in arbitrary 3D volume remains unsolved. We thus introduce a novel framework, AVP-AP, the first to use Atlas Prompting for self-supervised Automatic View Positioning in the 3D CT volume. Specifically, this paper first proposes an atlas prompting method, which generates a 3D canonical atlas and trains a network to map slices into their corresponding positions in the atlas space via a self-supervised manner. Then, guided by atlas prompts corresponding to the given query images in a reference CT, we identify the coarse positions of slices in the target CT volume using rigid transformation between the 3D atlas and target CT volume, effectively reducing the search space. Finally, we refine the coarse positions by maximizing the similarity between the predicted slices and the query images in the feature space of a given foundation model. Our framework is flexible and efficient compared to other methods, outperforming other methods by 19.8% average structural similarity (SSIM) in arbitrary view positioning and achieving 9% SSIM in two-chamber view compared to four radiologists. Meanwhile, experiments on a public dataset validate our framework's generalizability.

Distributed Graph Neural Networks for Optimizing Wireless Networks: Message Passing Over-the-Air

Distributed power allocation is important for interference-limited wireless networks with dense transceiver pairs. In this paper, we aim to design low signaling overhead distributed power allocation schemes by using graph neural networks (GNNs), which are scalable to the number of wireless links. We first apply the message passing neural network (MPNN), a unified framework of GNN, to solve the problem. We show that the signaling overhead grows quadratically as the network size increases. Inspired from the over-the-air computation (AirComp), we then propose an Air-MPNN framework, where the messages from neighboring nodes are represented by the transmit power of pilots and can be aggregated efficiently by evaluating the total interference power. The signaling overhead of Air-MPNN grows linearly as the network size increases, and we prove that Air-MPNN is permutation invariant. To further reduce the signaling overhead, we propose the Air message passing recurrent neural network (Air-MPRNN), where each node utilizes the graph embedding and local state in the previous frame to update the graph embedding in the current frame. Since existing communication systems send a pilot during each frame, Air-MPRNN can be integrated into the existing standards by adjusting pilot power. Simulation results validate the scalability of the proposed frameworks, and show that they outperform the existing power allocation algorithms in terms of sum-rate for various system parameters.