Beyond Uniform Token-Level Trust Region in LLM Reinforcement Learning

Reinforcement learning with verifiable rewards (RLVR) has become standard for improving LLM reasoning. However, existing PPO-style trust-region mechanisms remain position-agnostic by enforcing uniform thresholds across all tokens independently. This pointwise treatment conflicts with autoregressive generation in two critical ways. First, uniform thresholds ignore autoregressive asymmetry. Early-stage deviations produce compounding sequence-level drift, causing static thresholds to under-regulate early divergence and excessively constrain late-stage exploration. Second, evaluating token-level divergence in isolation overlooks cumulative prefix drift, granting the same divergence allowance regardless of how far the conditioning history has already deviated from the rollout policy. To address this limitation, we propose CPPO (Cumulative Prefix-divergence Policy Optimization), a token-level masking rule that aligns updates with a finite-horizon policy-improvement bound via two coupled mechanisms. First, a position-weighted threshold imposes stricter limits at early positions whose effects persist longer, relaxing constraints for late-stage tokens. Second, a cumulative prefix budget tracks historical deviations, dynamically restricting further token-level deviation to prevent compounding errors along the prefix. Empirically, CPPO enhances training stability and significantly improves reasoning accuracy across various model scales.

Data Fusion for BS-UE Cooperative MIMO-OFDM ISAC

Integrated sensing and communication (ISAC) is a promising technique for expanding the functionalities of wireless networks with enhanced spectral efficiency. The 3rd Generation Partnership Project (3GPP) has defined six basic sensing operation modes in wireless networks. To further enhance the sensing capability of wireless networks, this paper proposes a new sensing operation mode, i.e., the base station (BS) and user equipment (UE) cooperative sensing. Specifically, after decoding the communication data, the UE further processes the received signal to extract the target sensing information. We propose an efficient algorithm for fusing the sensing results obtained by the BS and UE, by exploiting the geometric relationship among BS, UE and targets as well as the expected sensing quality in the BS monostatic and BS-UE bistatic sensing. The results show that the proposed data fusion method for cooperative sensing can effectively improve the position and velocity estimation accuracy of multiple targets, and provide a new approach on the expansion of the sensing pattern.