Agent Step Value: Auditing Evaluator-Channel Reversals in Black-Box Agent Traces

When evaluator-derived step rewards are pooled or compared across scoring channels, their sign is treated as transportable. Yet the same frozen transition can change sign with the scoring channel. Process rewards vary agent states, while evaluator audits vary scoring configurations; neither first difference isolates their interaction. We define Agent Step Value (ASV) as channel-indexed target-margin gain and identify the missing state-by-channel interaction by replaying complete faces. Of 1,100 PubMed open question-answering transitions, 1,004 were complete across four cyclic layouts. Their mean update is positive under direct scoring (+0.163 [0.102, 0.218]) and negative with an externally generated view (-0.160 [-0.244, -0.079]), giving an interaction of -0.323 [-0.418, -0.232]. Across paired transitions, 507/1,004 cross zero in one direction or the other. Matched templates isolate a generated-minus-quote interaction of -1.121 [-1.534, -0.703]. Its direction remains negative after the readout and evaluator-stack changes, and quote yields a higher paired area under the receiver operating characteristic curve (AUC) for stored success at all three bridge vertices. Because the two views preserve the same task information conditional on the retained state, a representation-invariance null predicts equal responses and success rankings across them. ASV provides a transport audit for evaluator-derived step measurements; causal actor credit lies outside its estimand.

Agent Step Value: State-Transition Measurement with State-Grounded LLM Evaluators

Most agent evaluations collapse a multi-step trace into a final answer, a success flag, or a trajectory-level score. These aggregates obscure the diagnostic question developers need most: which action changed the state in a useful direction? We introduce Agent Step Value (ASV), a state-transition measurement framework that scores each observed action by the change it induces in a state-grounded evaluator's distribution over fixed candidate outcomes. ASV renders redacted before/after state projections, uses a stateless LLM evaluator to assign candidate log scores, and reports both gold-free belief diagnostics and offline oracle validation metrics. A label-free rationale pass separates evaluator deliberation from one-token option scoring, preserving candidate likelihoods while exposing leakage and floor-score events. On 100 reviewed open-QA evidence-seeking tasks with live PubMed retrieval, a partially live DeepSeek actor, and DeepSeek log-probability scoring, ASV evaluates 1,100 steps and 2,200 states. Under the fixed-layout rationale-conditioned protocol, mean gold-margin gain is -2.335 (trajectory-bootstrap 95\% CI [-3.395, -1.272]), entropy movement is 0.000, and mean Bayesian surprise is 2.693. ASV therefore localizes constructive and destructive belief pivots that final-answer scores and entropy-only step metrics miss. We release the standalone ASV Eval toolkit.