Adaptive Knowledge Routing for Complex Reasoning in LLMs¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/ordealm/PACER
Area: LLM Reasoning
Keywords: Speech Reasoning, Self-Correction, Condition Updating, Reinforcement Learning, Perception-Aware RL
TL;DR¶
This paper formulates self-corrected speech reasoning as an explicit condition-updating problem, constructs the controlled 13k-sample SCUR-Bench benchmark, identifies a widespread "final-condition adoption failure" across Speech LLMs, and proposes PACER—a GRPO-based reinforcement framework combining structured condition-aware rewards with speech-evidence contrast that slashes condition-related reasoning errors by 67.5%.
Background & Motivation¶
As speech large language models (Speech LLMs) rapidly advance from basic speech recognition and acoustic perception toward full-duplex conversational reasoning and interactive problem-solving, their deployment in everyday voice-driven applications has become increasingly viable. However, natural spontaneous human speech inherently abounds with self-corrections, where a speaker articulates an initial condition and subsequently revises it either immediately or retrospectively. Unlike acoustic disfluencies such as filled pauses, hesitations, or exact phrase repetitions, self-correction fundamentally alters the premise structure of the underlying task: obsolete conditions must be actively purged, while the revised, final valid conditions must serve as the exclusive operational foundation for downstream multi-step reasoning. Existing robustness benchmarks predominantly treat speech disfluencies as unstructured input noise and measure coarse aggregate performance degradation, leaving the internal mechanics of condition-state tracking largely unexamined.
A conventional failure attribution would typically classify reasoning breakdowns into either low-level speech perception errors (failing to recognize what the speaker corrected) or text-side logical miscalculations. Yet deep empirical analysis unveils a far more nuanced and pervasive cognitive breakdown across current state-of-the-art models: Speech LLMs frequently demonstrate near-perfect condition recovery in transcription tests and even explicitly list the corrected values in the opening lines of their textual response, yet completely revert to obsolete conditions once multi-step deduction unfolds. We term this structural phenomenon "final-condition adoption failure"—corrected conditions are readily recoverable and verbally acknowledged, but systematically fail to serve as active premises in subsequent reasoning steps.
To bridge this critical empirical and methodological gap, the authors construct SCUR-Bench, a controlled benchmark containing 13,295 structured instances organized along two orthogonal axes of intrinsic reasoning difficulty and self-correction complexity. Complemented by a rigorous three-path diagnostic protocol that cleanly decouples speech-side recovery, end-to-end speech reasoning, and text-only reference reasoning, the study demonstrates that final-condition adoption failure is a pervasive cross-model bottleneck. Core idea: Formulate self-corrected speech reasoning as an explicit condition-update problem, and introduce PACER—a GRPO-based reinforcement learning framework that couples fine-grained structured condition rewards with random speech-evidence contrast, compelling the language policy to maintain an active condition state grounded in acoustic evidence without updating the speech encoder.
Method¶
Overall Architecture¶
PACER is built upon the Group Relative Policy Optimization (GRPO) framework, specifically tailored to teach the text-side language policy to maintain and track dynamic task premises under frozen acoustic encoders. Given a spoken question \(x\) containing natural self-corrections, the frozen speech encoder extracts an intact representation \(z = \mathcal{E}_{\text{sp}}(x)\). The active policy \(\pi_\theta\) samples a group of \(G\) candidate responses \(\{y_i\}_{i=1}^G\) conditioned on \(z\). The architecture operates via two synchronized pathways: on the representation side, an auxiliary perturbed branch applies random temporal masking over speech frames to produce \(\tilde{z}\), constructing an unsupervised speech-evidence contrast term via token-level KL divergence between policy distributions to preserve acoustic sensitivity. On the supervision side, each candidate response is parsed and evaluated by an online structured judge (powered by Qwen3-14B) across decoupled condition adoption, reasoning logic consistency, and final-answer correctness dimensions. These fine-grained rewards, coupled with format regularizers, guide token-level policy updates via clipped surrogate advantages.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Spoken Input with Self-Correction x"] --> B["Frozen Speech Encoder Extracts z"]
B --> C["Policy Model Samples G Response Candidates"]
B --> D["Representation-Level Random Masking z̃"]
C & D --> E["Speech-Evidence Contrast & Entropy Constraints<br/>KL(π(·|z) || π(·|z̃)) Preserves Acoustic Sensitivity"]
C --> F["Online Structured LLM Multi-Path Judge<br/>Decouples Condition / Logic Chain / Final Answer"]
F --> G["Structured Condition-Aware Reward Calculation<br/>Rewards Active Use, Penalizes Obsolete Residue & Conflicts"]
E & G --> H["Group Advantage Normalization & Clipped GRPO Update<br/>Optimizes Text-Side Policy Parameters θ"]
Key Designs¶
1. Structured Condition-Aware Reward: Decoupling Premise Adoption from Surface Answer Alignment
In self-corrected reasoning, relying solely on verifiable final-answer matching introduces severe risks of false-positive credit assignment: a model might guess a number by chance, or superficially mention the corrected premise in its preface while quietly calculating with obsolete numbers. To counteract this, PACER implements a three-route structured online evaluator using Qwen3-14B that parses candidate responses into condition, reasoning, and answer fields, returning structured JSON verdicts. The condition score \(S_{\text{cond}}\) explicitly separates the shallow mention ratio \(m_i\) from the actual in-chain usage ratio \(u_i\), assigning dominant positive weight to active deployment (\(\alpha_u = 0.70\) vs. \(\alpha_m = 0.30\)). It simultaneously enforces a comprehensive penalty set \(\mathcal{C}^-\) covering obsolete-condition leakage, old–new premise conflicts, invalid condition contamination in reasoning, and wrong-premise chains. The logic score \(S_{\text{logic}}\) evaluates reasoning consistency under the adopted conditions across five structural verdicts (chain completeness, arithmetic correctness, state-update validity, self-consistency, and answer support). The answer score \(S_{\text{ans}}\) acts as a strict binary gate, granting reward only when the answer is correct and strictly grounded in the updated premises:
This mechanism rigorously separates superficial format mimicry from authentic premise-grounded derivation, penalizing models that fail to carry updated conditions into actual computation.
2. Speech-Evidence Contrast: Unsupervised Acoustic Perturbation for Preserving Evidence Sensitivity
The three-path diagnosis reveals that language policies undergoing text-centric reinforcement learning easily become desensitized to acoustic variations, causing them to neglect subtle spoken corrections even when acoustic cues are present in the encoder representations. Rather than fine-tuning the heavy speech encoder or requiring expensive frame-level correction timestamps, PACER introduces an unsupervised representation-level contrastive regularizer. Given intact speech representation \(z\), a random temporal mask over speech frames yields \(\tilde{z}\). PACER measures the token-level predictive distribution discrepancy across the intact and perturbed representations:
This contrast term does not function as an isolated correction-span detector; instead, it serves as an auxiliary sensitivity signal that penalizes the language policy for ignoring speech representation changes. The structured condition rewards then guide this heightened acoustic responsiveness specifically toward adopting final valid conditions. Combined with bidirectional predictive entropy constraints \(\Omega_{\text{ent}}\) over both branches, the optimization prevents policy collapse and excessive certainty drift.
Loss & Training¶
The complete PACER objective integrates clipped surrogate policy optimization, reference policy regularization, speech-evidence contrast, and predictive entropy constraints over frozen Qwen2.5-Omni-7B encoders:
Training uses a group sampling size \(G=8\), learning rate \(5 \times 10^{-7}\) with a linear schedule, and reference KL penalty coefficient \(\beta = 0.05\). The default reward weights are configured as condition weight \(\lambda_c = 0.35\), logic weight \(\lambda_l = 0.35\), and answer weight \(\lambda_a = 0.30\). The speech temporal masking ratio is set to \(0.5\) and the contrast coefficient \(\gamma = 0.001\). Optimization runs for 2 epochs across two GPUs using DeepSpeed ZeRO-3 and gradient checkpointing, ensuring stable training dynamics without speech encoder drift.
Key Experimental Results¶
Main Results¶
On the full SCUR-Test split (2,490 instances), Qwen2.5-Omni-7B was evaluated under diverse baseline and training configurations. Core evaluation metrics include final-answer error rate (Err.), condition-related error rate over all instances (CondErr/All), condition error share among failures (CondErr/Err), corrected-condition adoption rate (Adopt.), obsolete-condition usage rate (OldCond), and shallow mention-only rate (Mention-Only):
| Setting / Baseline | Err. ↓ (%) | CondErr/All ↓ (%) | CondErr/Err ↓ (%) | Adopt. ↑ (%) | OldCond ↓ (%) | Mention-Only ↓ (%) |
|---|---|---|---|---|---|---|
| Gold-text NoTrain (Speech-free Upper Bound) | 21.77 | 6.99 | 32.10 | 88.36 | 3.43 | 10.61 |
| NoTrain (Zero-Shot Base Model) | 27.91 | 14.46 | 51.80 | 83.46 | 9.74 | 16.90 |
| Whisper ASR + Qwen2.5-Omni (Cascade) | 32.65 | 18.63 | 57.07 | 81.89 | 13.93 | 19.26 |
| SFT-only (Supervised Fine-Tuning) | 33.37 | 7.31 | 21.90 | 92.90 | 0.51 | 12.06 |
| GRPO-Answer (Outcome-Reward Only) | 19.12 | 5.54 | 28.99 | 88.87 | 1.23 | 9.55 |
| GRPO-Structured (No Speech Contrast) | 18.27 | 5.22 | 28.57 | 89.73 | 0.68 | 8.45 |
| PACER (Full Proposed Framework) | 16.79 | 4.70 | 27.99 | 90.64 | 0.58 | 7.96 |
Ablation Study¶
To verify the distinct necessity of each reward component in PACER, systematic ablations were conducted while keeping all contrastive and architectural hyperparameters fixed:
| Ablation Configuration | Err. ↓ (%) | CondErr/All ↓ (%) | CondErr/Err ↓ (%) | Adopt. ↑ (%) | OldCond ↓ (%) | Mention-Only ↓ (%) |
|---|---|---|---|---|---|---|
| PACER (Full Model) | 16.79 | 4.70 | 27.99 | 90.64 | 0.58 | 7.96 |
| w/o Logic Reward | 17.23 | 5.18 | 30.07 | 90.01 | 0.85 | 8.50 |
| w/o Condition Reward | 17.11 | 5.46 | 31.92 | 89.50 | 0.95 | 7.93 |
| w/o Answer Reward | 17.27 | 5.70 | 33.02 | 89.53 | 0.53 | 9.15 |
Key Findings¶
- Supervised Fine-Tuning Induces Reasoning Atrophy: The SFT-only baseline achieved the highest corrected condition adoption (92.90%) and suppressed obsolete usage to 0.51%, yet its overall error rate surged to 33.37%. Fine-grained diagnosis reveals its post-adoption logic accuracy (Logic@Adopt) collapsed to 70.12% (compared to 87.15% for PACER), proving that standard cross-entropy SFT overfits to template memorization and severely degrades mathematical and logical generalization.
- Cascaded ASR Compounds Update Failures: The Whisper large-v3 + Qwen2.5-Omni pipeline suffered a 32.65% error rate—substantially worse than the end-to-end untrained model (27.91%). Condition-update failure among errors remained at an alarming 75.22%, demonstrating that cascading errors in recognizing rapid conversational repair phrases directly break downstream premise tracking.
- Robustness Under High Revision Complexity: In challenging settings such as multi-slot immediate correction (D2) and long-distance retrospective correction (D5), untrained baseline errors reached 31.44% and 43.13%, respectively. PACER effectively cut D2 error to 14.89% (a 52.6% relative error reduction), demonstrating that active condition-state reinforcement effectively resolves slot-binding confusions and retrospective memory overwrites.
Highlights & Insights¶
- Establishment of the Decoupled Diagnostic Paradigm: By systematically separating speech condition recovery (via transcription), condition-state adoption, and logical inference across three evaluation paths, this work pinpoints "final-condition adoption failure" as a distinct behavioral bottleneck, proving that high ASR fidelity does not imply effective premise utilization.
- Synergy Between Representation Perturbation and Structured RL: The framework demonstrates that random acoustic masking can serve as an effective sensitivity anchor when paired with structured reward shaping, allowing the language policy to stay attuned to continuous acoustic cues without requiring explicit alignment supervision.
- Generalizable Online Judge Architecture: The three-route JSON evaluation design (decoupling entity mention from operational usage and penalizing premise residue) offers a robust blueprint for diagnosing and training models in interactive tasks involving dynamic state updates and constraint revisions.
Limitations & Future Work¶
- Reliance on Synthesized Speech: SCUR-Bench audio is synthesized using Edge-TTS neural Mandarin voices. While linguistically diverse, it does not fully encompass spontaneous speech acoustic complexities such as overlapping speech, background acoustic clutter, prosodic stress shifts, and conversational interruptions.
- Scope Limited to Single-Turn Utterances: Although high-difficulty splits (D4/D5) simulate complex chained and retrospective revisions within long single turns, condition tracking across multi-turn conversational exchanges and full-duplex barge-in dialogues remains an open frontier.
- Broader Cross-Architecture Validation Needed: PACER was primarily trained and evaluated on Qwen2.5-Omni-7B. Investigating its generalizability across varied base architectures (e.g., LLaMA-Omni, MiniCPM-o) and exploring independent multilingual corpus collection represent essential next steps.
Related Work & Insights¶
- vs Disfl-QA & VocalBench-DF: Prior speech disfluency benchmarks treat repairs and restarts primarily as acoustic or syntactic noise for robust parsing or QA. This work establishes that self-correction is fundamentally a dynamic premise-update problem requiring active belief revision.
- vs Standard GRPO / RLVR (DeepSeekMath): Standard RLVR relies purely on verifiable final answers, which remains blind to whether a correct answer was derived from obsolete premises or lucky guessing. PACER introduces process-level structured condition supervision to enforce verifiable condition adoption.
- vs CPPO / Perception-Aware Policy Optimization: While vision-language perception-aware RL primarily tackles visual hallucination, PACER successfully translates this concept to continuous speech representations, demonstrating that representation-level contrastive sensitivity is crucial for tracking dynamic temporal evidence.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Formulates a fundamental yet overlooked condition-updating challenge in speech reasoning with an elegant three-path diagnostic framework and PACER optimization.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Exemplary 13k-sample benchmark construction, extensive multi-model evaluation, Chinese-English parallel validation, rigorous ablation, and human audit.
- Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally structured narrative, mathematically precise formulations, clear figures, and deep analytical insights.
- Value: ⭐⭐⭐⭐⭐ Delivers critical methodologies and empirical benchmarks for transitioning Speech LLMs into reliable, real-world full-duplex conversational reasoning agents.