Skip to content

Adaptive Knowledge Routing for Complex Reasoning in LLMs

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/preordinary/ASRD
Area: LLM Reasoning
Keywords: diffusion language models, revocable parallel decoding, anchor tokens cache, error propagation, orthogonal perturbation

TL;DR

Addressing error propagation and local error reinforcement in revocable parallel decoding for masked diffusion LLMs, this paper proposes ASRD, a training-free embedding-space framework that leverages a temporal Anchor Tokens Cache to steer mask generation via entropy-weighted convex blending and verify tentative candidates via orthogonal residual probing, achieving up to a 6.4% accuracy gain and a 7.2x wall-clock speedup.

Background & Motivation

Diffusion Large Language Models (dLLMs, such as LLaDA and Dream) decode discrete text sequences by iteratively unmasking tokens from a fully masked canvas, bypassing the strictly left-to-right factorization inherent in autoregressive (AR) models and offering substantial theoretical potential for parallel generation. However, conventional threshold-based parallel unmasking strategies suffer from an acute trade-off between decoding throughput and final output quality: setting an aggressive, lower confidence threshold commits more tokens per forward pass to reduce overall denoising steps, but inevitably admits low-confidence predictions whose erroneous representations propagate uncontrollably into subsequent iterations. To address this deficiency, recent revocable parallel decoding paradigms (e.g., WINO, Saber) incorporate draft-and-verify mechanisms that inspect committed tokens within the evolving sequence context and remask unreliable positions back to [MASK].

Crucially, in order to preserve parallel efficiency, existing revocable decoding algorithms execute verification and generation within a shared forward pass, forcing the model to operate over a mixed-quality context that uncontrollably interleaves verified facts with tentative, low-confidence drafts. This mixed-quality context precipitates two fatal structural failures: first, Error Propagation, wherein freshly generated [MASK] tokens attend unconditionally to unverified pending tokens across bidirectional self-attention layers, indiscriminately absorbing contaminated representations and broadcasting errors across the entire decoding trajectory; second, Local Error Reinforcement, wherein multiple erroneous pending tokens situated in close proximity mutually corroborate each other during the verification phase, forming a locally self-consistent yet globally invalid cluster that easily bypasses naive confidence thresholds and Top-1 stability checks.

Overcoming these dual failure modes without sacrificing FlashAttention compatibility requires an inference framework that can explicitly separate the trustworthy sequence backbone from fragile drafts and directly steer unconfirmed positions. The authors tackle this challenge by restricting all interventions strictly to the input word embedding space, bypassing expensive attention-logit reweighting or architectural alterations. Core idea: explicitly decouple the decoding context into temporally consistent anchor tokens and uncertain candidates via an Anchor Tokens Cache (ATC), then intervene strictly within the input embedding space by injecting entropy-weighted anchor priors into masked positions to block error propagation and applying orthogonal residual probes to tentative candidates to shatter local consensus reinforcement.

Method

Overall Architecture

ASRD (Anchor Supervised Revocable Decoding) is a completely training-free, plug-and-play decoding paradigm designed for masked diffusion language models. At each decoding iteration, instead of treating all unmasked tokens uniformly, ASRD segregates threshold-passed tokens into a trusted set of Anchor Tokens and a tentative set of Pending Candidates based on multi-step temporal consistency, actively maintaining an Anchor Tokens Cache (ATC) governed by a FIFO eviction policy.

Prior to each diffusion forward pass, ASRD computes the anchor centroid across the active ATC embeddings and executes two complementary embedding-space updates: (1) Anchor-Guided Generation replaces the generic [MASK] token embeddings with an entropy-modulated convex blend toward the anchor centroid, injecting reliable global guidance and implicitly realigning attention away from noisy drafts; (2) Anchor-Perturbed Verification computes an orthogonal residual probe from the anchor centroid relative to each pending token's committed embedding and injects this stress-testing perturbation into the candidate representation. After a single forward pass, any candidate that fails the Top-1 stability check is remasked, newly qualified tokens are promoted to anchors or pending drafts, and the ATC is refreshed.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: Mixed-Quality Context<br/>Composed of Anchors, Pending Drafts, and Masks"] --> B["Anchor Tokens Cache (ATC)<br/>Temporal Consistency Check & FIFO Queue"]
    B --> C["Anchor-Guided Generation<br/>Entropy-Weighted Convex Blend into Mask Embeddings"]
    B --> D["Anchor-Perturbed Verification<br/>Orthogonal Residual Probing of Pending Tokens"]
    C --> E["Single Forward Pass<br/>Native Execution Compatible with FlashAttention"]
    D --> E
    E --> F["State Update & Revocation<br/>Top-1 Stability Flip Remasking & ATC Update"]

Key Designs

1. Anchor Tokens Cache (ATC): Exponentially suppressing false anchors via temporal consistency

Addressing the reality that single-step high-confidence tokens often suffer from transient probability spikes that fluctuate across subsequent steps, ASRD refuses to grant supervisory status to any freshly unmasked token immediately. Instead, a position \(i\) is promoted to the anchor candidate set \(A_t\) if and only if its Top-1 prediction remains strictly invariant across \(k\) consecutive decoding steps: \(\hat{a}_{t-k+1}^i = \dots = \hat{a}_t^i\). Under idealized conditions where the intrinsic denoising distribution exhibits a positive semantic margin \(\Delta_{\text{sem}} > 0\) over incorrect competitors and the differential noise is symmetric and i.i.d., the probability that an incorrect token survives as the Top-1 prediction for \(k\) consecutive steps decays exponentially:

\[P(\hat{a}_{t-k+1}^i = \dots = \hat{a}_t^i = w) \le \exp(-\lambda k)\]

where \(\lambda = \ln(1/q) > 0\). To track the evolving sequence backbone without accumulating stale history, newly identified anchors are appended to a fixed-capacity FIFO queue \(I_t = \text{FIFO}_m(I_{t-1} \cup A_t)\). This eviction rule perfectly aligns with the progressive left-to-right unmasking frontier of diffusion models. During the initial \(k-1\) steps where history is insufficient, the cache is seeded with the single lowest-entropy unmasked token at each step, preventing cold-start failure while guaranteeing an exceptionally pure semantic backbone.

2. Anchor-Guided Generation: Entropy-weighted convex blend for implicit attention rectification

Addressing the structural vulnerability wherein an uninformative generic emask embedding freely attends to and absorbs errors from neighboring pending drafts, ASRD provides an anchor-derived semantic prior directly at the input embedding layer. At step \(t\), the system aggregates the input embeddings of all active positions in the cache into an anchor centroid \(\bar{c}_t = \frac{1}{|I_t|} \sum_{j \in I_t} e_j^t\). For every masked position \(i \in M_t\), its input embedding is updated via a convex blend toward \(\bar{c}_t\):

\[e_i^t = (1 - \gamma_i^t) e_{\text{mask}} + \gamma_i^t \bar{c}_t, \quad \text{where} \quad \gamma_i^t = \alpha \bar{E}_i^t\]

Here, \(\alpha \in [0, 1]\) modulates the global guidance magnitude, while \(\bar{E}_i^t \in [0, 1]\) represents the min-max normalized prediction entropy evaluated from the previous step's logits. High-entropy positions with high uncertainty receive strong anchor supervision, whereas low-entropy, confident masks remain largely untouched. Mechanistically, this input shift translates into an induced Query shift \(q'_i = (1 - \gamma_i^t) q_i + \gamma_i^t q_{I_t}\) that biases the query representation toward the subspace spanned by anchor keys \(\langle k_u, q'_{i} \rangle\), thereby reallocating attention probability away from noisy pending candidates and toward verified anchor tokens without explicitly modifying attention matrices or breaking FlashAttention.

3. Anchor-Perturbed Verification: Orthogonal residual probing to break local error reinforcement

Addressing the failure mode where erroneous pending tokens mutually corroborate each other to form a locally self-consistent consensus cluster that deceives conventional verification, ASRD deploys a counterfactual stress-test probe. Direct overwriting would be destructive to valid drafts, while parallel perturbation along the committed direction would artificially reinforce spurious predictions. ASRD resolves this by isolating the component of the anchor centroid \(\bar{c}_t\) that is strictly orthogonal to the committed token embedding \(e_i^t\):

\[d_i^t = \bar{c}_t - \frac{\langle \bar{c}_t, e_i^t \rangle}{\|e_i^t\|_2^2} e_i^t, \quad e_i^t \leftarrow e_i^t + \beta d_i^t\]

Because \(\langle d_i^t, e_i^t \rangle = 0\), this probe measures the candidate's local margin of robustness against an anchor-aligned perturbation. Legitimate tokens aligned with the global context yield minimal residual norms and possess ample logit margins that resist the perturbation. Conversely, tokens sustained purely by local error reinforcement possess fragile margins; under the orthogonal push, their post-perturbation Top-1 prediction flips (\(\arg\max_v p_i^s(v) \ne \breve{x}_i\)), immediately failing the stability check and triggering remasking back to [MASK].

Loss & Training

ASRD is entirely training-free and operates purely at inference time, leaving the host diffusion model's weights unmodified and fully preserving FlashAttention exact kernel execution. Across all experiments, hyperparameters remain remarkably robust: consistency window \(k=2\); cache capacity \(m \in \{12, 16, 20\}\); guidance strength \(\alpha \in [0.05, 0.2]\); perturbation strength \(\beta \in [0.1, 0.4]\); and base drafting threshold \(\tau \in \{0.6, 0.7, 0.8\}\). All main evaluations are executed deterministically at temperature 0.

Key Experimental Results

Main Results

Evaluated on instruction-tuned diffusion backbones (LLaDA-Ins-8B and Dream-Ins-7B) across code generation (HumanEval, MBPP) and mathematical reasoning (GSM8K, MATH500), ASRD consistently outperforms the standard diffusion decoder as well as state-of-the-art revocable baselines WINO and Saber.

Backbone Length Benchmark Baseline (Acc / Speed) WINO (Acc / Speed) Saber (Acc / Speed) ASRD (Ours) (Acc / Speed) Improvement over Baseline
LLaDA-Ins-8B 256 HumanEval 38.7% / 1.0× 37.5% / 2.3× 39.4% / 2.4× 41.5% / 3.1× +2.8% (Steps: 256.0 → 75.4)
LLaDA-Ins-8B 256 MBPP 36.8% / 1.0× 36.2% / 1.9× 37.8% / 2.1× 38.6% / 2.5× +1.8% (Steps: 256.0 → 88.3)
LLaDA-Ins-8B 256 GSM8K 77.4% / 1.0× 77.3% / 2.9× 77.8% / 1.9× 79.5% / 3.2× +2.1% (Steps: 256.0 → 56.9)
LLaDA-Ins-8B 256 MATH500 33.8% / 1.0× 34.2% / 2.0× 34.8% / 1.8× 35.2% / 2.9× +1.4% (Steps: 256.0 → 74.8)
LLaDA-Ins-8B 512 HumanEval 43.9% / 1.0× 43.9% / 2.8× 44.5% / 2.3× 48.8% / 3.6× +4.9% (Steps: 512.0 → 127.8)
LLaDA-Ins-8B 512 MBPP 38.2% / 1.0× 37.4% / 2.4× 37.8% / 2.6× 39.4% / 2.9× +1.2% (Steps: 512.0 → 110.4)
LLaDA-Ins-8B 512 GSM8K 80.3% / 1.0× 78.7% / 3.6× 79.7% / 3.2× 81.1% / 4.3× +0.8% (Steps: 512.0 → 88.5)
LLaDA-Ins-8B 512 MATH500 37.8% / 1.0× 35.6% / 2.6× 37.4% / 2.5× 38.8% / 3.8× +1.0% (Steps: 512.0 → 119.3)
Dream-Ins-7B 256 HumanEval 54.9% / 1.0× 51.3% / 2.4× 52.5% / 1.5× 56.1% / 3.1× +1.2% (Steps: 256.0 → 82.6)
Dream-Ins-7B 256 MATH500 37.8% / 1.0× 38.4% / 1.6× 38.8% / 1.4× 41.6% / 3.5× +3.8% (Steps: 256.0 → 78.1)
Dream-Ins-7B 512 MBPP 56.2% / 1.0× 55.4% / 5.2× 55.8% / 1.3× 57.2% / 7.2× +1.0% (Steps: 512.0 → 74.5)
Dream-Ins-7B 512 MATH500 38.6% / 1.0× 39.8% / 2.8× 41.4% / 2.0× 45.0% / 5.4× +6.4% (Steps: 512.0 → 94.8)

Ablation Study

Module-level ablations on LLaDA-Ins-8B disentangle the contributions of Anchor-Guided Generation and Anchor-Perturbed Verification, while design-level ablations scrutinize the mathematical form of the perturbation and guidance.

Length Variant / Configuration HumanEval (Acc / Speed) MBPP (Acc / Speed) GSM8K (Acc / Speed) MATH500 (Acc / Speed) Note / Analysis
256 ASRD (Full) 41.5% / 3.1× 38.6% / 2.5× 79.5% / 3.2× 35.2% / 2.9× Complete dual-intervention pipeline
256 w/o Anchor-Guided Gen (\(\gamma_i^t=0\)) 39.0% (-2.5) / 2.8× 37.6% (-1.0) / 2.3× 78.4% (-1.1) / 3.0× 34.0% (-1.2) / 2.7× Mask absorbs errors from unverified context
256 w/o Anchor-Perturbed Ver (\(\beta=0\)) 40.3% (-1.2) / 3.5× 38.0% (-0.6) / 2.9× 79.1% (-0.4) / 3.5× 33.6% (-1.6) / 3.2× Inability to destabilize local error clusters
512 ASRD (Full) 48.8% / 3.6× 39.4% / 2.9× 81.1% / 4.3× 38.8% / 3.8× Advantage widens over longer sequences
512 w/o Anchor-Guided Gen (\(\gamma_i^t=0\)) 47.5% (-1.3) / 3.3× 38.4% (-1.0) / 2.7× 80.8% (-0.3) / 4.1× 38.6% (-0.2) / 3.6× Requires 148.2 steps vs 127.8 steps for Full
512 w/o Anchor-Perturbed Ver (\(\beta=0\)) 46.9% (-1.9) / 4.0× 38.6% (-0.8) / 3.2× 80.4% (-0.7) / 4.6× 37.0% (-1.8) / 4.1× Faster but hurts precision on rigorous tasks
512 Design: Constant \(\alpha\) magnitude 46.4% (-2.4) / 3.4× - 80.3% (-0.8) / 4.3× - Validates importance of entropy modulation
512 Design: Parallel probe along \(e_i^t\) 42.6% (-6.2) / 3.2× - 75.9% (-5.2) / 3.9× - Artificially reinforces committed false tokens
512 Design: Gaussian isotropic noise 41.4% (-7.4) / 3.0× - 75.1% (-6.0) / 3.8× - Corrupts valid tokens without anchor semantics

Key Findings

  • Targeted resolution of distinct failure modes: Removing Anchor-Guided Generation inflicts the steepest penalty on code tasks (HumanEval drops by 2.5%), as unguided masks readily absorb syntax and naming hallucinations from pending neighbors. Conversely, disabling Anchor-Perturbed Verification predominantly degrades rigorous mathematical reasoning (MATH500 drops by 1.6% to 1.8%), indicating that breaking local consensus reinforcement is paramount for multi-step arithmetic.
  • Amplified performance at scale and horizon: As sequence length doubles from 256 to 512, the accuracy margin of ASRD over standard decoding increases substantially (e.g., from +3.8% to +6.4% on Dream-Ins-7B for MATH500), underscoring that the ATC provides an increasingly crucial global roadmap over extended generation contexts.
  • Orthogonality is the load-bearing design choice: Probing along the parallel direction of the candidate token embedding results in a severe 6.2% accuracy drop on HumanEval, while isotropic Gaussian noise degrades accuracy by 7.4%. Only an orthogonal residual probe can reliably destabilize fragile clusters without destroying valid semantic representations.
  • Drastic reduction of long error spans: Independent token-level error diagnostics using Claude-Opus-4.6 reveal that ASRD reduces the long-span error ratio \(R\) (erroneous tokens belonging to contiguous runs \(\ge 15\)) from 59.73% to 16.28% on HumanEval and from 48.35% to 11.83% on GSM8K, successfully shattering stubborn error clusters into easily correctable isolated tokens.
  • High throughput via step count amortization: Profiling on an NVIDIA A100 GPU reveals that ASRD adds only 7.6 ms per step (a modest 9.8% overhead over the baseline's 77.3 ms). Because anchor guidance produces higher-quality drafts that dramatically slash average decoding steps from 256 to 75.4, end-to-end wall-clock speedup reaches 3.1x to 7.2x.

Highlights & Insights

  • Embedding-level intervention preserves FlashAttention: By executing context rectification and candidate probing strictly in the input token embedding space, ASRD completely circumvents intrusive modifications to multi-head attention weights, ensuring zero runtime compilation friction and 100% native FlashAttention compatibility.
  • Theoretical underpinning of temporal consistency: The paper provides a formal proof showing that under symmetric differential noise, the probability of an incorrect candidate surviving across a \(k\)-step temporal window decays exponentially as \(\exp(-\lambda k)\), providing a rigorous mathematical rationale for anchor selection.
  • Non-destructive counterfactual stress-testing: Instead of attempting to magically regenerate correct tokens in a single shot, the orthogonal probe acts as an elegant one-sided fragility detector, purposefully probing whether a tentative prediction possesses genuine semantic stability or merely rests on spurious local consensus.

Limitations & Future Work

  • Exploration capped at 7B–8B model scale: The empirical validation focuses primarily on 7B and 8B parameter diffusion LLMs; validating whether identical anchor dynamics and consistency thresholds scale seamlessly to 70B+ or 100B parameter dLLMs remains an open question.
  • Brittleness under excessively large Semi-AR block sizes: Accuracy exhibits sharp drops when the Semi-AR block size is expanded to 128 (e.g., GSM8K drops to 72.4%), as anchor tokens become too sparsely scattered across the canvas to reliably supervise active generation frontiers.
  • Future work on dynamic probe strength: Currently, guidance and probe coefficients \(\alpha\) and \(\beta\) are static scalars; future research could explore dynamically adapting these parameters based on layer-wise attention entropy or contextual uncertainty estimates.
  • vs WINO (Hong et al., 2025): WINO employs dual confidence thresholds for draft-and-verify decoding, but both phases operate under an unfiltered mixed-quality context, leaving it vulnerable to local error reinforcement; ASRD introduces explicit anchor decoupling and orthogonal probing, reducing long contiguous error spans from 50.66% to 16.28%.
  • vs Saber (Dong et al., 2025): Saber pairs dynamic acceleration with backtracking remasking based on scalar confidence heuristics; ASRD directly addresses attention misallocation at the embedding level via entropy-modulated convex blending, producing superior drafts that cut overall decoding steps while attaining higher speedups.
  • vs Fast-dLLM (Wu et al., 2025): Fast-dLLM relies exclusively on aggressive parallel thresholding, which severely degrades output accuracy when threshold \(\tau\) is lowered; ASRD proves that anchor supervision is essential to maintain high generation fidelity under aggressive parallel speedups.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Decoupling mixed-quality contexts via temporal consistency and intervening strictly in the embedding space via orthogonal probing represents an exceptionally creative and elegant breakthrough for diffusion decoding.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Thoroughly covers four coding and math benchmarks, base, instruct, and RL-tuned models, block diffusion variants, open-ended generation, attention visualizations, and token-level error diagnostics.
  • Writing Quality: ⭐⭐⭐⭐⭐ Lucid narrative structure, mathematically rigorous formulations, and insightful ablation studies that systematically isolate every design component.
  • Value: ⭐⭐⭐⭐⭐ Plug-and-play, training-free, and fully compatible with FlashAttention, delivering substantial accuracy gains alongside up to 7.2x wall-clock speedups for practical diffusion LLM deployment.