Skip to content

Enhancing Long-Context Reasoning via Verifiable Atomic Skills

Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: Long-Context Reasoning, Atomic Skills, Verifiable Supervision, Reinforcement Learning, GRPO

TL;DR

This paper challenges the conventional monolithic treatment of long-context reasoning by decomposing it into a hierarchical spectrum of five verifiable atomic skills, proposing the Anchor-based Reasoning (AbR) framework to automatically synthesize strictly verifiable training data, and demonstrating that Reinforcement Learning (GRPO) on merely 4,000 samples improves DeepSeek-R1-distill-32B by an average of 7.7% across six leading long-context benchmarks.

Background & Motivation

Recent advancements in Large Language Models (LLMs) have substantially expanded effective context windows to hundreds of thousands or even millions of tokens, opening up unprecedented opportunities for analyzing multi-document repositories, synthesizing legal contracts, and processing extensive financial reports. However, despite these architectural scaling feats, a pronounced chasm persists between the raw scale of context models can ingest and the cognitive depth of reasoning they actually deliver over those extended sequences. Current mainstream methodologies attempt to bridge this divide by treating long-context reasoning as a monolithic, indivisible capability, curating massive task-specific long-document datasets and subsequently fine-tuning or running reinforcement learning on these end-to-end multi-hop queries.

Directly curating and supervising complex monolithic tasks is severely fraught with systemic failure modes. Real-world long-context queries frequently induce latent knowledge conflicts, subtle distractions, and multi-step reasoning dead-ends where hallucinated intermediate conclusions remain undetected. Crucially, this black-box complexity makes it extraordinarily difficult to provide strict, objective, and verifiable supervision for reinforcement learning. When relying on heuristic LLM-as-a-judge evaluators to assess free-form long-context outputs, models routinely suffer from reward hacking, subjective evaluation bias, and severe gradient vanishing caused by reward sparsity. Meanwhile, classical theories in cognitive science and discourse comprehension establish that human comprehension of complex documents is fundamentally a hierarchical, multi-stage process where individuals continually locate facts, filter noise, aggregate dispersed evidence, identify relational structures, and update dynamic mental states.

This paper initiates a fundamental paradigm shift: rather than optimizing long-context reasoning as a monolithic whole, it decomposes it into an orthogonal hierarchy of foundational atomic skills. By embedding algorithmically generated anchors into extensive noise contexts, every skill is mapped to deterministic, closed-form algorithmic answers, allowing strict exact-match verification to replace subjective evaluators. Core idea: Long-context reasoning is decomposed into a five-level cognitive spectrum comprising Foundational Retrieval, Anti-Interference, Global Integration, Relational Reasoning, and Dynamic State Tracking; the Anchor-based Reasoning (AbR) pipeline synthesizes controllable, strictly verifiable training data, enabling RL optimization on merely 4k samples to dramatically elevate general long-context reasoning capabilities.

Method

Overall Architecture

The AbR framework operationalizes the five atomic skills into scalable, controllable, and strictly verifiable training and evaluation instances. The data synthesis pipeline consists of three sequential stages: logical blueprint generation (defining the document-to-anchor mapping graph), question-answer pair generation with executable symbolic solver expressions, and multi-document context synthesis (interleaving anchors and queries into authentic, unrelated noise texts). The policy optimization phase employs Group Relative Policy Optimization (GRPO) augmented with dynamic sampling and pass-rate difficulty filtering, leveraging gpt-oss-120B strictly as a deterministic equivalence checker to award binary exact-match rewards.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Atomic Skill Taxonomy & Design Constraints<br/>Pipeline Coverage · Verifiable Supervision"] --> B["Logical Blueprint Generation<br/>Programmatic doc_mapping JSON Topology"]
    B --> C["QA Pair & Symbolic Solver Synthesis<br/>Deterministic Closed-Form Answer Expressions"]
    C --> D["Multi-Document Context Synthesis<br/>Unrelated Background Texts with Embedded Anchors"]
    D --> E["Pass-Rate Difficulty Filtering<br/>DeepSeek-V3.1 Rollouts Retained in 0.3-0.6 Window"]
    E --> F["GRPO Policy Optimization with Dynamic Sampling<br/>Critic-Free Group Relative Advantage Estimation"]
    F --> G["Deterministic Exact-Match Reward Mechanism<br/>gpt-oss-120B Equivalence Checker · Binary Rewards"]
    G --> H["Enhanced Long-Context Reasoning Model<br/>Breaking Retrieval Ceiling & Robustness Bottleneck"]

Key Designs

1. Atomic Skill Taxonomy & Dual Design Constraints: Structuring the Cognitive Spectrum To overcome the opacity of monolithic tasks, the authors formalize an atomic taxonomy governed by two strict design constraints: (C1) Pipeline Coverage, requiring that the skills collectively span the complete cognitive pipeline executed during complex long-context reasoning (\(\text{retrieve} \to \text{filter noise} \to \text{integrate} \to \text{deduce relations} \to \text{compute}\)); and (C2) Verifiable Supervision, demanding that every skill admits a purely algorithmic, strictly verifiable ground truth to enable exact-match rewards. The five skills are ordered by ascending cognitive complexity: - Foundational Retrieval (NIAH): The bedrock prerequisite testing whether a model can locate isolated facts (needles) arbitrarily positioned within extensive background text (haystacks) to overcome the "lost-in-the-middle" phenomenon. - Anti-Interference (Anti-interfere): Evaluates robustness to realistic distractors via two explicit patterns: Similarity Discrimination (distinguishing target anchors from syntactically or semantically similar decoy anchors) and Conflicting Information Resolution (resolving identical anchor keys distributed across conflicting document scopes). - Global Integration (Multi-source): Requires the model to break free from localized attention traps, scanning across disparate document boundaries to aggregate fragmented evidence into a unified logical premise. - Relational Reasoning (Logic): Enforces relational and structural understanding of the context, compelling the model to execute set operations (intersections, unions, relative positions, and frequency distributions) across virtual document containers, akin to running relational database queries over unstructured text. - Dynamic State Tracking (Calc-reason): The apex of the hierarchy, demanding multi-step computational reasoning where the model evaluates conditional contextual states, retrieves distributed values, holds dynamic variables in working memory, and executes sequential mathematical transformations to produce the final answer.

2. Three-Stage Automated AbR Dataset Construction Pipeline: Algorithmic Ground-Truth Guarantee To eliminate human annotation hallucinations and subjective LLM judge biases, the AbR pipeline synthesizes data from an algorithmic blueprint. - Stage 1 (Logical Blueprint Generation): Programmatically constructs a doc_mapping JSON file specifying which unique anchor strings \(\text{anc\_id}\) are assigned to each virtual document \(\text{doc\_id}\), establishing a known ground-truth topological distribution (e.g., \(\text{doc1}: [\text{anc4}, \text{anc5}, \text{anc3}]\)). - Stage 2 (Question-Answer Pair Generation): Generates meta-questions demanding relational or computational reasoning over the blueprint, pairing them with executable expressions representing procedural solutions \(\text{Solve}(Q(\text{anc}))\). For example, a relational query asks: "In the document containing both anchor5 and anchor3, what is the answer to the question for anchor4?", formalizing the target as \(\text{Solve}(Q(\text{anc}_4, d \in (D(\text{anc}_5) \cap D(\text{anc}_3))))\); a computational query sums answers across document occurrences: \(\sum_{i \in \{1, 3\}} \text{Solve}(Q(\text{doc}_i, \text{anc}_3))\). - Stage 3 (Multi-Document Context Synthesis): Gathers diverse, unrelated background corpora (e.g., financial reports, medical Q&A, historical essays, technical manuals) to serve as document containers, inserts the anchor-question pairs at randomized depths according to the blueprint, and outputs the final prompt-context tuple.

3. Pass-Rate Difficulty Filtering and Emergent Skill Ratios: Navigating the Zone of Proximal Development Randomly synthesized samples often introduce severe training instabilities: queries that are too simple produce uniform successes and vanishing policy gradients, while overly difficult queries induce reward sparsity. The authors implement a pass-rate filtering protocol using DeepSeek-V3.1 as an external reference model. For every candidate query in an initial vast pool, 16 independent rollouts are sampled to compute the pass rate: $\(\text{Pass-Rate} = \frac{\text{Correct Exact-Match Rollouts}}{16}\)$ Queries with pass rates \(>0.6\) are discarded as trivially easy, while those \(<0.3\) are removed as destabilizingly hard. Retaining only queries within \([0.3, 0.6]\) naturally induces an emergent difficulty-aware data mixture (Anti-interfere : Multi-hop : Multi-source : Logic : Calc-reason : NIAH = \(5 : 3 : 2 : 2 : 2 : 1\)). This curriculum-aligned filtering produces a compact, information-dense training set of merely 4,000 samples.

4. GRPO Policy Optimization with Dynamic Sampling and Exact-Match Verification Policy optimization utilizes Group Relative Policy Optimization (GRPO), eliminating the computational and memory overhead of a separate critic network for long contexts. For each query, the policy generates a group of \(G=16\) outputs \(\{o_1, \dots, o_G\}\). Given binary rewards \(r_i \in \{0, 1\}\), the advantage \(A_i\) is computed relative to the group mean: $\(A_i = r_i - \frac{1}{|G|} \sum_{j=1}^{|G|} r_j\)$ The clipped surrogate objective incorporates a Kullback-Leibler (KL) divergence penalty against the reference model \(\pi_{\text{old}}\): $\(\mathcal{L}_{\text{GRPO}}(\theta) = \mathbb{E}\left[ \min\left(\frac{\pi_\theta(a|s)}{\pi_{\text{old}}(a|s)} A_i, \text{clip}\left(\frac{\pi_\theta(a|s)}{\pi_{\text{old}}(a|s)}, 1-\epsilon, 1+\epsilon\right) A_i \right) - \beta D_{\text{KL}}(\pi_\theta \parallel \pi_{\text{old}}) \right]\)$ To prevent reward homogenization—where identical rewards across all rollouts collapse advantages to zero—Dynamic Sampling prunes non-informative trajectories. The reward function employs gpt-oss-120B purely as a deterministic answer extractor and normalization checker (checking numerical equivalence up to two decimal places and exact text matching). Crucially, gpt-oss-120B scores only 35.49% average accuracy on long-context benchmarks, confirming that RL improvements originate entirely from the AbR data rather than judge capabilities.

Loss & Training

  • Hardware & Parallelism: Trained on 64 \(\times\) NVIDIA H20 GPUs with Tensor Parallelism set to \(\text{TP}=8\) without CPU offloading.
  • Sequence Budget: Maximum sequence capacity of 32,768 tokens (allocated as 28,576 tokens for input context and 8,092 tokens for output generation).
  • Hyperparameters: Global batch size 128 (mini-batch 32), rollout generation batch size 256 (\(G=16\), temperature 0.85), learning rate \(2 \times 10^{-6}\), KL coefficient \(\beta = 0.001\), clipping parameters \(\epsilon_{\text{high}}=0.28, \epsilon_{\text{low}}=0.20\).
  • Cold-Start Adaptation: Base models (DeepSeek-R1-Distill-32B, Qwen2.5-14B/32B-Instruct) are trained directly via RL without prior long-context supervised fine-tuning. Standard instruct models receive a Chain-of-Thought (CoT) system prompt coupled with format-compliance rewards, whereas self-reasoning models run under default prompting.

Key Experimental Results

Main Results

Evaluation spans six challenging long-context benchmarks up to 128k context lengths (extrapolated using YaRN where required): Loogle, LongBench-v2, Loong, BrowsCompLong, Ruler-qa2, and MRCR.

Model Loogle LongBench-v2 Loong BrowsCompLong Ruler-qa2 MRCR Average
Gemini-3-pro (Closed SOTA) 52.86% 69.38% 65.43% 88.07% 83.01% 75.30% 72.34%
Kimi-K2-Thinking 51.50% 49.30% 58.01% 58.10% 49.98% 51.77% 53.11%
DeepSeek-V3.1 55.77% 52.88% 50.55% 56.27% 42.97% 46.62% 50.84%
Qwen3-235B-A22B-thinking 52.77% 49.30% 53.70% 50.76% 46.81% 44.61% 49.66%
QwenLong-L1-32B 49.32% 43.74% 44.68% 69.93% 47.70% 27.70% 47.18%
GPT-OSS-120B (Verifier) 40.15% 15.31% 22.83% 71.87% 39.92% 22.83% 35.49%
DeepSeek-R1-distill-32B (Base) 42.31% 43.94% 38.17% 64.22% 57.23% 31.94% 46.30%
+ LongReason (Existing SFT) 43.23% 44.14% 38.57% 57.90% 58.36% 32.85% 45.84%
+ QwenDocqa 47.96% 46.32% 39.98% 71.56% 67.45% 34.59% 51.31%
+ LoongRL (4k sample RL) 48.41% 45.73% 40.29% 70.74% 61.84% 37.23% 50.71%
+ Ours (4k AbR RL) 50.59% 49.70% 44.68% 73.09% 69.38% 36.74% 54.03%
+ Ours + LoongRL (Synergy) 55.59% 51.29% 44.45% 72.27% 67.01% 36.12% 54.46%

Across different model families and parameter scales (Table 2): - Qwen2.5-14B-Instruct: Improves from 35.59% to 45.83% (+10.24% absolute gain). - Qwen2.5-32B-Instruct: Improves from 41.54% to 45.50% (+3.96% absolute gain).

Ablation Study & Skill Inter-Dependencies

To systematically determine whether all atomic skills are necessary and how they interact, component ablation experiments were conducted on DeepSeek-R1-distill-32B (Figures 6 & 7).

Training Configuration Benchmark Impact Cognitive Role & Structural Diagnosis
Full AbR Mixture Highest across all benchmarks (+13.3% on Loogle, +9.8% on Ruler-qa2) Complete envelope over all ablated variants; establishes skill synergy
w/o Multi-source (Global Integration) Drops below Base model on Loong and LongBench-v2 Foundational Stabilizer: removal triggers single-doc over-specialization and skill imbalance
w/o Logic (Relational Reasoning) Severe collapse on BrowsCompLong and Ruler-qa2 Structural Backbone: long document browsing and constrained QA depend strictly on logical graphs
w/o Calc-reason (State Tracking) Broad performance drops across general benchmarks Universal Precision Enhancer: instills rigorous computational verification across all tasks
w/o Anti-interfere (Robustness) Probe score drops by -21.1% Confirms discrimination against noise lures cannot be acquired via standard retrieval

Key Findings

  • Superiority Over Monolithic Datasets: On the identical DeepSeek-R1-distill-32B backbone, training on 4,000 AbR samples achieves 54.03%, outperforming existing long-context datasets like LongReason (45.84%), QwenDocqa (51.31%), and LoongRL (50.71%), while surpassing massive open models such as Qwen3-235B (49.66%) and Kimi-K2-Thinking (53.11%).
  • Addressing the Importance-Proficiency Mismatch: Diagnostic correlation analyses reveal that Anti-Interference (\(\rho = 0.94\)) and Global Integration (\(\rho = 0.91\)) exhibit the highest correlation with real-world performance, yet base models score poorly on them (22%–26%). AbR pushes models past the clean retrieval ceiling (NIAH), boosting Logic from ~18% to 68.8% and Calc-reason from 42% to 80.7%.
  • Hierarchical Skill Coupling: Cross-probe ablation reveals an asymmetric cognitive dependency: removing Logic degrades Calc-reason probe accuracy by -12.3%, whereas removing Calc-reason impacts Logic by only -6.0%, proving that dynamic computational manipulation is hierarchically grounded in logical structuring. Furthermore, omitting Multi-source degrades all atomic capabilities (~ -10.6%), highlighting its stabilizing function.
  • Length Invariance & Short-Context Positive Transfer: On LongBench-v2 across context buckets (8k–128k), AbR shifts the accuracy curve upward uniformly, with a pronounced jump in the 32k–64k bucket (from 48.48% to 59.09%). Crucially, evaluation on standard short-context multi-hop benchmarks (2WikiMultihopQA, MuSiQue, HotpotQA) shows an 8.9% absolute gain (from 67.8% to 76.7%), demonstrating zero catastrophic forgetting.

Highlights & Insights

  • Verifiable Decomposition as an RL Antidote: By translating fuzzy, ill-posed multi-document reasoning into modular atomic skills equipped with algorithmic ground-truth formulas, the framework eliminates reliance on subjective LLM evaluators, establishing a leak-proof verification loop for reinforcement learning.
  • Extraordinary Data Efficiency: Demonstrates that advancing long-context reasoning does not mandate hundreds of thousands of synthetically generated long texts. Focusing on a precisely filtered, difficulty-aware set of 4,000 atomic samples delivers SOTA-tier reasoning gains.
  • Extensible Cognitive Blueprint: The five skills act as an open scaffold rather than a closed sandbox. The methodology seamlessly extends to new atomic abilities, such as cross-tabular reasoning or temporal ordering, through the identical anchor-embedding paradigm.

Limitations & Future Work

  • Constraint to Exact-Match Determinism: Current RL rewards necessitate closed-form objective answers (numbers, symbolic sets), intentionally focusing on extraction, logic, and computation; extending this to generative open-ended tasks (e.g., repository-level software engineering or abstractive summarization) requires developing robust long-form faithfulness verifiers.
  • Static Difficulty Windows: The data filtering thresholds (\([0.3, 0.6]\) pass rates) and mixture ratios are empirically derived; transitioning to dynamic, adaptive curriculum learning during RL rollouts remains an active future direction.
  • Taxonomy Breadth: The current five-skill taxonomy does not exhaustively address fine-grained temporal causality or multi-modal cross-document planning, leaving room for continuous horizontal expansion.
  • vs. Monolithic Synthetic Pre-training/SFT (LongReason, QwenDocQA, LongFaith): Previous methods generate holistic multi-hop question-answer pairs over entire documents, suffering from hallucinated rationales and ambiguous labels. AbR ensures zero-hallucination ground truth via programmatic anchors, providing pristine gradient signals.
  • vs. Unstructured Long-Context RL (LoongRL, QwenLong-L1): Traditional RL approaches optimize on general multi-doc QA datasets, frequently hitting the retrieval saturation ceiling. AbR targets the critical robustness and multi-source bottlenecks directly, demonstrating strong complementary stacking gains when combined with LoongRL (+0.43% additional boost to 54.46%).

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Deconstructs monolithic long-context reasoning into an algorithmically verifiable five-level cognitive spectrum.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Evaluated across six real-world long-context benchmarks, three short-context multi-hop datasets, 11 model diagnostic probes, and exhaustive cross-skill ablations.
  • Writing Quality: ⭐⭐⭐⭐⭐ Clear conceptual motivation, rigorous mathematical formulation, and compelling empirical validation.
  • Value: ⭐⭐⭐⭐⭐ Provides a data-efficient, scalable, and verifiable paradigm for post-training long-context reasoning models.