Skip to content

GRACE: Step-Level Benchmark for Faithful Reasoning over Context

Conference: EMNLP 2026
Paper: https://aclanthology.org/2026.emnlp-main.1582/
Code: https://github.com/pvhoang14/GRACE-benchmark
Area: LLM Reasoning
Keywords: Chain-of-Thought Faithfulness, Context Faithfulness, Step-Level Benchmark, Error Taxonomy, Process Reward Model

TL;DR

This paper proposes GRACE, the first step-level benchmark for context-grounded textual reasoning, featuring a data-driven two-track eight-category error taxonomy and a rigorous human-adjudicated evaluation set. The benchmark reveals a critical disconnect where 49.5% of traces with unfaithful intermediate steps still produce correct final answers, and shows that incorporating step-level context faithfulness into reinforcement learning as process rewards substantially boosts both reasoning reliability and task accuracy.

Background & Motivation

Chain-of-Thought (CoT) prompting has become a cornerstone technique for knowledge-intensive multi-hop question answering and rule-grounded deductive reasoning. When conditioned on explicit reference documents and contextual constraints, large language models produce step-by-step rationales that seem to render their inference transparent and verifiable. However, this apparent interpretability can be profoundly deceptive: an individual step can silently deviate from the source text, drawing invalid conclusions from valid evidence or fabricating unsupported premises to stitch together a coherent narrative. Crucially, because final answer selection or answer matching tolerates faulty derivation, models frequently arrive at correct final answers through invalid reasoning steps, meaning that standard outcome-based evaluations systematically overestimate true reasoning reliability.

Existing evaluation paradigms for hallucination and attribution operate primarily at the coarse-grained response level or isolated claim span level. While such methods can flag that a generation contains an unsupported claim, they treat the reasoning chain as an opaque black box—unable to diagnose precisely where in the multi-step chain an inferential breakdown occurred or what category of error caused it. This blind spot stems from three fundamental barriers: first, step-level verification has flourished in formal domains like mathematics and code where deterministic ground truth or unit tests exist, but fails to generalize to textual reasoning fraught with natural language ambiguity; second, prior literature has predominantly focused on "process faithfulness" (whether the rationales mirror the model's internal computational mechanism) while leaving "context faithfulness" (whether every inferential step is strictly justified by the provided source context) largely unexplored at the step level; third, no grounded error taxonomy existed to diagnose intermediate textual failures.

To bridge this diagnostic gap, the community requires an evaluation benchmark that assesses intermediate steps inside the reasoning chain, categorizing whether a failure stems from a deductive breakdown or a factual grounding defect. Core idea: by discovering a two-track, eight-category error taxonomy bottom-up through unsupervised clustering of tens of thousands of unconstrained step critiques and establishing a human-annotated test set on inter-judge disagreement boundaries, GRACE provides the first step-level context faithfulness benchmark for textual reasoning and validates its utility as a process reward for policy optimization.

Method

Overall Architecture

The GRACE pipeline is organized into three sequential stages: Data Curation & Trace Generation (Stage 1), Bottom-Up Taxonomy Discovery (Stage 2), and Agreement-Based Data Splitting & Human Annotation (Stage 3). Given reference documents \(R = \{r_1, \dots, r_k\}\), a question \(q\), and a reasoning trace \(S = (s_1, \dots, s_n)\) generated by one of 10 diverse LLMs with explicit step-wise citations, the evaluator is tasked with predicting for every step \(s_i\) a binary faithfulness label \(y_i \in \{\text{FAITHFUL}, \text{UNFAITHFUL}\}\), an error category \(g_i \in \mathcal{T}\), and a free-form natural language explanation \(e_i\).

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400, 'subGraphTitleMargin': {'top': 8, 'bottom': 16}}}}%%
flowchart TD
    A["Multi-Source Grounded Datasets<br/>MuSiQue / 2WikiMHQA / ReClor / LogiQA"] --> B["Multi-Model Trace Generation<br/>10 Open & Proprietary LLMs (160K traces, 640K steps)"]
    B --> C["Open Critique & Unsupervised Clustering<br/>Qwen3.5-27B critiques → BGE embeddings → UMAP+HDBSCAN"]
    C --> D["Two-Track Error Taxonomy Consolidation<br/>GRACE-Inference & GRACE-Grounding (8 categories)"]
    subgraph DUAL["Two-Track Taxonomy Structure"]
        direction TB
        D1["GRACE-Inference (Deductive)<br/>Reversed Reasoning / Wrong Argument Reading / Rule Violation / Overreaching Claim"]
        D2["GRACE-Grounding (Factual)<br/>Groundedness Violation / Contradiction / Confusion / Evidence Neglect"]
    end
    D --> DUAL
    DUAL --> E["Multi-Judge Tiering & Disagreement Adjudication<br/>Gemma/Qwen 3-judge vote → Gold/Silver/Disputed → Human meta-adjudication"]
    E --> F["Downstream Alignment & Process Supervision<br/>GRACE-train SFT diagnostic models / Faithfulness PRM for GRPO RL"]

Key Designs

1. Context-Grounded Dataset Filtering and Multi-Model Reasoning Trace Generation: Eliminating External Leaks and Covering Varied Failure Modes

Standard logical reasoning benchmarks (such as ReClor and LogiQA) contain numerous questions involving unstated counterfactuals or hypothetical external propositions (e.g., "which of the following, if true, would weaken the argument?"), making the definition of context faithfulness ill-posed. The authors deployed an automated prompt-based groundedness filter using Qwen3.5-27B to filter out questions requiring meta-logical fallacy labels or unstated premises, dropping 65% of ReClor and 52% of LogiQA. For evidence-grounded multi-hop reasoning, 3- to 4-hop answerable splits of MuSiQue and 2WikiMultiHopQA were curated. Across these four filtered datasets, single-shot CoT traces with explicit per-step reference bracket citations were generated using 10 models spanning four open-weight families (Qwen, Llama, Mistral, Gemma from 7B to 35B) and two frontier proprietary families (Gemini-3-Flash, GPT-4o-mini), producing an initial pool of 160K reasoning traces and over 640K individual steps.

2. Data-Driven Bottom-Up Discovery of Two-Track Error Taxonomy: Clustering Free-Form Critiques into Deductive and Grounding Tracks

Rather than prespecifying arbitrary human error labels, GRACE derived its taxonomy organically from observed failure mechanisms. Over 30K unfaithful steps identified across all model families were critiqued without category constraints by Qwen3.5-27B. The combined natural language error descriptions and self-generated tags were embedded via BGE, dimensionally reduced with UMAP, and clustered into 138 high-density clusters via HDBSCAN. Manual consolidation of shared error dynamics yielded 8 primary failure categories. An empirical audit revealed a decisive operational separation (\(\ge 97\%\) divergence): exam deduction tasks (LogiQA/ReClor) suffered almost exclusively from inference reasoning breakdowns (98.6%~99.1%), whereas multi-hop passage retrieval tasks (MuSiQue/2WikiMHQA) suffered almost exclusively from evidence handling failures (97.0%~99.9%). This separation formalized the two evaluation tracks: - GRACE-Inference Track: Evaluates deductive integrity against stated premises. - Reversed Reasoning: Inverts a logical or causal condition, confusing necessity with sufficiency or swapping antecedent and consequent. - Wrong Argument Reading: Misidentifies the passage's argument structure, treating a premise as a conclusion or refuting a target the text never asserted. - Rule Violation: Disregards an explicit constraint or stated boundary condition in the premise. - Overreaching Claim: Draws a conclusion stronger than warranted, converting possibility into certainty or ignoring qualifying hedges. - GRACE-Grounding Track: Evaluates factual faithfulness to reference texts. - Groundedness Violation: Introduces ungrounded claims via pure fabrication, parametric knowledge injection, or unevidenced speculation. - Contradiction: Direct opposition to an explicit factual statement, attribute, or numerical quantity in the reference. - Confusion: Accurate context facts attached to the wrong entity, transposed relationships, or mixed attributes. - Evidence Neglect: Falsely asserting that information is missing when explicitly provided, or failing to track dynamic narrative state updates. Crucially, when re-run from scratch using an out-of-family critique model (Gemma-4-31B-IT), the resulting critique embeddings mapped into the existing cluster centroids with an average cosine similarity of 0.849, reproducing the identical eight categories and confirming that the taxonomy captures invariant model failure modes rather than single-evaluator idiosyncratic wording.

3. Multi-Judge Agreement Partitioning and Expert Human Meta-Adjudication: Building a Robust Diagnostic Test Bed

To scale annotation while preserving high quality on boundary conditions, GRACE established a multi-judge protocol using three open LLMs (Gemma-4-27B, Qwen3.5-27B, Qwen3.5-35B-A3B). Steps achieving unanimous consensus on binary faithfulness (79.2%) and error category (62.1%) were designated Gold, while majority consensus formed Silver. To ensure that the test set remains challenging and resistant to saturation, the evaluation split was curated specifically from traces where LLM judges exhibited severe disagreement (Disputed). Four trained human annotators with expertise in LLM reasoning evaluated candidate test traces in double-blind pairs using a specialized three-panel web interface. In cases of inter-annotator disagreement, a fifth senior meta-annotator reviewed and adjudicated the conflicts through joint deliberation, achieving inter-annotator agreements of Cohen's \(\kappa = 0.82\) for binary faithfulness and \(\kappa = 0.75\) for error category. The final benchmark consists of 437 human-annotated test traces (2,044 steps) and a balanced training set of 6,915 consensus traces (29,612 steps).

4. Faithfulness-Aware Process Reward Formulation for GRPO: Guiding Downstream Reasoning Reliability

Beyond static diagnostic evaluation, GRACE demonstrates that step-level context faithfulness signals can directly guide model policy optimization. In a Group Relative Policy Optimization (GRPO) framework, each prompt samples a group of \(G\) candidate reasoning traces. While standard training relies purely on answer format compliance \(R_{\text{fmt}}\) and gold answer word-level overlap \(R_{\text{F1}} = \text{F1}(\hat{a}, a^*)\), GRACE incorporates a dense process reward \(R_{\text{proc}}\) provided by a lightweight Qwen3-4B classifier fine-tuned on GRACE-train. For a sequence of \(K\) steps where each step receives a binary faithfulness judgment \(s_k \in \{0, 1\}\), the process reward represents the trajectory's faithful step ratio:

\[R_{\text{proc}} = \frac{1}{K} \sum_{k=1}^K s_k\]

The combined reward \(R = R_{\text{fmt}} + R_{\text{F1}} + R_{\text{proc}}\) drives the normalized group advantage calculation \(A^{\text{GRPO}}_i = \frac{R_i - \bar{R}_g}{\sigma_g + \epsilon}\), penalizing trajectories that take ungrounded knowledge shortcuts or introduce logical leaps even if their final answer happens to match the ground truth.

Loss & Training

The training pipelines detailed in the paper encompass two paradigms: 1. Supervised Fine-Tuning (SFT): Evaluator models (Qwen3-4B/8B, Llama-3.1-8B) were fine-tuned on GRACE-train using next-token cross-entropy over step-level labels and natural language explanations. Optimization utilized LoRA (\(r=128, \alpha=32\) on all linear modules) with AdamW, learning rate \(1\times 10^{-4}\), effective batch size 16, over 3 epochs. 2. GRPO Policy Optimization: Policy models (Qwen3-1.7B, 4B, 8B) were trained on source task questions for 2,000 steps with LoRA (\(r=16, \alpha=32\)), learning rate \(5\times 10^{-5}\), group size \(G=16\), and clipping bound \(\epsilon_c=0.2\), comparing outcome-only rewards against composite outcome-plus-PRM rewards.

Key Experimental Results

Main Results

Evaluations on the human-annotated GRACE-test set (2,044 steps) across proprietary, frontier open-weight, and GRACE-trained models measuring step-level binary F1 (Step F1) and error-category macro-F1 (Cat Macro-F1).

Model Group / Model Name GRACE-Grounding (MuSiQue / 2Wiki) Step F1 / Cat F1 GRACE-Inference (LogiQA / ReClor) Step F1 / Cat F1 Average Step F1 Average Cat Macro-F1
Proprietary Frontier
Gemini-3.1-Pro 78.06 / 69.56 83.28 / 78.77 81.46 72.08
GPT-5.4 80.50 / 53.53 73.20 / 52.18 78.53 58.43
Gemini-3-Flash 69.47 / 56.75 73.65 / 59.17 75.19 63.36
GPT-5.4-mini 74.12 / 37.16 59.69 / 33.71 66.85 41.57
Open-Weight Frontier
Gemma-4-31B 72.63 / 55.47 76.69 / 52.02 77.46 58.50
Qwen3.5-35B-A3B 75.50 / 51.64 71.12 / 37.04 75.70 47.15
Qwen3.5-27B 67.59 / 52.10 75.76 / 44.27 74.59 54.36
Qwen3-14B 60.92 / 35.26 49.04 / 25.19 56.60 35.17
Qwen3-8B 63.47 / 31.37 42.06 / 16.43 49.56 27.21
Llama-3.1-8B 53.37 / 20.30 41.96 / 32.05 45.76 26.88
GRACE-Trained (SFT)
Qwen3-8B-SFT 75.68 / 61.95 62.59 / 54.80 71.21 61.23
Qwen3-4B-SFT 71.98 / 63.31 63.30 / 62.26 68.57 64.76
Llama-3.1-8B-SFT 75.74 / 53.37 54.61 / 48.50 66.04 58.03

Ablation Study 1: Impact of Inference Strategy and Error Taxonomy Prompting

Ablations contrasting all-steps joint evaluation with taxonomy (All), step-by-step sequential evaluation with taxonomy (SbS), and binary evaluation omitting the taxonomy (Bin).

Model & Strategy Configuration Grounding Step F1 Grounding Cat F1 Inference Step F1 Inference Cat F1 Average Step F1 Average Cat F1 Empirical Observations
Gemma-4-31B (All) 77.3 58.9 77.7 58.1 77.5 58.5 Best overall Step F1 across both tracks
Gemma-4-31B (SbS) 65.2 70.2 69.4 48.7 67.3 59.5 Step F1 degrades sharply (-10.2) while Grounding Cat F1 rises
Gemma-4-31B (Bin) 69.9 – 67.6 – 68.8 – Removing taxonomy drops binary Step F1 by 8.7 points
Qwen3.5-27B (All) 74.7 64.4 74.5 44.3 74.6 54.4 Balanced joint reasoning across full chain
Qwen3.5-27B (SbS) 75.7 64.7 75.6 48.0 75.6 56.3 Benefits slightly (+1.0/+1.9) from reduced input complexity
Qwen3.5-27B (Bin) 72.1 – 65.3 – 68.7 – Removing taxonomy drops binary Step F1 by 5.9 points
Qwen3-8B-SFT (All) 77.3 69.6 65.1 52.9 71.2 61.2 SFT models favor full-trace context over SbS
Qwen3-8B-SFT (SbS) 77.3 59.0 64.2 57.4 70.7 58.2 Category diagnosis drops 3.0 points

Ablation Study 2: Process Reward Supervision in GRPO Reinforcement Learning

Downstream task F1 performance on Inference and Grounding tracks, alongside step-level faithfulness scores (evaluated by frozen Qwen3.5-27B and Gemma-4-31B judges).

Model Scale RL Policy Configuration Inference Task F1 Grounding Task F1 Step Faithfulness (Qwen) Step Faithfulness (Gemma) Core Phenomenon & Insight
1.7B Base Model 50.7 17.7 67.7 68.2 Weak initial factual grounding baseline
1.7B GRPO (F1 reward only) 53.5 39.1 63.6 64.8 Reward hacking: Task F1 gains cause faithfulness to drop ~4 pts
1.7B GRPO (F1 + PRM) 56.0 40.1 74.6 74.1 PRM suppresses ungrounded shortcuts; Faithfulness jumps +11 pts
4B Base Model 69.5 49.1 77.3 74.3 Moderate pre-trained zero-shot performance
4B GRPO (F1 reward only) 70.2 53.8 77.7 75.6 Outcome reward plateaus early at 0.62–0.64
4B GRPO (F1 + PRM) 72.3 55.2 78.7 76.2 PRM sustained gradient signal breaks through to 0.65–0.66
8B Base Model 69.7 53.2 78.3 78.4 Strong zero-shot reasoning ability
8B GRPO (F1 reward only) 73.5 60.2 76.1 77.5 Task F1 improves but intermediate faithfulness degrades
8B GRPO (F1 + PRM) 73.9 63.5 79.1 79.8 Simultaneously achieves highest task F1 and ~80% step faithfulness

Key Findings

  • Correct Outcomes Mask Severe Step Unfaithfulness (49.5% Disconnect): Across the GRACE test set, 49.5% of traces exhibiting at least one unfaithful step still produced the correct final answer (47.3% in Inference, 51.6% in Grounding). Case studies show that models frequently fall back on memorized world knowledge shortcuts (e.g., injecting ungrounded geographic facts about Stockholm districts to answer correctly) or stumble through erroneous premises that are accidentally corrected by a later step. Outcome-based evaluation creates a false sense of security regarding model reliability.
  • Categorization Gap Highlights Diagnostic Complexity: Even the leading proprietary model, Gemini-3.1-Pro, experienced an 9.38-point drop from Step F1 to Cat Macro-F1 (81.46 vs. 72.08). For strong open-weight models like Gemma-4-31B, this gap expanded to 18.96 points. Recognizing that an inference step is flawed is far simpler than diagnosing whether the flaw represents an inverted implication, an overreaching claim, or an entity confusion.
  • SFT on GRACE-Train Empowers Small Models: Fine-tuning compact models on the consensus training set effectively bridges the capability divide. Qwen3-4B-SFT attained an average Cat Macro-F1 of 64.76, outperforming Gemini-3-Flash (63.36) and GPT-5.4 (58.43), indicating that structured diagnostic taxonomies can be effectively internalized via targeted supervision.
  • Explicit Taxonomies Provide Vital Scaffolding: Omitting the error taxonomy during inference caused average Step F1 to collapse by 8.7 points on Gemma-4-31B and 5.9 points on Qwen3.5-27B. Structured categorical definitions provide essential inductive scaffolding that guides LLMs in scrutinizing claims against context.

Highlights & Insights

  • Data-Driven Discovery of Natural Task Separation: By applying unsupervised clustering directly to unconstrained model critiques, the study objectively demonstrated that context reasoning failures naturally decouple into deductive inferential fallacies and factual grounding violations.
  • Exposing the Pitfalls of Outcome-Only RL: The study offers compelling empirical proof that optimizing models purely on answer accuracy encourages reward hacking—models exploit parametric memory shortcuts that degrade intermediate reasoning faithfulness. Step-level process rewards provide the necessary guardrail to align accuracy with procedural integrity.
  • Dual Structural Regimes Prevent Benchmark Overfitting: By deliberately pairing short-context/long-chain logical deduction (median 128 words, 5 steps) with long-context/short-chain multi-hop QA (median 1,562 words, 4 steps), GRACE forces evaluation models to exhibit versatile reasoning verification strategies.

Limitations & Future Work

  • Scope Restricted to Unstructured English Text: The benchmark is currently bounded by plain English narrative passages, lacking evaluation over semi-structured tables, relational knowledge bases, code artifacts, or multimodal context inputs.
  • Single Model Seed in Initial Open Critique: The initial open critique generation utilized a single model (Qwen3.5-27B) to draft error reflections before clustering. Although cross-validated by Gemma-4-31B, minor prompt-specific blind spots in discovering extremely rare error modes may persist.
  • Temporal Drift of Frontier Architectures: As test-time search and long-context reasoning models (such as o1 or R1) become ubiquitous, intermediate error modes may mutate, necessitating ongoing periodic test set expansion and taxonomy refinement.
  • vs REVEAL (Jacovi et al., 2024): REVEAL relies on post-hoc retrieved evidence from Wikipedia where errors may stem from retriever inadequacies rather than the model's CoT, and labels errors with binary/attribution indicators without diagnostic categorization. GRACE feeds all context directly as input, ensuring that errors reflect true model failures, and introduces an eight-class error taxonomy.
  • vs VersaPRM (Zeng et al., 2025): VersaPRM expands process supervision to non-math domains but relies on fully synthetic LLM annotations assigning a coarse quality score (Good/OK/Bad) without grounding verification. GRACE enforces context faithfulness and features human-adjudicated ground truth.
  • vs RAGTruth (Niu et al., 2024) & FAVABench (Mishra et al., 2024): These benchmarks target hallucinations at the span or response level, evaluating what the model generated but ignoring how it reasoned. GRACE opens the reasoning chain to evaluate each intermediate step.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Establishes the first step-level context faithfulness benchmark for textual reasoning with an empirically discovered two-track error taxonomy.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous data pipeline covering 10 diverse models, 160K traces, multi-judge tiering, double-blind human adjudication, and validation via SFT and GRPO RL.
  • Writing Quality: ⭐⭐⭐⭐⭐ Exemplary organization and prose clarity, sharp conceptual delineation between process and context faithfulness, and rich qualitative case studies.
  • Value: ⭐⭐⭐⭐⭐ Highly actionable resource for process reward modeling, trustworthy RAG systems, and mitigating reward hacking in mathematical/textual reinforcement learning.