Concise and Sufficient Sub-Sentence Citations for Retrieval-Augmented Generation¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/chen-kkk/cs-citation
Area: LLM / NLP
Keywords: Retrieval-Augmented Generation, Fine-Grained Citation Generation, Sub-Sentence Citations, Synthetic Data Filtering, Two-Stage Fine-Tuning
TL;DR¶
Addressing the issue where sentence-level citations in RAG are frequently cluttered with irrelevant clauses while isolated keywords lack necessary referential antecedents, this paper defines a concise and sufficient sub-sentence citation task, constructs a 2,000-instance benchmark with 95% inter-annotator agreement, and proposes a dual-stage filtered data augmentation and two-stage fine-tuning framework that achieves 80.46% F1 on Llama-3.1-8B while reducing human verification time by 41%.
Background & Motivation¶
Large language models have become the standard backbone for open-domain question answering and knowledge-intensive applications, yet their persistent susceptibility to factual hallucinations continues to undermine their deployment in high-stakes environments. Retrieval-augmented generation (RAG) mitigates this vulnerability by grounding model outputs in externally retrieved documents. However, retrieved passages are frequently extensive, noisy, or only tangentially related to the target query, placing an overwhelming cognitive reading burden on human users who attempt to verify the correctness of generated responses. To alleviate this challenge, an extensive line of recent research has focused on generating citations that directly link claims to supporting context spans, evolving from coarse paragraph or fixed-chunk references to finer-grained sentence-level attributions.
Nonetheless, complete sentences are seldom the optimal unit of empirical reference for efficient human verification. In practice, a full sentence often embeds redundant subordinate clauses, peripheral background descriptions, or unrelated parallel entities, resulting in substantial over-citation that dilutes the reader's attention. Conversely, if a system naively excises only the localized lexical string carrying the answer, it frequently suffers from severe under-citation due to missing pronominal referents, elided grammatical subjects, or context-dependent semantic modifiers, leaving the isolated fragment ambiguous when viewed independently. Meanwhile, post-hoc attribution and feature-attribution methods that produce scattered token-level saliency scores or token heatmaps remain poorly suited for end-user interaction, as fragmented lexical tokens fail to provide coherent, readable evidence.
This fundamental tension between conciseness and referential sufficiency highlights the necessity of descending beyond sentence boundaries while actively preserving necessary antecedents and natural phrasing. Core idea: formalize the task of concise and sufficient sub-sentence citation generation, curate a high-agreement double-annotated benchmark across three distinct citation archetypes, and build a scalable data augmentation pipeline combining rule-based verification, multi-dimensional evaluator filtering, and two-stage fine-tuning to deliver minimal, self-contained evidence spans.
Method¶
Overall Architecture¶
The proposed system establishes an end-to-end framework for generating direct answers alongside minimal, self-contained supporting facts for RAG question answering. Given an input question \(q\) and a retrieved context \(C\), the generator outputs the final answer \(a\) along with a set of explicit evidence spans \(E = \{e_1, e_2, \dots, e_m\}\) copied verbatim from \(C\), with each span anchored to its corresponding line identifier to ensure complete provenance without synthetic hallucination.
To overcome the severe annotation bottleneck associated with fine-grained span labeling while safeguarding against pseudo-label corruption, the framework operates across a multi-stage construction and training lifecycle: first, a double-annotated benchmark of 2,000 instances is curated with an exact-match consensus criterion; second, a large unlabeled QA corpus is populated with candidate sub-sentence citations that undergo a two-tier filtering cascade composed of hard rule validation and soft evaluator scoring; finally, open-source base LLMs are fine-tuned via a two-stage curriculum that transitions from broad structural pattern learning on filtered synthetic data to rigorous boundary calibration on gold human annotations.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input: Question q + Retrieved Context C"] --> B["High-Agreement Human Benchmark Construction<br/>XOR QA / SQuAD / HotpotQA Double Annotation (2K)"]
B --> C["Dual-Tier Filtered Synthetic Augmentation<br/>Unlabeled Pool Generation โ Rule Checks + 6-Dim Evaluator Scoring"]
C --> D["Two-Stage Curriculum Fine-Tuning<br/>Filtered Synthetic Pre-Training โ Human Gold Calibration"]
D --> E["Output: Answer a + Minimal Self-Contained Spans E"]
Key Designs¶
1. High-Agreement Human Benchmark Construction: Formalizing Three Sub-Sentence Paradigms and Disambiguated Standards
To establish a rigorous foundation for sub-sentence attribution, this design formalizes three operational criteriaโsufficiency (retaining all necessary subjects and contextual anchors), conciseness (excluding non-essential clauses and modifiers), and readability (preserving natural clausal or phrasal syntactic boundaries rather than isolated keywords). Instances are explicitly categorized into three distinct operational types: Type-1 comprises cases where the complete sentence is already naturally concise and sufficient; Type-2 encompasses sentences containing extraneous clauses where only the focal answer-bearing clause should be extracted; Type-3 represents complex cases where the answer-bearing clause alone lacks an explicit referent (e.g., beginning with a demonstrative or third-person pronoun), requiring the extraction of an antecedent span alongside the core evidence. Drawing from XOR QA, SQuAD, and HotpotQA to cover cross-lingual, reading-comprehension, and multi-hop reasoning contexts, two annotators independently labeled candidates under an exact span-match agreement protocol (reaching 95% consensus). Disagreements were strictly discarded to ensure a zero-ambiguity gold benchmark of 2,000 instances (split 1,000 train / 1,000 test), with challenging Type-3 cases comprising 50% of the distribution.
2. Dual-Tier Filtered Synthetic Augmentation: Rule-Based Validation and Multi-Dimensional Evaluator Scoring
Because fine-grained span extraction is exceptionally vulnerable to label noise, naively training on unfiltered synthetic citations generates severe degradation due to ungrounded spans, incorrect answers, or syntax truncation. This design introduces a hybrid filtering cascade combining deterministic rule validation with an LLM evaluator scoring model. The rule-based validation checks JSON parsing integrity, answer consistency against ground truth via normalized fuzzy matching (\(\ge 0.85\)), exact span presence in context, and context-relative length ratio penalties \(r = L_c / L_x\) (applying heavy penalties if \(r > 0.60\) or if span count exceeds 5), yielding a rule score \(s_h \in [0, 1]\); instances must satisfy \(s_h \ge 0.50\) and span existence to proceed. Passing candidates are subsequently graded by a fine-tuned evaluator across six normalized dimensions: factual fidelity \(f_f\), answer correctness \(f_a\), sufficiency \(f_s\), conciseness \(f_c\), coherence \(f_r\), and overall impression \(f_o\), aggregated via the weighted objective:
The composite quality score is formulated as \(S = 100 \times (0.40s_h + 0.60s_e)\). Retaining only instances with \(S \ge 80\) yields 10,000 pristine synthetic training examples that reliably balance evidential compactness with referential completeness.
3. Two-Stage Curriculum Fine-Tuning: Decoupling Structural Pattern Acquisition from Boundary Calibration
Directly pooling large-scale synthetic data with scarce human annotations causes the noisy tail of synthetic spans to dilute the sharp boundary distinctions provided by human experts. This design implements a two-stage curriculum fine-tuning regimen. In the first stage, the base generator is fine-tuned over the 10,000 evaluator-filtered synthetic instances to acquire the broad structural mechanics of sub-sentence extractionโsuch as learning the target JSON schema, extracting single clauses from compound sentences, and simultaneously identifying separated antecedent noun phrases. In the second stage, training continues exclusively on the 1,000 (or 500) human gold instances with a decayed learning rate. This sequential separation ensures the model first learns broad grammatical extraction patterns across diverse discourse structures and subsequently calibrates its decision boundaries strictly to the target standard of minimal sufficiency and syntactic readability.
Loss & Training¶
The generation model is optimized using standard causal language modeling autoregressive cross-entropy loss over the formatted JSON output tokens. All base architectures are adapted using LoRA parameter-efficient fine-tuning with rank \(r = 8\), scaling factor \(\alpha = 32\), and dropout 0, incorporating the LoRA+ optimizer setting with a B-matrix learning rate multiplier ratio of 16. Optimization employs AdamW with an initial learning rate of \(5 \times 10^{-5}\) and a cosine decay schedule. Models are trained for 4 epochs in bf16 mixed precision with gradient clipping at max norm 1.0, utilizing a per-device batch size of 2 and 8 gradient accumulation steps (effective batch size 16). The synthetic pre-training checkpoint serves as the explicit weight initialization for the subsequent manual refinement stage.
Key Experimental Results¶
Main Results¶
Evaluation is conducted on the 1,000-instance human-annotated test set. Each supporting fact is structured as a tuple of text span and line identifier. Precision (P) and Recall (R) are computed by matching predicted and reference citations sharing identical line IDs via character 3-gram overlap F1 under a greedy best-match assignment, yielding citation F1. Citation Length (CL) measures the average number of normalized non-space characters in predicted citations. The table below presents the core performance across three open-source foundations (quoted directly from Table 1 of the paper):
| Foundation Model | Strategy / Setup | Recall (R) | Precision (P) | Citation F1 (%) | ROUGE-L | BLEU | Citation Length (CL) โ |
|---|---|---|---|---|---|---|---|
| Llama-3.1-8B | Base (Zero-shot / Few-shot) | 26.85 | 18.09 | 20.61 | 58.49 | 43.87 | 142.21 |
| Llama-3.1-8B | Synthetic Only (10K) | 77.81 | 79.33 | 77.86 | 86.40 | 78.35 | 69.15 |
| Llama-3.1-8B | Manual Only (0.5K) | 75.28 | 76.13 | 74.84 | 85.51 | 77.68 | 70.07 |
| Llama-3.1-8B | Full Method (10K + 0.5K) | 78.10 | 78.30 | 77.53 | 86.17 | 77.90 | 65.12 |
| Llama-3.1-8B | Manual Only (1K) | 77.23 | 78.50 | 77.15 | 86.26 | 77.97 | 67.56 |
| Llama-3.1-8B | Full Method (10K + 1K) | 81.20 | 80.98 | 80.46 | 87.75 | 80.72 | 66.72 |
| Qwen2.5-7B | Base (Zero-shot / Few-shot) | 27.57 | 28.71 | 27.89 | 63.83 | 50.09 | 137.07 |
| Qwen2.5-7B | Synthetic Only (10K) | 77.80 | 79.17 | 77.67 | 86.28 | 78.23 | 68.30 |
| Qwen2.5-7B | Manual Only (1K) | 75.46 | 75.55 | 74.74 | 84.95 | 76.75 | 70.05 |
| Qwen2.5-7B | Full Method (10K + 1K) | 78.58 | 77.46 | 77.81 | 86.01 | 77.64 | 67.52 |
| GLM-4-9B | Base (Zero-shot / Few-shot) | 34.47 | 35.77 | 34.74 | 59.63 | 47.81 | 128.32 |
| GLM-4-9B | Synthetic Only (10K) | 73.91 | 74.76 | 73.70 | 83.58 | 75.38 | 75.20 |
| GLM-4-9B | Manual Only (1K) | 71.42 | 72.57 | 71.99 | 82.45 | 73.44 | 73.74 |
| GLM-4-9B | Full Method (10K + 1K) | 74.23 | 73.58 | 73.90 | 83.45 | 75.57 | 71.12 |
Comparative evaluation against existing citation frameworks, attribution methods, heuristic refinement, and strong prompting baselines is detailed below (quoted directly from Table 3 of the paper):
| Paradigm Category | Method / Configuration | Citation F1 (%) | Citation Length (CL) โ |
|---|---|---|---|
| Sentence-Level Systems | LongCite-Llama-3.1-8B | 45.29 | 248.96 |
| Sentence-Level Systems | ALCE | 52.46 | โ |
| Post-Hoc Attribution | RAG-Ex 2.0 | 58.91 | โ |
| Context Ablation | ContextCite | 47.70 | 143.28 |
| Localized Attribution | LAQuer | 70.04 | 58.97 |
| Heuristic Shortening | LongCite + Qwen refine | 51.20 | 94.62 |
| Heuristic Shortening | LongCite + GPT-4o refine | 53.61 | 73.86 |
| Prompted Frontier Models | GPT-4o prompting (10-shot) | 65.26 | โ |
| Prompted Frontier Models | GPT-5.4 prompting | 67.42 | 70.60 |
| Prompted Frontier Models | Qwen-3.6 prompting | 65.16 | 65.14 |
| Proposed Approach | Ours (Llama-3.1-8B Full Method) | 80.46 | 66.72 |
Ablation Study¶
Ablation on evaluator filtering quality control (Table 2) and training strategy comparison (Table 6) on Llama-3.1-8B:
| Experimental Group | Configuration / Condition | Citation F1 (%) | Key Mechanism & Finding |
|---|---|---|---|
| Data Filtering Impact | Manual Only 1K | 77.15 | Clean human baseline |
| Data Filtering Impact | Synthetic 10K w/o filtering | 73.24 | Unfiltered noise drops F1 by 3.91% below manual |
| Data Filtering Impact | Synthetic 10K w/ evaluator filtering | 77.86 | Evaluator filtering filters out noise, beating manual |
| Data Filtering Impact | Synthetic 10K filtered + Manual 1K | 80.46 | Dual-source synergy achieves optimal calibration |
| Training Scheme (1K Manual) | Synthetic Only (10K) | 77.86 | Synthetic coverage baseline |
| Training Scheme (1K Manual) | Synthetic + Manual shuffled | 78.35 | Direct pooling dilutes human boundary precision |
| Training Scheme (1K Manual) | Two-Stage Fine-Tuning | 80.46 | Sequential curriculum yields additional +2.11% gain |
Furthermore, human blind evaluation over 100 randomly sampled test instances (quoted from Table 5) confirms that the proposed method surpasses ContextCite and LAQuer in Sufficiency (84.5% vs 60.0% / 75.0%), Conciseness (78.0% vs 33.0% / 70.5%), and Readability (81.5% vs 42.0% / 71.0%), while slashing total human verification time from 23.5 minutes (ContextCite) and 16.5 minutes (LAQuer) down to 13.9 minutes.
Key Findings¶
- Data Filtering is Indispensable for Fine-Grained Tasks: Training on 10K unfiltered synthetic citations causes F1 to collapse from 77.15% (manual only) to 73.24%, proving that boundary noise severely misleads supervised alignment. Applying rule-and-evaluator filtering elevates performance to 77.86%, surpassing manual-only training and demonstrating that high-precision filtering converts synthetic generation into an effective supervisor.
- Two-Stage Training Prevents Signal Dilution: Shuffled mixed training yields 78.35% F1, whereas two-stage curriculum training reaches 80.46% F1 (+2.11%). Pre-training on broad synthetic patterns followed by second-stage refinement on pristine human data prevents high-volume synthetic instances from washing out the gold boundary signals.
- Heuristic Sentence Shortening Fails to Bridge the Gap: Prompting Qwen or GPT-4o to refine and shorten sentence-level LongCite outputs improves F1 only marginally (from 45.29% to 51.20% and 53.61%), remaining far below our 80.46%. This underscores that sub-sentence citation is not simple text compression, but requires recovering implicit subjects and disconnected antecedents.
- Type-3 Antecedent Recovery Remains the Core Bottleneck: Granular category analysis reveals that Type-1 achieves 88.40% F1, Type-2 reaches 79.94% F1, whereas Type-3 drops sharply to 66.91% F1. Because Type-3 comprises 50% of the benchmark, multi-span antecedent co-reference resolution represents the predominant locus of remaining model errors.
Highlights & Insights¶
- Reconceptualizing Fine-Grained Citations Around Semantic Sufficiency: Rather than treating citation refinement as mechanical token compression or saliency scoring, the paper formalizes the linguistic necessity of antecedents, demonstrating that true verification conciseness requires multi-span co-reference preservation to maintain independent human interpretability.
- Cascaded Dual-Tier Quality Filtering Paradigm: The combination of deterministic format/length heuristics and a multi-dimensional LLM evaluator provides an actionable, battle-tested recipe for synthetic alignment in sensitive tasks where token boundary shifts dramatically alter correctness.
- Direct End-to-End Human Time Savings: By directly measuring total human verification duration (reduced from 23.5 to 13.9 minutes), the authors demonstrate empirical operational utility, proving that removing non-essential clauses and maintaining explicit subjects directly lowers cognitive load.
Limitations & Future Work¶
- Constraint to Short-Form Question Answering: The benchmark centers primarily on concise, factoid QA benchmarks (XOR QA, SQuAD, HotpotQA) rather than complex, long-form expository generation featuring interwoven claims and multi-sentence dependencies.
- Computational Overhead of Filtering: Generating large synthetic candidate pools and executing multi-metric evaluator scoring incurs considerable computational expenditure during offline data construction.
- Hard Extractive Span Boundaries: Reliance on exact verbatim span extraction may occasionally fail to produce fully natural phrases when the underlying source document features highly convoluted or fragmented syntactic clauses.
Related Work & Insights¶
- vs Sentence-Level Systems (LongCite, ALCE, ReClaim): Sentence-level models force the citation of entire sentences, routinely including 50%+ irrelevant context; the proposed approach refines citations to sub-sentence spans, reducing citation length from ~249 characters to ~67 characters while boosting verification precision.
- vs Post-Hoc Attribution Methods (ContextCite, RAG-Ex): Feature attribution frameworks use input perturbation or attention weights, yielding scattered, fragmented token highlights that are slow and unintuitive for users; this method generates grammatically cohesive clausal spans that preserve natural reading flow.
- vs Localized Attribution (LAQuer): LAQuer is designed for user-directed query spans and frequently extracts overly terse, single-word phrases lacking antecedent context; this work automatically generates self-contained sub-sentence evidence spans during standard QA generation.
Rating¶
- Novelty: โญโญโญโญ [Well-motivated task formalization balancing conciseness and antecedent sufficiency with clean categorization]
- Experimental Thoroughness: โญโญโญโญโญ [Extensive comparisons across 3 open-source models, 10 baselines, thorough ablations, and real-world human timing verification]
- Writing Quality: โญโญโญโญโญ [Exemplary clarity, compelling motivation figures, well-structured methodology, and self-contained narrative prose]
- Value: โญโญโญโญโญ [Provides a direct, actionable solution for reducing user cognitive load in real-world RAG verification interfaces]