Abstractive Snippet Generation¶

Conference: ACL 2025
Code: None
Area: Text Generation
Keywords: Abstractive Snippet Generation, Search Engine Snippets, Query-Focused Summarization, Abstractive Summarization, Information Retrieval

TL;DR¶

This paper proposes an abstractive snippet generation method for search engines. By utilizing query-aware summarization generation techniques, it generates more concise and informative text snippets for search result pages compared to traditional extractive snippets, significantly improving the user search experience.

Background & Motivation¶

Background: Search engines typically generate a text snippet for each webpage when displaying search results, helping users quickly determine whether the webpage content meets their needs. Currently, major search engines employ extractive methods (extractive snippets), directly selecting sentence fragments from the webpage that contain the query keywords.

Limitations of Prior Work: Extractive snippets suffer from several key issues: first, they often truncate sentences, leading to incomplete semantics; second, the selected fragments may be scattered across different parts of the document, lacking coherence; third, they fail to synthesize information from multiple paragraphs to address complex queries; fourth, when query words occur infrequently in the document, the quality of the generated snippet is poor.

Key Challenge: Users hope to quickly obtain answers to their questions through snippets, but extractive methods can only perform "locate and display" rather than "understand and summarize." A semantic gap exists between the search query's informational needs and the original expression of the document.

Goal: Design an end-to-end abstractive snippet generation system capable of generating concise, accurate, and coherent summary snippets based on user queries and target documents.

Key Insight: Drawing on the framework of Query-Focused Summarization (QFS), snippet generation is redefined as a conditional generation problem while introducing special constraints tailored to search scenarios.

Core Idea: Snippet generation is modeled as a query-conditioned abstractive summarization task. By combining retrieval relevance signals and generation quality control, the model generates summary snippets that are both faithful to the document and optimized for the query.

Method¶

Overall Architecture¶

The input consists of a user query \(q\) and a candidate document \(d\), and the output is a short, abstractive summary snippet \(s\). The model architecture is based on a pre-trained sequence-to-sequence model (such as BART or T5), completing the task through two stages: joint query-document encoding and controlled generation.

Key Designs¶

Query-Aware Document Encoding:
- Function: Integrates query information into the document encoding process, enabling the encoder to focus on the parts of the document most relevant to the query.
- Mechanism: Employs a cross-attention mechanism to inject query tokens as extra context into the document's self-attention layers. For long documents, paragraph-level retrieval is first used to filter the top-\(k\) most relevant paragraphs, followed by fine-grained encoding. The query-document relevance score \(r(q, p_i)\) is used to weight the contributions of different paragraphs.
- Design Motivation: Documents can be long and have uneven information density; the query provides a critical signal for information filtering.
Faithfulness-Aware Decoder:
- Function: Ensures that the content of the generated snippet is faithful to the original document during the generation process, avoiding hallucinations.
- Mechanism: Introduces a copy mechanism during decoding, allowing the model to directly copy key terms and factual information from the document. Meanwhile, a faithfulness constraint is set up, aligning the generated snippets with the document content in the semantic space via contrastive learning. At each step, the decoder calculates both generation and copy probabilities, dynamically selecting between them via a gating mechanism.
- Design Motivation: The core requirement of search snippets is to accurately reflect the document content; hallucinatory content can seriously mislead users.
Length and Information Density Control:
- Function: Controls the length and information density of the generated snippet to make it suitable for display in search results.
- Mechanism: Introduces a length reward mechanism that encourages the model during training to generate snippets within a specified length range (typically 80-160 words). Simultaneously, an information coverage metric is used to measure how well the generated snippet satisfies the informational needs of the query, acting as an auxiliary training objective.
- Design Motivation: Search snippets have strict limits on display space, requiring maximization of information density within a limited word count.

Loss & Training¶

The primary training loss is the standard cross-entropy loss for sequence generation. Auxiliary losses include: contrastive faithfulness loss (ensuring semantic consistency between the generated content and the source document), query coverage loss (ensuring the snippet addresses the query requirements), and a length penalty term. The training data is derived from two distantly supervised sources: click-through data from search engine logs and human-annotated query-snippet pairs.

Key Experimental Results¶

Main Results¶

Method	ROUGE-L	BERTScore	Faithfulness	User Preference Rate
Ours	38.7	72.4	91.3%	64.2%
BART-QFS	35.2	69.8	87.5%	48.7%
Extractive Baseline	31.4	65.3	96.8%	35.1%
GPT-3.5 (zero-shot)	33.9	70.1	82.4%	52.0%

Ablation Study¶

Configuration	ROUGE-L	Faithfulness	Description
Full model	38.7	91.3%	Full model
w/o Query-Aware Encoding	34.1	90.8%	ROUGE dropped by 4.6 after removing query information
w/o Copy Mechanism	37.2	84.7%	Faithfulness drops significantly without copying
w/o Length Control	36.8	89.5%	Variance of generated length increases

Key Findings¶

An obvious trade-off exists between faithfulness and quality of expression: purely extractive methods achieve the highest faithfulness but have poor readability, while purely generative methods are flexible but prone to hallucinations. The hybrid copy-generator strategy of this work achieves a good balance.
Query-aware encoding yields the most significant improvements for queries with complex informational needs (such as "how/why" queries), while offering limited gains for simple navigational queries.
User preference evaluations indicate that humans clearly prefer abstractive snippets over traditional extractive ones.

Highlights & Insights¶

Successfully applies the theoretical framework of query-focused summarization to the industrial scenario of search engine snippet generation, demonstrating strong practical value. Since search engine snippets serve as the entry point for billions of searches daily, improving them has a massive impact.
The design of the faithfulness-aware decoder cleverly balances flexibility and accuracy; this idea can be transferred to other generation tasks requiring strict factual consistency (such as medical text summarization).
Proposes specialized evaluation dimensions for snippet generation (query coverage, info density, and faithfulness), complementing the shortcomings of traditional ROUGE metrics.

Limitations & Future Work¶

The training data relies on search engine logs, which may introduce data bias and privacy issues.
Applicability to multilingual scenarios requires further validation.
Real-time requirements: Search engines demand extremely low latency (millisecond level), making the inference latency of abstractive generation the main barrier to deployment.
Exploring the integration of Retrieval-Augmented Generation (RAG) frameworks with snippet generation could further improve quality.

vs. QFS (Query-Focused Summarization): Traditional QFS focuses on generating longer summaries, whereas this work focuses on short snippet generation, which requires additional consideration of display constraints and user interaction modalities.
vs. BART/T5 for Summarization: General summarization models do not consider query constraints; this work addresses this limitation through query-aware encoding.
This work is closely related to answer generation in RAG systems, where snippet generation can be viewed as a specific application of the RAG generation phase.
It is also related to answer snippet extraction in information extraction, but this work adopts a generative rather than an extractive approach.

Rating¶

Novelty: ⭐⭐⭐ Application innovation based on the existing QFS framework; method-level novelty is moderate, but the choice of application scenario is precise.
Experimental Thoroughness: ⭐⭐⭐⭐ Combines automatic evaluation with human preference evaluation; the multidimensional evaluation design is worth emulating.
Writing Quality: ⭐⭐⭐⭐ The problem definition is clear, motivations are sufficient, and the practical value is well-defined.
Value: ⭐⭐⭐⭐ Offers direct industrial deployment value for optimizing search engine experience, with a broad scope of impact.