SumRank: Aligning Summarization Models for Long-Document Listwise Reranking¶
Conference: EMNLP 2026
Paper: ACL Anthology
Authors: Jincheng Feng, Wenhan Liu, Zhicheng Dou
Affiliations: Gaoling School of Artificial Intelligence, Renmin University of China
Area: LLM Reasoning
Keywords: Long-Document Ranking, Listwise Reranking, Document Summarization, Reinforcement Learning Alignment, GRPO
TL;DR¶
Proposes SumRank, a per-document summarization model aligned with downstream listwise reranking through a three-stage training framework (cold-start SFT, specialized static RL candidate construction, and rank-driven GRPO with explicit negative rejection), compressing lengthy documents into high-density rank-aligned summaries that achieve state-of-the-art ranking accuracy across TREC DL and BEIR while drastically reducing inference latency.
Background & Motivation¶
In modern information retrieval (IR), web search engines, and autonomous deep research agents, listwise reranking powered by Large Language Models (LLMs) has emerged as the state-of-the-art paradigm for prioritizing search results. Unlike pointwise or pairwise approaches that score candidates in isolation or pairs, listwise reranking feeds an entire candidate list into the LLM context window simultaneously. This enables the self-attention mechanism to jointly observe all candidate units and capture fine-grained cross-document relative relevance patterns. However, real-world retrieval tasks frequently operate over lengthy documents—such as multi-thousand-word web pages, legal opinions, and detailed academic articles—rather than short hundred-word passages. Directly feeding a list of long documents into an LLM reranker introduces severe context modeling degradation due to the quadratic complexity of standard self-attention, resulting in unacceptable inference latency and diminished ranking quality caused by dispersed noisy text.
To handle lengthy inputs, conventional search pipelines have relied on heuristic rule-based strategies such as truncation (e.g., FirstP, which preserves only the leading few hundred tokens) or chunk-based pooling (e.g., MaxP, which scores chunked passages and aggregates maximum scores). Yet, these strategies suffer from fundamental trade-offs: truncation irreversibly discards critical evidence buried deep in the body or conclusion of the document, while chunking destroys global semantic coherence across disjoint sections. Conversely, compressing long texts using off-the-shelf sequence-to-sequence summarization models (e.g., BART, LED, PEGASUS) or zero-shot foundational LLMs proves equally ineffective. Generic summarizers are pre-trained to produce smooth, generalized narratives optimized for human readability, and in doing so, they routinely discard rare domain keywords, precise quantitative metrics, and subtle conditional constraints that ranking models strictly require to compute discriminative relevance scores.
Overcoming the long-standing conflict between long-document ranking effectiveness and end-to-end computational efficiency requires fundamentally bridging the gap between text compression and downstream relevance discrimination. Core idea: Decouple long-document listwise reranking into an aligned "Summarize-then-Rank" pipeline by introducing SumRank, a lightweight summarizer trained via a three-stage framework—cold-start distillation, specialized candidate construction, and rank-driven GRPO alignment—to compress long documents into compact, relevance-grounded summaries with autonomous negative rejection, empowering downstream listwise LLM rankers to achieve SOTA accuracy at minimal inference overhead.
Method¶
Overall Architecture¶
SumRank adopts a two-stage "Summarize-then-Rank" paradigm. Given an input query \(q\) and an initial candidate document list \(\mathcal{D} = \{d_1, d_2, \dots, d_N\}\) (e.g., retrieved via BM25), the system first deploys a lightweight generative summarizer \(\pi_\theta\) to independently compress each candidate long document into a concise query-grounded summary \(s_i = \pi_\theta(d_i, q)\). Next, the compact summary list \(\mathcal{S} = \{s_1, s_2, \dots, s_N\}\) is fed into a downstream listwise LLM reranker \(R_\phi\) (such as Qwen2.5-32B-Instruct), which operates with a sliding window mechanism (window size \(w=20\), step size \(sz=10\)) to produce the final global document permutation \(\pi\).
To train the lightweight summarizer to preserve the exact signals necessary for downstream listwise ranking, SumRank establishes a structured three-stage training pipeline: 1. Stage 1 (Cold-Start SFT): Distills basic document compression and query-faithful extraction capabilities from a strong teacher LLM (Qwen2.5-72B-Instruct) on MS MARCO into the student model via full-parameter supervised fine-tuning; 2. Stage 2 (RL Data Construction): Gathers query-candidate groups containing guaranteed positive and negative instances, and pre-generates a static baseline summary list using the SFT checkpoint to serve as an anchor; 3. Stage 3 (Rank-Driven Alignment via GRPO): Randomly selects a single target document, samples multiple summary rollouts from the active policy, substitutes each rollout into the static background list, and computes relative policy advantages using downstream ranking NDCG@10 combined with an explicit rejection penalty.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Query & Candidate Long Documents<br/>Query q & Candidate Docs D"] --> B["Stage 1: Cold-Start SFT<br/>Teacher Distillation for Basic Compression"]
B --> C["Stage 2: RL Data Construction<br/>Build Stable Static Background Summary List"]
C --> D["Stage 3: Rank-Driven Alignment via GRPO<br/>Group Relative Policy Optimization on LLM Ranker"]
D --> E["Dynamic Reward & Explicit Rejection Penalty<br/>Dynamic Reward Conditioning & Hard Negative Penalty"]
E --> F["Inference: Summarize-then-Rank Pipeline<br/>Sliding Window Listwise Reranking over Summaries"]
Key Designs¶
1. Cold-Start Supervised Fine-Tuning: Overcoming Initial Policy Collapse in Unconstrained RL
Directly initiating reinforcement learning from a general-purpose base model over listwise ranking metrics is prone to severe instability, high variance, and mode collapse because the unconstrained generative action space is vast. To provide a robust optimization baseline, SumRank first distills the summarization capabilities of an authoritative teacher model (Qwen2.5-72B-Instruct) into lightweight student models (Qwen2.5-3B and 7B). A standardized prompt template instructs the teacher to either extract verbatim/faithful answers when relevance is present or output an exact refusal string when irrelevant. The student model is trained on 40,000 MS MARCO instances by minimizing token-level autoregressive cross-entropy loss:
This cold-start foundation ensures that the model internalizes basic document compression syntax, faithful extraction constraints, and refusal rules, establishing a solid policy initialization before encountering non-differentiable downstream ranking metrics.
2. Static Background Reference Frame: Eliminating Listwise Multi-Agent Noise and Scaling Overhead
In listwise ranking RL, evaluating an entire group of \(G\) sampled rollout summaries across an \(N\)-document candidate list would naively require generating \(N \times G\) summaries at every optimization step, creating prohibitive computational bottlenecks. More critically, because listwise ranking measures relative preference across items, allowing all \(N\) summaries to vary simultaneously introduces environmental non-stationarity: an improvement in relative rank could stem from background summary degradation rather than the target summary's intrinsic quality. To address this, SumRank freezes a static background summary list \(L_S = \{s_1, \dots, s_N\}\) generated once by \(\pi_{\text{SFT}}\) for each query group. During RL, only one target document \(D_t\) is randomly chosen, and the policy samples \(G\) candidate rollouts \(Y = \{y_1, \dots, y_G\}\). To evaluate rollout \(y\), a hybrid evaluation list \(\hat{L}_S = \{s_1, \dots, s_{t-1}, y, s_{t+1}, \dots, s_N\}\) is assembled by swapping only position \(t\). This clever design reduces per-step generative complexity from \(O(N \times G)\) to \(O(G)\) while establishing an invariant reference baseline that isolates target rollout quality.
3. Rank-Driven GRPO with Dynamic Rejection Rewards: Aligning Summary Generation with Listwise NDCG
To optimize the policy model directly against IR evaluation objectives without maintaining an expensive Critic network, SumRank leverages Group Relative Policy Optimization (GRPO). The hybrid candidate list \(\hat{L}_S\) is fed into the downstream LLM reranker (Qwen2.5-32B-Instruct), and the resulting listwise NDCG@10 is computed as the base ranking reward \(R(y)\). To prevent false-positive alignment where the model hallucinates query relevance for irrelevant documents to artificially boost rank, a lexical matching indicator \(M(y) \in \{0, 1\}\) checks whether the summary contains the safe phrase "No Relevant Information Found". The composite reward \(r(y)\) dynamically conditions on ground-truth target label \(D_t\):
where \(\lambda = 0.25\) penalizes failing to reject a true negative. The rewards are normalized across the group of \(G\) rollouts using Z-score standardization \(\hat{A}_i = (r_i - \mu_Y) / \sigma_Y\), directly driving the policy to preserve sharp relevance signals for positive documents and produce minimal refusal tokens for negatives.
Loss & Training¶
The active policy \(\pi_\theta\) is updated by maximizing the clipped surrogate GRPO objective anchored against the cold-start reference policy \(\pi_{\text{SFT}}\):
where \(\rho_{i,t}(\theta) = \frac{\pi_\theta(y_{i,t} \mid X, y_{i,<t})}{\pi_{\text{old}}(y_{i,t} \mid X, y_{i,<t})}\) represents the importance sampling ratio, and the KL divergence penalty coefficient is set to \(\beta = 0.001\) to prevent policy drift and mode collapse. Training is executed using the verl framework across 2,500 MS MARCO query groups (candidate size \(N=10\), positive/negative injection parameter \(k=1\)). The policy samples \(G=8\) rollouts per prompt, trained for 5 epochs with a learning rate of \(1 \times 10^{-6}\) on NVIDIA A800 GPUs.
Key Experimental Results¶
Main Results¶
Experiments are evaluated on both In-Domain (ID) benchmarks from TREC Deep Learning (DL19 through DL23, 5 datasets) and Out-of-Domain (OOD) benchmarks from BEIR (COVID, News, NFCorpus, Robust04, SciFact, 5 datasets). All methods rerank the top-100 candidates retrieved by BM25. The downstream listwise reranker is Qwen2.5-32B-Instruct with sliding window \((w=20, sz=10)\), evaluated via NDCG@10.
| Model Category | Method | DL19 | DL20 | DL21 | DL22 | DL23 | TREC ID Avg. | BEIR OOD Avg. |
|---|---|---|---|---|---|---|---|---|
| Initial Retrieval | BM25 | 51.76 | 52.86 | 51.16 | 29.93 | 29.46 | 43.03 | 48.27 |
| Rule-Based Truncation | FirstP-128 | 64.41 | 61.60 | 66.84 | 40.82 | 42.39 | 55.23 | 53.41 |
| Rule-Based Truncation | FirstP-256 | 65.22 | 61.07 | 67.56 | 40.66 | 41.58 | 55.15 | 53.63 |
| Rule-Based Truncation | FirstP-512 | 65.94 | 61.25 | 67.20 | 39.82 | 41.14 | 55.07 | 53.69 |
| Rule-Based Chunking | MaxP | 65.59 | 62.71 | 66.71 | 39.81 | 38.96 | 54.76 | 54.06 |
| Seq2Seq Summarizers | BART-summarizer | 62.05 | 61.48 | 67.53 | 40.07 | 41.66 | 54.96 | 51.32 |
| Seq2Seq Summarizers | LED-summarizer | 59.80 | 57.82 | 67.22 | 40.18 | 41.36 | 53.28 | 52.27 |
| Seq2Seq Summarizers | PEGASUS | 62.02 | 60.83 | 67.53 | 40.46 | 41.84 | 54.54 | 52.87 |
| Zero-Shot LLM Summarizers | Qwen2.5-3B-Instruct | 58.97 | 54.45 | 63.67 | 37.17 | 35.52 | 49.96 | 54.32 |
| Zero-Shot LLM Summarizers | Qwen2.5-7B-Instruct | 59.72 | 62.03 | 66.15 | 35.40 | 40.24 | 52.71 | 54.97 |
| Proposed Framework | SumRank (3B) | 67.06 | 62.99 | 68.09 | 42.22 | 43.76 | 56.82 | 58.01 |
| Proposed Framework | SumRank (7B) | 67.30 | 62.95 | 69.30 | 42.74 | 44.31 | 57.32 | 59.00 |
Ablation Study¶
An ablation study on training pipelines and data construction strategies was conducted using SumRank (3B) across all five TREC DL benchmarks (DL19–DL23), reporting TREC Average NDCG@10 and absolute degradation \(\Delta\):
| Category | Model Variant / Strategy | TREC Avg. NDCG@10 | Performance Drop (\(\Delta\)) | Analysis & Key Takeaway |
|---|---|---|---|---|
| Full Model | SumRank (3B) Full Pipeline | 56.82 | - | Complete 3-stage training pipeline with static background |
| Training Approach | • w/o Cold-Start SFT | 55.88 | -0.94 | Lacks prior format and refusal distillation; unstable RL exploration |
| Training Approach | • w/o Rank-Driven GRPO | 54.92 | -1.90 | Largest degradation; SFT alone reverts to generic summaries lacking rank signals |
| Data Construction | • Truncation FirstP128 | 54.88 | -1.94 | Truncated background injects lead-bias noise into evaluation list |
| Data Construction | • Truncation FirstP256 | 54.92 | -1.90 | Increasing truncation length fails to alleviate noise in background ranking context |
| Data Construction | • Update per epoch | 56.32 | -0.50 | Dynamic background introduces non-stationary shifting baselines during RL |
Key Findings¶
- Rank-driven GRPO alignment is the single most vital component: Removing the GRPO alignment stage induces the steepest performance drop (-1.90, falling to 54.92). While SFT teaches the model length and basic formatting, it cannot guide the model to favor discriminative ranking tokens. GRPO directly optimizes downstream listwise permutations, preventing summaries from degrading into unhelpful generic overviews.
- Static background contexts provide superior stability over dynamic updates: Freezing the non-target summaries using \(\pi_{\text{SFT}}\) not only reduces generative rollouts from \(O(N \times G)\) to \(O(G)\), but also outperforms updating background summaries each epoch (56.82 vs. 56.32). The static background eliminates environmental noise, ensuring advantage values \(\hat{A}_i\) strictly reflect improvements made by the candidate rollout.
- Small aligned models decisively outshine larger unaligned models: SumRank (3B) achieves 56.82 on TREC DL and 58.01 on BEIR, outperforming zero-shot Qwen2.5-3B (49.96 / 54.32) by large margins and substantially exceeding the 2x larger zero-shot Qwen2.5-7B (52.71 / 54.97). Task-specific downstream alignment proves vastly more effective than mere model parameter scaling.
- Dramatically reduced end-to-end inference latency: Raw long documents on TREC DL average 4,100 to 6,500 tokens, imposing massive latency bottlenecks on self-attention rankers. SumRank compresses relevant documents to ~100–180 tokens and filters irrelevant ones with the 4-token refusal phrase (reducing the overall average candidate length to 12–25 tokens). Consequently, end-to-end latency is significantly faster than both naive truncation (FirstP-512) and Seq2Seq summarizers.
Highlights & Insights¶
- Direct downstream listwise metric alignment: Instead of relying on proxy semantic overlap metrics like ROUGE or BLEU, SumRank turns downstream ranking NDCG directly into a reinforcement learning reward signal, ensuring summaries are explicitly optimized for candidate discriminability.
- Static reference frame formulation: Solves the core tractability and variance bottlenecks of listwise multi-document RL by fixing non-target documents to an established baseline, offering an elegant blueprint for multi-document collaborative LLM optimization.
- Self-regulating negative rejection mechanism: Explicitly penalizing unflagged negative documents drives the summarizer to autonomously detect and discard irrelevant content, simultaneously pruning noisy distractor evidence and shrinking downstream prompt tokens.
Limitations & Future Work¶
- Exploration constrained to lightweight scales: Due to academic computational limits, experiments were evaluated only on 3B and 7B models. Exploring the alignment behavior and capacity scaling of 14B or 32B models remains an open direction.
- Two-stage deployment complexity: The pipeline still relies on separate summarization and ranking models, requiring distinct checkpoints and memory allocations in production serving environments.
- Future direction: Developing an end-to-end unified architecture where a single model jointly compresses document context and produces calibrated listwise permutations in a unified forward pass.
Related Work & Insights¶
- vs. Heuristic Truncation & Pooling (FirstP / MaxP / PARADE): Truncation relies heavily on document lead bias and fails catastrophically on datasets like SciFact and COVID where critical evidence resides in the middle or end; MaxP incurs high computational costs and disrupts global narrative coherence. SumRank compresses whole documents globally, delivering superior out-of-domain generalizability.
- vs. Seq2Seq Summarization (BART / LED / PEGASUS): Traditional summarizers prioritize linguistic fluidity and general human summarization, smoothing away vital keywords and numerical evidence. SumRank's rank-driven alignment preserves exact matching signals and decisive context.
- vs. Zero-Shot Foundational LLM Summarization: Generic LLM summarization suffers from excessive verbosity and frequent hallucinations on negative documents. SumRank enforces tight query-conditioned brevity and explicit rejection boundaries.
Rating¶
- Novelty: ⭐⭐⭐⭐☆ (Pioneers listwise ranking reward alignment for document summarization with a tractable static background RL formulation)
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ (Evaluated across 5 TREC DL benchmarks and 5 diverse BEIR datasets, supplemented by full ablation, latency breakdown, and rejection audits)
- Writing Quality: ⭐⭐⭐⭐⭐ (Exceptionally well-structured paper with rigorous formulations, clear motivations, and detailed qualitative case studies)
- Value: ⭐⭐⭐⭐⭐ (Highly practical for enterprise web search, long-context RAG, and deep research agents facing context window and latency limits)