Query Pipeline Optimization for Cancer Patient Question Answering Systems¶

Conference: ACL 2026 arXiv: 2412.14751 Code: None Area: Medical Imaging Keywords: Cancer QA, RAG Query Pipeline, Hybrid Retrieval, Semantic Segmentation, Metadata-Aware

TL;DR¶

This paper proposes CoMeta, a three-tier controllable metadata-aware RAG framework for Cancer Patient Question Answering (CPQA). It integrates Clinical Hybrid Semantic-symbolic Document Retrieval (CHSDR), which fuses real-time Boolean search via E-Utilities with MedCPT semantic retrieval, and employs Semantically Enhanced Overlapping Segmentation (SEOS) to prevent context fragmentation. On the CMMQA dataset, CoMeta improves Claude-3-Haiku answer accuracy by 5.24% over CoT and approximately 3% over naive RAG.

Background & Motivation¶

Background: LLMs have shown promise in medical question answering, but hallucination poses risks to patient safety. RAG mitigates hallucination by grounding outputs in external evidence. Existing medical RAG systems predominantly adopt dense retrieval paradigms, using domain-specific embedding models (e.g., MedCPT) for vector similarity search over offline indices. Advanced strategies such as hybrid search, adaptive retrieval, and recursive search are essentially optimizations over static indices.

Limitations of Prior Work: (1) Staleness–Semantics Dilemma: Standard query pipelines (dense or BM25) operate on static, metadata-blind indices and risk retrieving outdated evidence, while real-time metadata-aware interfaces such as E-Utilities are semantically fragile for informal patient queries. (2) Retrieval Depth Paradox: Review articles benefit from full-text retrieval to capture high-level treatment syntheses, whereas primary studies typically require only abstract retrieval to avoid methodological noise—most pipelines apply uniform retrieval depth across all article types. (3) Context Fragmentation: Encoder-agnostic chunking strategies (fixed-length or lexical) sever the association between clinical qualifiers (e.g., specific mutation criteria) and treatment statements, producing recommendations that appear evidence-supported yet lack critical constraints.

Key Challenge: Existing systems are forced to trade off between semantic robustness (static indices) and retrieval controllability (real-time interfaces), failing to simultaneously satisfy the tripartite requirements of CPQA for timeliness, metadata awareness, and semantic integrity.

Goal: To design a RAG framework specifically tailored for CPQA that enforces controllability along three dimensions: (1) robustness against the staleness–semantics dilemma; (2) metadata-aware adaptive retrieval depth based on publication type; (3) preservation of relational integrity in clinical logic through encoder-aware segmentation.

Key Insight: Rather than further optimizing static-index pipelines, this work integrates E-Utilities as a real-time, metadata-aware sparse backend into the RAG system—a design that is orthogonal and complementary to prior RAG optimizations.

Core Idea: Achieve "symbolic–semantic complementarity" by fusing real-time E-Utilities Boolean search with semantic retrieval, combined with publication-type-adaptive depth calibration and encoder-aware semantic segmentation, to construct an end-to-end controllable cancer QA pipeline.

Method¶

Overall Architecture¶

CoMeta adopts a hierarchical query pipeline design comprising two layers: document-level and passage-level. At the document level, CHSDR performs hybrid retrieval and metadata filtering. At the passage level, SEOS handles semantically aware segmentation followed by two-stage (embedding + reranking) fine-grained retrieval. The overall architecture prioritizes enforcing controllability at every stage of the retrieval lifecycle.

Key Designs¶

Clinical Hybrid Semantic-Symbolic Document Retrieval (CHSDR):
- Function: Overcomes the staleness–semantics dilemma and retrieval depth paradox, enabling robust retrieval across query types (standard vs. clinical narrative).
- Mechanism:
  - Adaptive Boolean Query Execution (Adapt-E): An LLM-based rewriter performs error correction, normalization, intent analysis, clinical abstraction (mapping to PICO elements), Boolean expression generation, and temporal constraint generation on patient queries. The generated queries are executed in decreasing order of strictness (strict Boolean → clinical abstraction → relaxed Boolean) until a sufficient number of documents is retrieved.
  - Hybrid Semantic-Symbolic Retrieval: E-Utilities symbolic search and MedCPT semantic retrieval are fused via Reciprocal Rank Fusion (RRF). Both retrieval streams return PMIDs as unified document keys.
  - Metadata Utilization: Publication types (D1: PubMed abstract / D2: PMC review full-text / D3: non-review PMC paper), publication dates, and abstract availability are parsed from E-Utilities XML to enable adaptive retrieval depth.
- Design Motivation: The real-time Boolean search of E-Utilities provides metadata control and timeliness, while MedCPT semantic retrieval compensates for its fragility on informal queries. The adaptive execution strategy ensures sufficient evidence retrieval for complex queries through progressive relaxation.
Semantically Enhanced Overlapping Segmentation (SEOS):
- Function: Performs semantically aware segmentation of documents prior to passage retrieval to prevent context fragmentation.
- Mechanism: Inspired by TextTiling but incorporating three key innovations: (a) domain-specific dense embeddings replace bag-of-words representations to handle medical terminology and discourse relations; (b) a target token budget determines the optimal number of partitions \(N\), selecting the top-\(N\) semantic minima as breakpoints rather than relying on fragile similarity thresholds; (c) the sentence overlap at each breakpoint is adaptively determined by the degree of semantic continuity, preserving unresolved semantic dependencies. Adjacent chunk identifiers are explicitly stored to enable cross-segment context recovery.
- Design Motivation: Fixed-length chunking severs syntactic dependencies mid-sentence. TextTiling's lexical overlap fails in biomedical literature characterized by high synonymy and complex semantic transitions. SEOS explicitly accounts for the interaction between chunk size and encoder performance.
Publication-Type-Adaptive Retrieval Depth:
- Function: Calibrates retrieval depth according to literature type to resolve the retrieval depth paradox.
- Mechanism: Experimental analysis reveals that the proportion of PMC review articles in Top-5 evidence increases more (0.10 → 0.12) than other PMC papers (0.28 → 0.32). Integrating reviews (D1+D2) improves accuracy from 44.00% to 46.00%, while further adding non-review full texts (D1+D2+D3) maintains accuracy but reduces Precision/Recall/F1. Accordingly, CoMeta calibrates retrieval depth by publication type before passage retrieval: review articles retrieve full text, while primary studies use abstracts only.
- Design Motivation: Review articles synthesize findings across studies, aligning with the broad scope of patient queries. Non-review full texts are typically context-specific, and their core clinical findings can be represented by abstracts; retrieving full texts introduces noise that overwhelms the model.

Loss & Training¶

CoMeta is an inference-time framework and involves no model training. For dataset construction, CMMQA (520 cancer-related questions) is built by filtering HealthSearchQA and MIRAGE benchmarks using MeSH terms, with Llama-3-70B used to rewrite questions into clinical narrative variants. Retrieval evaluation uses PubMedQA and BioASQ (with gold-standard citations); passage retrieval evaluation uses synthetic QA pairs generated from PubMed abstracts, PMC full texts, and medical textbooks.

Key Experimental Results¶

Main Results¶

Overall Performance on CMMQA (Claude-3-Haiku)

Method	MMLU	MedQA	MedMCQA	PMQA	BioASQ	Avg
LLM + CoT	78.26	68.60	65.59	45.00	80.49	67.15
Naive RAG	82.61	67.44	65.59	56.67	81.71	69.48
CoMeta	82.61	69.77	68.82	65.00	81.71	72.39

CHSDR Ablation (Document Retrieval Performance)

Method	BioASQ Hit@10 (Standard)	BioASQ Hit@10 (Narrative)	PubMedQA Hit@10 (Standard)	PubMedQA Hit@10 (Narrative)
E-utils	52.44	1.22	41.67	0.00
Adapt-E	65.85	50.00	48.33	8.33
MedCPT	63.41	41.46	10.00	3.33
Hybrid	80.49	60.98	46.67	10.00

Ablation Study¶

SEOS vs. Fixed Segmentation Strategies (Passage Retrieval Accuracy %)

Segmentation Strategy	PubMedBERT	BM25	MedCPT
512 (Overlap 0)	46	20	22
512 (Overlap 32)	52	18	24
512 (Overlap 128)	42	16	22
SEOS (Ours)	54	36	38

Zero-Hit Failure Rate Comparison

Dataset–Setting	E-utils	Adapt-E (Ours)
PubMedQA – Standard	22/60	0/60
PubMedQA – Narrative	55/60	0/60
BioASQ – Standard	18/82	0/82
BioASQ – Narrative	76/82	0/82

Key Findings¶

CHSDR hybrid retrieval improves Hit@10 on BioASQ from 52.44% (E-utils) to 80.49%, demonstrating that semantic retrieval successfully recalls documents missed by symbolic search.
Adapt-E's adaptive query execution reduces zero-hit failures from 55/60 to 0/60 on the PubMedQA narrative setting, representing a qualitative improvement in retrieval robustness.
SEOS outperforms all fixed segmentation strategies across retrievers; the advantage is most pronounced on BM25 (20% → 36%), indicating that semantically aware segmentation is effective across different retrieval paradigms.
PMC review articles provide substantially higher retrieval value than non-review PMC papers—incorporating reviews yields a 2% accuracy gain, whereas further adding non-review full texts reduces F1.
The average 2.91% accuracy improvement of CoMeta underestimates its actual contribution: an 8.33% gain is observed on PubMedQA where retrieval constitutes the bottleneck, while ceiling effects prevent measurable gains on MMLU/BioASQ where retrieval is already saturated.

Highlights & Insights¶

E-Utilities is repositioned from a traditional Boolean search tool to a real-time, metadata-aware backend for RAG systems—a design paradigm that is orthogonal and complementary to existing RAG optimizations.
The adaptive query execution strategy (strict → relaxed progressive fallback) is a concise yet practically powerful engineering innovation that completely eliminates the zero-hit problem.
The systematic analysis of why average accuracy underestimates the contribution (ceiling effects, retrieval robustness blind spots, evidence timeliness blind spots) demonstrates deep experimental rigor.

Limitations & Future Work¶

Validation is primarily conducted in the cancer QA domain; although the authors argue this is a subset of general medical QA, generalization to other medical subdomains requires further verification.
No comparison is made with emerging advanced semantic segmentation strategies.
The dataset scale (520 questions) is relatively limited and may not capture the full diversity of clinical scenarios.
The framework depends on the real-time availability of NCBI E-Utilities, which may be constrained in certain deployment environments.
Future directions include adaptive retrieval mechanisms (dynamically deciding whether and how to retrieve) and broader backbone model validation.

vs. MedRAG/Self-BioRAG: These systems optimize retrieval strategies over static indices; CoMeta introduces a real-time metadata-aware backend, representing an orthogonal design dimension.
vs. Pure E-Utilities: E-Utilities is semantically fragile for informal queries (55/60 zero-hit failures); CoMeta's LLM rewriter and adaptive execution completely resolve this issue.
vs. TextTiling: TextTiling employs bag-of-words representations and fixed thresholds, which fail in the high-synonymy environment of biomedical literature; SEOS replaces these with dense embeddings and a target budget.

Rating¶

Novelty: ⭐⭐⭐⭐ Integrating E-Utilities as a real-time RAG backend is a novel design paradigm; SEOS represents a meaningful improvement over existing segmentation methods.
Experimental Thoroughness: ⭐⭐⭐⭐ Covers multiple medical QA datasets with detailed ablations and retriever–reranker combination analyses, though dataset scale is limited.
Writing Quality: ⭐⭐⭐⭐ Problem formulation is clear (three dilemmas) and analysis is thorough, though certain sections are somewhat verbose.
Value: ⭐⭐⭐⭐ Provides a practical query pipeline optimization solution for medical RAG with direct reference value for clinical applications.