Skip to content

content_hash: 1496615670a57ca1

Semantic-Anchored Evidential Fusion for Domain-Robust Whole-Slide Survival Analysis

Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/YuchengXing99/SAEFS
Area: Medical Imaging
Keywords: Computational Pathology, Whole-Slide Images, Survival Analysis, Evidential Deep Learning, Cross-Center Domain Shift

TL;DR

To tackle catastrophic performance degradation in whole-slide image (WSI) survival analysis caused by cross-center staining and scanner domain shifts, SAEFS extracts domain-invariant semantic anchors via template-structured VQA to guide visual feature extraction and fuses multimodal evidence via Dirichlet subjective logic and a cautious conjunction rule, achieving state-of-the-art zero-shot cross-center prognosis and reliable calibration.

Background & Motivation

Whole-slide images (WSIs) are gigapixel digital scans of pathology tissue sections capturing cellular morphology, tissue architecture, and the tumor microenvironment. Weakly supervised survival analysis grounded in multiple instance learning (MIL) has become the dominant paradigm for clinical prognosis and personalized treatment planning. However, existing survival algorithms are overwhelmingly trained and validated on single-center cohorts or pooled datasets, obscuring severe cross-center domain shifts. In actual multi-center clinical deployments, staining protocols, scanner optics, and laboratory preparation pipelines vary considerably across healthcare centers. As a result, state-of-the-art MIL models (such as TransMIL) trained on The Cancer Genome Atlas (TCGA) experience an average C-index drop of roughly 0.19 when evaluated zero-shot on external cohorts like the National Lung Screening Trial (NLST), creating a severe bottleneck for clinical translation.

Conventional interventions against domain shifts primarily rely on image-level stain normalization or domain adaptation. However, color space transformations fail to resolve deep architectural variations and scanner-level optical distortions, while domain adaptation (whether supervised or source-free) fundamentally requires access to batches of target-domain data during training or test-time adaptation. In clinical reality, cross-center slides arrive asynchronously, individually, and under strict privacy barriers that prohibit collecting target cohorts in advance. The root cause of this generalization breakdown is feature entanglement: standard visual encoders inevitably entangle prognostically meaningful morphology with center-specific confounders like stain intensity and color temperature profiles. Indeed, empirical studies confirm that current pathology foundation models encode center-of-origin signatures even more strongly than tumor biological signals. In contrast, human pathologists generalize effortlessly across hospitals because they reason using high-level pathology semantics—such as tumor grade, necrosis extent, cellular atypia, and immune infiltration—which remain inherently invariant across clinical centers. Nevertheless, manual slide-level semantic annotations from expert pathologists are prohibitively expensive and cannot scale.

This work asks whether a survival framework can automatically derive domain-invariant semantic anchors to overcome cross-center distribution shift without requiring any target-domain data. The core idea is to extract domain-invariant semantic anchors via template-structured closed-form VQA, construct complementary semantic-guided and global WSI evidence streams, and fuse visual and semantic evidence via Dirichlet subjective logic coupled with a cautious conjunction rule that accounts for inter-modal correlation and prevents overconfident predictions.

Method

Overall Architecture

The SAEFS architecture consists of four tightly integrated stages: semantic anchor generation, dual-stream WSI visual evidence extraction, text semantic evidence extraction, and cautious belief fusion for discrete survival prediction. Given a patient's WSI, a pathology-specialized VQA model is prompted offline with 12 closed-form clinical questions to generate structured, hallucination-resistant semantic descriptions. A vision-language model (VLM) text encoder encodes these descriptions into semantic anchor embeddings, while the patch encoder extracts visual bag features from the tissue tiles. The dual-stream visual module derives both a text-guided local attention feature and an unconditioned global attention feature, mixing them into a unified visual evidence vector. Concurrently, the text anchor representations are directly projected into a semantic evidence vector. Both evidence sources are mapped to Dirichlet opinions (belief masses and epistemic uncertainty) via Subjective Logic, and then integrated using the cautious conjunction rule to preserve uncertainty under domain shift and inter-modal correlation.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: Patient Whole-Slide Image (WSI)"] --> B["Template-based VQA Semantic Anchor Generation<br/>12 closed-form clinical questions for structured text"]
    A --> C["WSI Patch Feature Extraction<br/>Foundation model extracts N patch embeddings"]
    B --> D["Semantic Text Embedding Encoding<br/>VLM text encoder produces Q anchor vectors"]

    C --> E["Dual-Stream WSI Evidence Extraction<br/>Semantic-guided patch retrieval + global attention pooling"]
    D --> E
    D --> F["Direct Text Evidence Projection<br/>Mean pooling followed by Softplus to K intervals"]

    E --> G["Adaptive Evidence Mixing<br/>Mixing coefficient λ balances global and guided visual evidence"]

    G --> H["Subjective Logic Belief & Uncertainty Mapping<br/>Dirichlet parameters map to opinions (b, u)"]
    F --> H

    H --> I["Cautious Belief Fusion<br/>Commonality minimization to prevent correlated overconfidence"]
    I --> J["Output: Calibrated Survival Probabilities & Uncertainty"]

Key Designs

1. Template-based Closed-Form VQA Semantic Anchors: Grounding Domain-Invariant Semantics

Standard end-to-end vision models easily overfit source-domain stain and scanner artifacts, whereas high-level pathology concepts are naturally stable across clinical centers. However, unconstrained LLM/VLM generation frequently produces hallucinations and large stylistic variations. SAEFS introduces a structured semantic space using \(Q=12\) clinically grounded, multiple-choice questions designed via GPT (probing cellular atypia, histologic grade, necrosis extent, stromal architecture, and inflammatory infiltration; e.g., "What is the level of cellular atypia in this slide? A. Mild... B. Moderate... C. Severe... D. Not assessable"). A pathology-specialized VQA model selects from predefined choices, yielding a bounded, hallucination-resistant semantic profile \(T_i = [T_i^1, \dots, T_i^Q]\) encoded by the VLM text encoder into embeddings \(P_i \in \mathbb{R}^{Q \times D}\). Quantitative analysis demonstrates that the cross-center Maximum Mean Discrepancy (MMD) of these VQA semantic features drops by 73% compared to raw pixel-derived WSI features, establishing a robust anchor for cross-center transfer.

2. Complementary Dual-Stream Visual Evidence Extraction and Adaptive Mixing: Balancing Guided Focus and Global Context

Relying solely on text semantics discards fine-grained microscopic histology and spatial microenvironmental context, while relying solely on unguided MIL leaves attention weights vulnerable to out-of-domain staining artifacts. SAEFS addresses this trade-off via a dual-stream visual evidence extractor over patch feature bag \(X = [\mathbf{x}_1, \dots, \mathbf{x}_N]^\top \in \mathbb{R}^{N \times D}\). In the text-guided stream, each semantic anchor embedding \(\mathbf{p}^q\) acts as a query to retrieve prognostically relevant tissue patches via temperature-scaled cosine similarity matching: $\(\mathbf{f}_{tg}^q = \sum_{i=1}^N \frac{\exp(\alpha \cdot \text{sim}(\mathbf{p}^q, \mathbf{x}_i))}{\sum_{n=1}^N \exp(\alpha \cdot \text{sim}(\mathbf{p}^q, \mathbf{x}_n))} \mathbf{x}_i\)$ Averaging across all \(Q\) anchors followed by linear projection and Softplus activation yields guided evidence \(\mathbf{e}_{tg} \in \mathbb{R}_{\ge 0}^K\). Concurrently, an unconditioned global attention branch pools patch features into \(\mathbf{f}^{wo}\) and projects them into global visual evidence \(\mathbf{e}^{wo} \in \mathbb{R}_{\ge 0}^K\). The combined visual evidence is formed adaptively using mixing parameter \(\lambda \in [0, 1]\): \(\mathbf{e}_{\text{path}} = (1-\lambda)\mathbf{e}^{wo} + \lambda\mathbf{e}_{tg}\), successfully focusing on clinically meaningful morphology while preserving holistic slide context.

3. Direct Text Semantic Evidence Projection: Providing a Domain-Stable Diagnostic Baseline

Beyond directing visual feature pooling, the high-level semantic descriptions themselves provide an autonomous prognostic signal. SAEFS mean-pools the \(Q\) semantic anchor embeddings and maps them through a dedicated linear layer with Softplus activation: $\(\mathbf{e}^{\text{text}} = \text{Softplus}\left( \mathbf{W}^{\text{text}} \left( \frac{1}{Q} \sum_{q=1}^Q \mathbf{p}^q \right) + \mathbf{b}^{\text{text}} \right) \in \mathbb{R}_{\ge 0}^K\)$ This text evidence stream functions as a pure semantic prior that is entirely decoupled from patch-level pixel statistics, providing a dependable baseline when slide scanning quality degrades or color distributions shift severely.

4. Cautious Belief Fusion: Mitigating Inter-Modal Correlation and Overconfidence

The visual evidence \(\mathbf{e}_{\text{path}}\) and semantic evidence \(\mathbf{e}^{\text{text}}\) parameterize Dirichlet distributions \(\boldsymbol{\alpha} = \mathbf{e} + 1\), which map under Subjective Logic into belief masses \(b_k = e_k / S\) and epistemic uncertainty \(u = K / S\) (where Dirichlet strength \(S = \sum_{k=1}^K \alpha_k\), satisfying \(\sum_{k=1}^K b_k + u = 1\)). Because the visual guided stream is steered by the semantic anchors, the two modalities are intrinsically dependent. Conventional fusion rules (such as Dempster's rule) assume source independence, causing double-counting of correlated signals and severe overconfidence on incorrect intervals. SAEFS employs the cautious conjunction rule operating on commonality functions \(q_k = b_k + u\): $\(q_k^{\text{fused}} = \min(q_k^{\text{path}}, q_k^{\text{text}}), \qquad u^{\text{fused}} = \min(u_{\text{path}}, u_{\text{text}})\)$ Unnormalized beliefs are recovered as \(\tilde{b}_k^{\text{fused}} = \max(q_k^{\text{fused}} - u^{\text{fused}}, 0)\) and normalized by \(Z = \sum_j \tilde{b}_j^{\text{fused}} + u^{\text{fused}}\). By taking the minimum over commonality, cautious fusion enforces uncertainty preservation: when either modality exhibits high uncertainty or when modal conflict occurs, the fusion preserves conservative uncertainty rather than allowing overconfident predictions to dominate.

Loss & Training

Discrete survival analysis partitions survival time into \(K\) intervals. For sample \(i\) with discrete interval \(y_i \in \{0, \dots, K-1\}\) and censorship indicator \(c_i \in \{0, 1\}\) (\(c_i=1\) denoting right-censoring), the fused Dirichlet parameters \(\alpha_k^{\text{fused}} = b_k^{\text{fused}} \cdot \frac{K}{u^{\text{fused}} + \epsilon}\) define interval probability \(p_k^{(i)} = \alpha_k^{(i)} / S^{(i)}\). The total objective combines the survival negative log-likelihood with a non-event Dirichlet KL regularizer: $\(\mathcal{L} = \mathcal{L}_{\text{surv}} + \lambda_t \cdot \mathcal{L}_{\text{KL}}\)$ The survival loss optimizes the event interval hazard for uncensored patients and the cumulative survival probability beyond \(y_i\) for censored patients: $\(\mathcal{L}_{\text{surv}} = -\sum_{i=1}^N \left( (1 - c_i) \log p_{y_i}^{(i)} + c_i \log \sum_{k > y_i} p_k^{(i)} \right)\)$ The regularizer \(\mathcal{L}_{\text{KL}} = \sum_{i=1}^N \text{KL}(\text{Dir}(\tilde{\boldsymbol{\alpha}}^{(i)}) \parallel \text{Dir}(\mathbf{1}))\) shrinks evidence for non-ground-truth intervals toward a uniform Dirichlet prior, where \(\tilde{\boldsymbol{\alpha}}\) excludes ground-truth interval evidence. The weight \(\lambda_t = \min(1, t / T_{\text{anneal}})\) anneals linearly from 0 to 1 over \(T_{\text{anneal}}\) steps, allowing unobstructed evidence discovery early in training.

Key Experimental Results

Main Results

The framework is trained exclusively on TCGA (spanning LUAD, UCEC, and KIRC) and evaluated zero-shot across four external cohorts from independent medical centers: CPTAC-LUAD, CPTAC-UCEC, CPTAC-KIRC, and NLST-LUAD. Evaluation metrics include concordance index (C-index, higher is better), Integrated Brier Score (IBS, lower is better), and Integrated Negative Binomial Log-Likelihood (INBLL, lower is better).

Method Type CPTAC-LUAD (C↑ / IBS↓) CPTAC-UCEC (C↑ / IBS↓) CPTAC-KIRC (C↑ / IBS↓) NLST-LUAD (C↑ / IBS↓) Avg C-index ↑ Avg IBS ↓ Avg INBLL ↓
ABMIL Unimodal Visual 0.464 / 0.702 0.534 / 0.685 0.600 / 0.814 0.535 / 0.706 0.533 0.727 2.617
TransMIL Unimodal Visual 0.500 / 0.946 0.636 / 0.833 0.622 / 0.868 0.516 / 0.908 0.568 0.889 5.136
DSMIL Unimodal Visual 0.469 / 0.613 0.607 / 0.662 0.637 / 0.772 0.534 / 0.525 0.562 0.643 1.920
ILRA Unimodal Visual 0.519 / 0.974 0.638 / 0.877 0.616 / 0.904 0.596 / 0.865 0.593 0.905 6.567
OTSurv Unimodal Visual 0.529 / 0.570 0.612 / 0.702 0.652 / 0.724 0.644 / 0.523 0.609 0.630 1.721
MCAT Multimodal Concat 0.440 / 0.286 0.602 / 0.214 0.591 / 0.217 0.460 / 0.266 0.523 0.246 0.828
SurvPath Multimodal Concat 0.426 / 0.289 0.572 / 0.220 0.627 / 0.213 0.431 / 0.279 0.514 0.250 0.836
PS3 Multimodal Concat 0.455 / 0.287 0.666 / 0.218 0.596 / 0.229 0.529 / 0.254 0.562 0.247 0.993
SAEFS (Ours) Evidential Fusion 0.663 / 0.376 0.682 / 0.270 0.677 / 0.272 0.662 / 0.278 0.671 0.299 0.755

Ablation Study

Ablation analysis averaged across the four unseen external cohorts under zero-shot transfer from TCGA (\(\Delta\) computed relative to the full SAEFS model):

Variant Configuration C-index ↑ (\(\Delta\)) IBS ↓ (\(\Delta\)) INBLL ↓ (\(\Delta\)) Note
A1 WSI-only (\(\lambda = 0\)) 0.5980 (-0.0860) 0.2933 (-0.0098) 0.7452 (-0.0196) Removing text guidance leads to significant drop under domain shift
A2 TG-WSI only (\(\lambda = 1\)) 0.6408 (-0.0432) 0.2964 (-0.0067) 0.7490 (-0.0158) Confirms global visual context provides complementary information
A3 Text-only 0.5327 (-0.1513) 0.3769 (+0.0738) 0.9417 (+0.1769) High-level semantics alone lack fine-grained morphological grounding
A4 Concatenation fusion 0.5310 (-0.1530) 0.2119 (-0.0912) 0.7551 (-0.0098) Naive fusion double-counts correlated evidence, yielding severe miscalibration
A5 Dempster fusion rule 0.5758 (-0.1082) 0.3113 (+0.0082) 0.7824 (+0.0176) Violated independence assumption degrades cross-center transfer
Full SAEFS (Full Model) 0.6840 (—) 0.3031 (—) 0.7648 (—) Best trade-off between discrimination and probability calibration

Key Findings

  • Pronounced Advantages Under Severe Distribution Shift: On CPTAC-LUAD, where domain shift is most severe, pure visual baselines plateau below 0.529 C-index, whereas SAEFS achieves 0.663 (+0.134 improvement over the strongest unimodal baseline). Performance margins expand as domain discrepancy widens.
  • Cautious Fusion Prevents Misleading Overconfidence: Although concatenation fusion (A4) yields an artificially low IBS (0.2119), its C-index plummets to 0.5310, and calibration curves reveal severe deviation from observed outcomes. Cautious fusion explicitly preserves uncertainty under modal correlation, providing the best INBLL (0.755) and faithful calibration.
  • Lightweight Architecture: SAEFS requires only 270K trainable parameters, roughly 90% fewer than TransMIL (2.41M) and MCAT (2.74M). Offline caching of VQA answers ensures that neither training nor inference incurs heavy runtime overhead.

Highlights & Insights

  • Paradigm Shift from Pixel Statistics to Pathology Semantics: Quantitative MMD analysis provides empirical proof: while raw WSI features exhibit high cross-center divergence (MMD = 0.1988), VQA semantic features exhibit an MMD of only 0.0532 (a 73% reduction), mimicking human pathologists who transfer expertise via invariant clinical concepts.
  • Principled Fusion for Correlated Multimodal Evidence: Rather than naively assuming modal independence (as in Dempster-Shafer theory), SAEFS integrates cautious conjunction over commonality functions to handle inter-modal dependencies and avoid overconfidence under domain shift.
  • Target-Free Zero-Shot Deployment: Unlike UDA or test-time adaptation methods that require target-domain slide batches, SAEFS operates zero-shot on isolated individual slides, making it immediately viable for privacy-restricted, asynchronous clinical diagnostic workflows.

Limitations & Future Work

  • Static Template Expressiveness: The fixed set of 12 closed-form questions, while effective at bounding the semantic space and preventing hallucinations, may lack the expressive granularity required to capture rare morphological variants or unconventional invasion patterns. Exploring dynamic, open-ended VLM probing is an important next step.
  • Scope of Cancer Types: Validation is currently focused on lung adenocarcinoma (LUAD), uterine corpus endometrial carcinoma (UCEC), and kidney renal clear cell carcinoma (KIRC). Expanding evaluations to more structurally heterogeneous malignancies like sarcomas and lymphomas remains open.
  • Direct Semantic Confidence Modeling: Future extensions could leverage VLMs with calibrated token-level confidence scores to directly parameterize Subjective Logic priors.
  • vs Pure Visual MIL (TransMIL, ILRA, OTSurv): Unimodal models fail to disentangle prognostic morphology from scanner and staining variations, suffering severe C-index drops (up to 0.19) across centers. SAEFS anchors feature pooling with domain-invariant semantics, sustaining robust cross-center discrimination.
  • vs Multimodal Survival Models (MCAT, SurvPath, PS3): Traditional multimodal methods combine modalities via cross-attention or vector concatenation, ignoring inter-modal correlation and generating overconfident, poorly calibrated survival estimates under domain shift. SAEFS explicitly models reliability and epistemic uncertainty via cautious belief fusion.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ First systematic formulation of semantic-anchored survival analysis with cautious belief fusion to overcome histopathology domain shift.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Zero-shot evaluation across four external cohorts from independent medical centers, supported by MMD divergence analysis, KM survival stratification, and calibration curves.
  • Writing Quality: ⭐⭐⭐⭐⭐ Clear motivation, rigorous mathematical formulation of evidential subjective logic, and insightful ablation studies.
  • Value: ⭐⭐⭐⭐⭐ Offers a practical, target-free blueprint for deploying AI computational pathology across diverse clinical centers.