MOOZY: A Patient-First Foundation Model for Computational Pathology

Conference: ECCV2026
Paper: ECCV Official Page
Code: https://github.com/AtlasAnalyticsLab/MOOZY
Area: Medical Imaging
Keywords: computational pathology, whole-slide representations, patient-level aggregation, masked self-distillation, multi-task learning

TL;DR

MOOZY learns spatial slide representations through masked self-distillation, then jointly trains patient-level representations with a case transformer and 333 clinical tasks, achieving macro-average weighted F1 of 0.769 and balanced accuracy of 0.702 across 16 held-out tasks, while its macro-average AUC remains slightly below TITAN.

Background & Motivation

Pathology foundation models commonly partition a whole-slide image (WSI) into patches, extract patch features, and train a separate multiple instance learning (MIL) aggregator for each downstream task. This captures local cellular morphology but does not directly solve two clinical problems: how distant regions within a slide jointly provide evidence, and how multiple slides from the same patient support a case-level conclusion. Scaling the patch encoder does not automatically teach either relationship.

Recent slide encoders pretrain reusable whole-slide representations, but many depend on private data, paired reports, or molecular data. Even when multiple slides are accepted, common approaches either merge all patches into one large bag or average independently encoded slide embeddings. The former weakens the slide hierarchy, while the latter lacks learnable inter-slide interactions. The patient remains an identifier used to combine results rather than the unit explicitly modeled during training.

MOOZY therefore separates learning into two stages: establish spatial representations using public slides, then align them with clinical semantics using existing classification and survival labels. It requires neither complete labels for every patient nor newly paired reports. Core Idea: first learn to represent individual slides, then let slides from the same patient interact through a case transformer, using sparse multi-task supervision to train reusable case representations.

Method

Overall Architecture

The input is one or more WSIs from a case, and the output is a case embedding. Stage 1 trains slide representations; Stage 2 jointly updates the slide encoder and case transformer. Transfer to a new task freezes the representation model and trains a downstream probe instead of directly reusing the pretraining task heads.

The pipeline comprises spatial feature grids, multi-view masked self-distillation, a case transformer, and sparse multi-task semantic alignment, in that order. The latter two are jointly trained in Stage 2. Task labels provide training supervision only and are unnecessary for extracting case embeddings.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Multiple WSIs<br/>from one patient"] --> Grid["Spatial Feature Grids"]
    Grid -->|Stage 1| SSL["Multi-View Masked<br/>Self-Distillation"]
    SSL -->|Pretrained slide encoder| Case["Case Transformer"]
    Grid -->|Stage 2 whole-slide encoding| Case
    Case --> Align["Sparse Multi-Task<br/>Semantic Alignment"]
    Labels["Existing classification<br/>and survival labels"] -.->|Training supervision only| Align
    Case -->|Representation extraction| Output["Case embedding<br/>and downstream probe"]

Key Designs

1. Spatial Feature Grids: preserve tissue locations and allocate capacity to slide context

Preprocessing uses AtlasPatch and SAM2 to identify tissue, partitions it into non-overlapping 224 x 224 patches, and extracts features at 20x and 40x magnification. The frozen patch encoder is a ViT-S with 21.67M parameters. Features and coordinates are arranged in a two-dimensional grid with a binary mask indicating valid tissue locations. Different magnification grids from the same slide are independent Stage 1 training samples, not inputs to a cross-magnification fusion module.

The slide encoder projects patch features to 768 dimensions through a linear layer and GELU, prepends a CLS token and 4 register tokens, and applies a 6-layer, 12-head transformer. Two-dimensional ALiBi constructs attention biases from coordinate distances; CLS and register tokens remain spatially neutral, and an attention mask excludes background positions. The encoder therefore receives a spatial tissue layout rather than a location-free feature bag. The cached ALiBi equation contains missing characters, so this note describes the mechanism from the readable prose without reconstructing an exact equation.

2. Multi-View Masked Self-Distillation: learn slide representations through cross-region agreement and missing-tissue prediction

Stage 1 independently samples 2 global views of 20 x 20 tokens and 4 local views of 12 x 12 tokens from the full valid tissue grid, rather than deriving all views from one fixed region. A minimum valid-tissue ratio prevents mostly empty crops. Contiguous rectangular blocks mask global views at ratios sampled uniformly from 0.1 to 0.5; local views do not receive this masking. Hiding continuous tissue regions encourages reliance on broader morphological context.

The student processes the different views, while an exponential moving average teacher supplies soft targets from global views. CLS self-distillation aligns global prototype distributions across views; masked patch prediction matches teacher distributions at masked valid tissue positions. Both use projection heads with a normalized bottleneck and prototype layer, and teacher-output centering discourages collapse. The target is a teacher feature distribution, not raw pixels. Cross-view agreement and local missing-region prediction jointly shape the whole-slide representation.

3. Case Transformer: learn relationships between slides at the patient level

Stage 2 uses full-slide feature grids instead of the cropped views used in Stage 1. When the valid-token count exceeds the hardware-dependent limit, stratified random sampling preserves whole-slide coverage before valid tokens are compacted and passed to the slide encoder. Each slide produces a 768-dimensional vector. The case aggregator therefore processes a small collection of slide vectors rather than concatenating every patch from every slide into one sequence.

A learnable CASE token is prepended to these slide vectors and processed by a 3-layer, 12-head transformer with LayerScale. The final normalized CASE representation becomes the case embedding. Attention allows each slide to be influenced by other slides, unlike fixed mean pooling. Single-slide cases also pass through this module to maintain a consistent representation space. These are slide-level relationships; the method does not imply that corresponding tissue regions across slides have been registered.

4. Sparse Multi-Task Semantic Alignment: train shared case representations with incomplete clinical labels

The case embedding feeds task-specific linear or MLP heads for 205 classification and 128 survival tasks. Classification uses inverse-frequency class weights and label smoothing. Survival tasks cover overall survival (OS), disease-specific survival (DSS), disease-free interval (DFI), and progression-free interval (PFI), using a discrete-time hazard model for events and censored observations. Time-bin boundaries follow training event-time quantiles, with an event-count-dependent bin count targeting 8 and ranging from 2 to 16.

Public cohorts have incomplete annotations, so losses are computed only for cases and tasks with valid supervision and then averaged across active tasks in the batch. The method does not fabricate missing labels; different clinical objectives update a shared representation. After pretraining, the task heads are discarded while the slide encoder and case transformer are retained. Thus, not requiring paired reports does not mean the entire pipeline is unsupervised: Stage 2 explicitly depends on existing clinical labels.

A Worked Example

Consider a case with two WSIs, one available classification label, and no usable survival outcome. This is an illustrative data-flow example, not an additional experimental case from the paper.

Each slide undergoes tissue segmentation, patch encoding, and spatial-grid construction, with token capping applied independently when needed. The shared slide encoder produces two 768-dimensional vectors. The case transformer processes the CASE token together with these vectors to form one patient-level representation.

During training, this case contributes supervision only to its known classification task, not to the missing survival task. During transfer, the same case-aggregation pathway is retained, but a target-task probe is trained on frozen case embeddings instead of reusing the pretraining classification head. A single-slide case follows the same pathway with fewer slide vectors.

Loss & Training

The two Stage 1 objectives are added directly, with no additional weighting introduced in the main text:

\[ \mathcal{L}_{\mathrm{stage1}} = \mathcal{L}_{\mathrm{cls}} + \mathcal{L}_{\mathrm{mim}}. \]

The CLS term is cross-entropy between teacher and student view distributions, not a clinical-label classification loss. Teacher momentum follows a cosine schedule from 0.996 to 1.0. Training uses AdamW, 8 GPUs, an effective batch of 1,024 slides, and 200 epochs, with approximately 436 GPU-hours reported.

Stage 2 averages losses over tasks with usable supervision in the current batch:

\[ \mathcal{L}_{\mathrm{stage2}} = \frac{1}{|\mathcal{T}_{\mathrm{active}}|}\sum_{t\in\mathcal{T}_{\mathrm{active}}}\mathcal{L}^{(t)}. \]

Stage 2 uses 8 GPUs, an effective batch of 1,024 cases, and 20 epochs, taking approximately 512 GPU-hours. The base learning rate is \(5\times10^{-5}\), gradient clipping is 0.3, and label smoothing is 0.03. A task-wise stratified 5% validation holdout monitors overfitting. The cached survival-risk aggregation equation is damaged, and the full discrete-hazard negative log-likelihood is deferred to an appendix absent from the cache, so neither is reconstructed here.

Data counts require care: 77,134 denotes Stage 1 feature grids, comprising 53,286 at 20x and 23,848 at 40x, not 77,134 patients. The Stage 2 labeled subset contains 41,089 cases and 45,179 unique WSIs, with tasks drawn from 56 public datasets. Of the 333 tasks, 56 are slide-level and 277 are case-level.

Key Experimental Results

Main Results

The paper evaluates frozen representations on 16 held-out tasks: 15 case-level tasks and the slide-level IDH Status task. MLP probing uses five-fold evaluation with case-level fold grouping. Baseline slide encoders average slide embeddings, whereas MOOZY uses its native case embedding. The following selection from Table 1 reports means and standard deviations across folds; F1 is class-weighted F1.

Held-out task	Metric	TITAN	MOOZY
Residual Cancer Burden	F1	\(0.43\pm0.07\)	\(0.56\pm0.05\)
Residual Cancer Burden	ROC-AUC	\(0.58\pm0.07\)	\(0.74\pm0.04\)
ACVR2A mutation	F1	\(0.87\pm0.05\)	\(0.91\pm0.05\)
IDH Status	F1	\(0.94\pm0.02\)	\(0.97\pm0.02\)
BRCA PAM50 subtype	F1	\(0.72\pm0.04\)	\(0.63\pm0.03\)
UCEC genomic subtype	F1	\(0.66\pm0.04\)	\(0.56\pm0.05\)

Across tasks, macro-average weighted F1 is 0.769 and balanced accuracy is 0.702, both the highest among the compared slide encoders. Weighted ROC-AUC is 0.763, below TITAN's 0.768. Here, macro-average means averaging task-level metrics across tasks; it should not be confused with class-macro F1 within one task.

Table 2 also compares five patch encoders, each paired with five MIL architectures and averaged over those architectures. The strongest such averaged baseline, CONCH v1.5, achieves 0.740 F1, 0.720 AUC, and 0.661 balanced accuracy. MOOZY improves these by 0.029, 0.043, and 0.041, respectively. These are comparisons against MIL architecture averages, not evidence that every individual MIL configuration is surpassed by the same margins.

Ablation Study

The following values come from Table 3 and are macro-averages across the 16 held-out tasks. AUC denotes weighted ROC-AUC.

Config	F1	AUC	Balanced accuracy
Stage 1 only	0.743	0.715	0.662
Stage 2 only, without case aggregator	0.743	0.721	0.664
Stage 2 only, with case aggregator	0.731	0.697	0.659
MOOZY, mean slide pooling at inference	0.749	0.737	0.682
Full MOOZY	0.769	0.763	0.702

Footnote 10 explicitly states that the full MOOZY variant without the case aggregator still includes that module during training, discarding it only at inference in favor of mean pooling. The final two rows therefore directly test how a trained model reads out case representations, not a fully retrained module-removal ablation.

Key Findings

• Relative to mean pooling at inference, the full model gains 0.020 F1, 0.026 AUC, and 0.020 balanced accuracy. This supports the value of the learned case readout but does not attribute the entire difference to inter-slide interactions during training.
• Without self-supervised initialization, adding the case aggregator reduces F1 from 0.743 to 0.731. The case module is not an independently effective plug-in; suitable slide representations are an important prerequisite.
• Molecular subtyping reveals clear weaknesses: BRCA PAM50 and UCEC F1 are both below TITAN. Aggregate leadership does not imply uniform task-level superiority.
• Total size is 85.77M parameters: 21.67M in the patch encoder and 64.10M in the slide and case encoders combined. This is approximately one fourteenth of GigaPath's 1.22B, although CHIEF remains smaller in absolute parameter count.

Highlights & Insights

• The unit of reusable representation becomes the patient rather than the slide, instead of combining slides only at the final prediction stage. For clinical workflows with multiple specimens, this modeling unit better reflects the decision structure than simply enlarging a patch encoder.
• Spatial learning precedes clinical semantic alignment through multi-task labels. The weaker case-aggregator results without self-supervised initialization suggest that this ordering is more than an engineering convenience.
• Sparse existing public labels replace a requirement for paired reports and complete outcomes on every case. Other multi-examination medical settings could reuse this organization of shared representations and valid-label masking, although task compatibility would need fresh validation.

Limitations & Future Work

• The authors acknowledge that the output supports global case-level prediction, not dense prediction or segmentation. Attention heatmaps are not validated lesion-segmentation outputs.
• They also call for dedicated tasks that require deeper multi-slide reasoning. RC Burden is evaluated at case level but has a slide-to-case ratio of 1.0, while IDH is slide-level; improvements on these tasks cannot directly establish cross-slide integration ability.
• The main text states that the 16 evaluation tasks are excluded from all training stages, but precise patient exclusions, data overlap, and probe-search protocols are deferred to Appendix G. This cache lacks the appendix, so those details cannot be independently confirmed, and task holdout should not automatically be interpreted as complete pretraining patient or cohort holdout.
• Pathologist review of heatmaps covers only 20 representative WSIs from 8 evaluation cohorts. This small qualitative analysis cannot replace external clinical validation or establish causal alignment between attention and diagnosis.
• Public data reduce dependence on private collections, but the two training stages still require approximately 436 and 512 GPU-hours, respectively, in addition to patch extraction. Fewer parameters do not by themselves demonstrate validated training or deployment efficiency.

Related Work & Insights

• vs task-specific MIL: MIL usually trains an aggregator anew on frozen patch features for each target. MOOZY pretrains slide and case representations, then adapts through a lightweight probe. The probe still uses downstream labels, so this is not zero-shot prediction.
• vs PRISM / TITAN: These approaches use signals such as slide-text pairing to learn representations. MOOZY does not require paired reports but relies on large-scale existing clinical labels. TITAN's stronger macro-average AUC and some molecular-subtyping results suggest that the supervision sources may be complementary.
• vs DINO / iBOT-style self-distillation: MOOZY applies global distribution matching and masked prediction to pathology patch-feature grids, incorporating spatial coordinates and contiguous block masking. The extension to patient-level modeling comes from subsequent case aggregation and joint clinical multi-task training.

Rating

• Novelty: 4/5. Learnable patient-level aggregation and two-stage semantic alignment form a clear combination, while self-distillation and transformer components build on established methods.
• Experimental Thoroughness: 4/5. Evaluation covers 16 tasks, slide encoders, and MIL baselines, but dedicated multi-slide reasoning tests and fully retrained module-removal controls remain limited.
• Writing Quality: 4/5. The main argument and central ablation are clear; interpreting Table 3 requires its footnote, and some cached equations cannot be quoted directly.
• Value: 4/5. The work offers a public-data route to patient-level representation learning, without yet establishing readiness for clinical deployment.