Modeling Whole-Slide Images as Dynamic Tumor Microenvironment Fields

Conference: NeurIPS2026 (task archive; reading version: arXiv v1)
arXiv: 2609.34451v1
Area: Medical Imaging
Keywords: whole-slide images, tumor microenvironment, soft region partitioning, concept guidance, pseudo-time evolution

TL;DR

TMEvolve models pathology slides as repeatedly updated latent region fields, learning slide representations through intra-region diffusion and concept-guided directed boundary interactions; it achieves the highest mean C-index on three of four survival cohorts, but its “evolution” denotes representation refinement rather than actual tumor chronology.

Background & Motivation

Whole-slide images contain numerous tissue patches, making detailed patch annotation expensive. Multiple instance learning (MIL) therefore typically extracts patch features and aggregates them into slide representations using attention or Transformers. ABMIL, DSMIL, and TransMIL identify predictive patches, but do not necessarily explicitly represent which tissue region those patches collectively form. Pathological evidence can also arise from the spatial organization of tumor nests, stroma, and immune-enriched compartments, together with their relationships at tissue interfaces. Selecting important individual patches can miss this intermediate-scale structure.

Region and graph methods already incorporate local tissue relationships, but fixed grids, predefined neighborhoods, and static partitions can cross natural boundaries and mix heterogeneous patches prematurely. Uniform smoothing over adjacent patches can also erase differences at tumor–stroma interfaces. The objective here is not to recover the actual disease history from a single slide, but to distinguish two forms of information propagation during representation learning: coherence should increase within a region, whereas heterogeneous regions require selective, directional interactions.

Core Idea: update region partitions together with patch states, diffuse within regions according to membership overlap, transmit visual and semantic differences across boundaries according to pathological concept responses and their spatial polarity, and aggregate the evolved regions for slide-level prediction.

Method

Overall Architecture

Inputs comprise pre-extracted patch features, normalized spatial coordinates, and task-relevant pathological concept descriptions. Outputs are survival, gene expression, or histological subtype predictions. CONCH v1.5 extracts visual features, while Qwen3-Embedding-8B encodes text. GPT-5 generates concept descriptions that pathology experts review. A learnable residual augments the concept embeddings, so the concept bank is not a completely fixed set of semantic probes.

The processing order is “Adaptive Soft Regions → Intra-region Diffusion → Concept-guided Boundary Flux → Region Refresh and Aggregation.” When evolution is incomplete, the last stage returns to the earlier stages to recompute regions, concept responses, and propagation weights. Gated attention aggregation occurs only after evolution completes. Training labels enter the task loss rather than the inference inputs; region quality receives no pixel-level segmentation supervision.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    I["Patch features<br/>and coordinates"] --> A["Adaptive Soft Regions"]
    A --> B["Intra-region Diffusion"]
    B --> C["Concept-guided<br/>Boundary Flux"]
    T["Expert-reviewed concepts<br/>Text encoding and residual"] --> C
    C --> D["Region Refresh<br/>and Aggregation"]
    D -->|Evolution incomplete: recompute| A
    D -->|Evolution complete: attention pooling| O["Task prediction"]
    O -.-> L["Task and diversity losses"]
    Y["Training labels"] -.-> L

The reaction–diffusion equation provides structural inspiration: local coordination, boundary propagation, and semantic source terms correspond to learnable graph updates. The main text and Appendix A explicitly do not treat the model as a continuous biological PDE solver. The pseudo-time index represents network update rounds, not follow-up time, cell migration speed, or disease stage.

Key Designs

1. Adaptive Soft Regions: form intermediate-scale tissue units using morphology and spatial continuity

Farthest point sampling (FPS) first selects spatially dispersed region seeds, with each region maintaining a visual prototype and spatial center. A patch’s matching energy for a region does not depend only on that patch: it averages the distance between normalized patch features and the region prototype, together with the distance between patch coordinates and the region center, over its spatial neighborhood. A softmax over negative energies produces soft memberships.

Regions are therefore constrained by both morphological similarity and spatial proximity rather than fixed grids. Soft memberships allow boundary patches to belong partially to multiple regions; regions are not predefined tumor or stroma labels. Updating features changes matching energies, allowing patches to reorganize rather than remain permanently locked into the initial partition.

Appendix B sets the spatial neighborhood size to 8. The paper mentions merging low-mass regions, but does not clearly specify the mass threshold, merge-target selection, or complete procedure. This note does not supply those implementation rules.

2. Intra-region Diffusion: propagate differences according to soft membership overlap rather than smoothing indiscriminately

Patches within a region need coordination, but patches at boundaries should not simply be mixed. The model computes the inner product of neighboring patches’ soft membership vectors and normalizes it over all neighbors of the target patch to obtain intra-region conductivity. More similar memberships make an edge more suitable for information exchange.

\[ G_{ij}^{q}=\frac{\sum_{k=1}^{K}S_{i,k}^{q}S_{j,k}^{q}}{\sum_{n\in\mathcal{N}(i)}\sum_{k=1}^{K}S_{i,k}^{q}S_{n,k}^{q}},\quad\widetilde{h}_{i}^{q}=h_{i}^{q}+\sum_{j\in\mathcal{N}(i)}G_{ij}^{q}W_{h}\left(h_{j}^{q}-h_{i}^{q}\right). \]

The soft membership matrix specifies patch-to-region affiliations, neighborhoods contain spatially adjacent patches, and the learnable transformation acts on differences between neighboring and current states. The residual update preserves the original state while preferentially coordinating neighbors within the same region. It does not directly replace all patches in a region with their average prototype.

Appendix A uses graph Dirichlet energy to explain the intuition behind difference propagation. However, actual conductivities depend on the current states and can have directed normalization, while updates also include learnable matrices and semantic terms. This does not establish strict energy descent, numerical stability, or biophysical conservation certificates for the complete network.

3. Concept-guided Boundary Flux: constrain heterogeneous-interface propagation by semantic contrast and direction

After diffusion, soft memberships aggregate region features and geometric centers, and each region selects the most similar pathological concept. Patch responses to that concept undergo membership weighting and exponential normalization to obtain a spatial concept-response centroid. Its displacement from the geometric center defines region polarity: it indicates where that concept concentrates within the region, not an observed direction of biological migration.

Each patch inherits the concept of its highest-membership region and computes a scaled dot-product response between its visual state and that concept. For a neighbor to send information to a target, the model jointly evaluates three factors: boundary heterogeneity indicated by low membership overlap, a positive concept-response difference favoring the source over the target, and positive alignment between the source-to-target direction and the source region’s polarity. Their product is normalized over the target neighborhood:

\[ \rho_{ij}^{q}=\frac{b_{ij}^{q}\left[a_{j}^{q}-a_{i}^{q}\right]_{+}\gamma_{ij}^{q}}{\sum_{n\in\mathcal{N}(i)}b_{in}^{q}\left[a_{n}^{q}-a_{i}^{q}\right]_{+}\gamma_{in}^{q}}. \]

The boundary coefficient is one minus the soft membership inner product. The response contrast retains only positive values, and the directional coefficient retains only the positive inner product between spatial displacement and unit polarity. Adjacency alone is therefore insufficient: semantic contrast and spatial direction must also support information flow. Uncertain soft memberships can also increase the boundary coefficient, so it is not a probability of a genuine tissue boundary or an expert annotation.

The same boundary conductivity controls two messages: visual feature differences and concept embedding differences. These are not independent concept prediction branches, but updates sharing propagation gates at heterogeneous interfaces:

\[ h_{i}^{q+1}=\widetilde{h}_{i}^{q}+\sum_{j\in\mathcal{N}(i)}\rho_{ij}^{q}W_{f}\left(h_{j}^{q}-h_{i}^{q}\right)+\sum_{j\in\mathcal{N}(i)}\rho_{ij}^{q}W_{c}\left(c_{j}^{q}-c_{i}^{q}\right). \]

This expression preserves the computational mechanism of main-text Eq. (8), omitting its explanatory underbraces. Importantly, boundary visual differences use the original states from the current round rather than replacing both endpoints with their intra-region-diffused states. The diffusion result enters through the residual base.

The paper does not specify how to handle a zero denominator in boundary conductivity or how to normalize a zero polarity vector. Positive response contrast and directional gates can jointly zero every neighbor term, making this an implementation detail requiring verification. Adding epsilon, uniform weights, or a zero-flux fallback and attributing it to the authors would be unwarranted.

4. Region Refresh and Aggregation: update structure together with representations before reading out slide-level evidence

After updating patches in each round, the model recomputes soft memberships and uses the new weights to update region prototypes and spatial centers. The next round recomputes concept selections, concept-response centroids, intra-region conductivities, and boundary conductivities. Unlike static graph message passing, propagation structure depends on the current representations; regions are not merely aggregation containers obtained from one clustering pass.

After the prescribed rounds, gated attention takes a weighted sum of final region representations, followed by dropout and a task head. Region representations contain morphological information and accumulated local and boundary interactions. The task head and attention pooling are readout mechanisms; attention weights should not be interpreted as independent biological causal contributions.

A Worked Example

For LUAD survival prediction, Appendix B uses 6 regions and 3 evolution rounds. Slide patches first form soft regions around FPS seeds. Morphologically similar, spatially adjacent glandular patches can have similar memberships, so intra-region diffusion preferentially coordinates them instead of assigning identical weights to every neighboring edge.

If a region’s concept response concentrates toward its stromal interface, its response centroid shifts relative to its geometric center. Whether a boundary message enters a neighboring region still depends on whether the source response is higher and whether the edge direction agrees with this polarity; a “tumor–stroma” interface does not automatically activate propagation. Patch memberships and region centers can change after updating. The next two rounds recompute these quantities from the new states, followed by region aggregation and a risk output.

This is a mechanism illustration, not the exact trajectory of a reported patient. It explains how three computational rounds refine representations, not how a tumor passes through three biological stages. The cached figure descriptions also do not provide exact boundary-response values.

Loss & Training

Training uses a task loss and a diversity regularizer on final soft region masses. A region’s mass is its mean membership weight across slide patches, and the regularizer discourages extreme imbalance:

\[ \pi_{k}^{Q}=\frac{1}{N}\sum_{i=1}^{N}S_{i,k}^{Q},\quad\mathcal{L}_{\mathrm{div}}=\sum_{k=1}^{K}\left(\pi_{k}^{Q}-\frac{1}{K}\right)^{2},\quad\mathcal{L}=\mathcal{L}_{\mathrm{task}}+\lambda_{\mathrm{div}}\mathcal{L}_{\mathrm{div}}. \]

The paper uses \(\lambda_{\mathrm{div}}=0.1\). Here “diversity” refers to balanced region usage mass, not a direct constraint on feature orthogonality or concept coverage. Excessively equal masses need not match genuine tissue proportions.

Appendix B specifies hidden dimension 512, Adam learning rate \(1\times 10^{-4}\), batch size 1 slide, and dropout 0.3. The paper describes the task loss only at the downstream-task level, without explicitly providing the forms of the survival, regression, or classification losses. Common MIL recipes should not be substituted as if reported.

All experiments use five-fold cross-validation: standard K-fold for survival and gene expression, and stratified K-fold for subtype classification. The paper does not explicitly state patient-grouped splitting. Some datasets contain multiple slides per case, so patient-level leakage prevention cannot be assumed.

Key Experimental Results

Main Results

The six datasets comprise TCGA LUAD, BLCA, and BRCA; private GBC; and the classification datasets BRACS and EBRAINS. Survival is evaluated on four cohorts, gene expression on the three TCGA cohorts, and classification uses 7/3 and 30/12 fine/coarse classes for BRACS and EBRAINS, respectively. The table below selects representative wins and non-wins from main-text Tables 1 and 2. Units are C-index ×100 and AUC/ACC percentages, reported as five-fold mean ± standard deviation.

Dataset	Task / Metric	TMEvolve	Comparator	Comparator result	Conclusion
LUAD	Survival C-index	69.5 ± 4.2	TransMIL	66.8 ± 4.8	Highest mean
GBC	Survival C-index	77.4 ± 3.8	ProtoSurv	76.2 ± 1.8	Highest mean
BLCA	Survival C-index	65.9 ± 5.7	TITAN	63.0 ± 2.0	Highest mean
BRCA	Survival C-index	70.3 ± 3.1	ProtoSurv	70.4 ± 1.1	Not best
EBRAINS	Fine AUC	98.8 ± 0.2	TITAN	98.2 ± 0.4	Highest mean
EBRAINS	Fine ACC	79.0 ± 1.3	TITAN	79.2 ± 1.8	Not best
EBRAINS	Coarse AUC	99.6 ± 0.2	QPMIL	99.6 ± 0.1	Tied mean
EBRAINS	Coarse ACC	93.7 ± 0.7	TITAN	93.8 ± 1.0	Not best
BRACS	Fine AUC	89.4 ± 0.6	Feather	88.0 ± 2.3	Highest mean
BRACS	Fine ACC	65.4 ± 1.0	CHIEF	62.7 ± 3.9	Highest mean
BRACS	Coarse AUC	92.6 ± 0.7	TITAN	93.5 ± 1.7	Not best
BRACS	Coarse ACC	83.5 ± 1.8	QPMIL	83.3 ± 2.8	Highest mean

“Competitive overall” is therefore more accurate than “consistently better on every metric.” The abstract’s broad advantage claim should not erase these non-wins, and standard deviations do not replace paired significance testing.

The main gene-expression presentation uses 8 genes per cohort selected beforehand for clinical relevance and interpretability. Appendix B states that 25 genes are processed per cohort, but Appendix E Tables 11–13 actually list 24 distinct genes each. This source inconsistency should not be collapsed into a single supposedly verified task count.

Appendix E also includes gene-level non-wins: LUAD MKI67 Pearson ×100 is 67.0 ± 4.0 for TMEvolve versus 72.3 ± 2.7 for TITAN; LUAD CD8A is 68.4 ± 5.0 for TMEvolve versus 65.9 ± 4.2 for CHIEF. Concept guidance does not guarantee improvements for every molecular marker, and expression correlation is not mutation detection capability.

Ablation Study

The following values come from main-text Table 3 and retain its 0–1 scale rather than the percentage scale above. BRACS columns correspond to fine-grained results. The table does not further specify the target aggregation scope for its gene Pearson columns; they should not be assumed to average all processed genes.

Config	BLCA C-index	LUAD C-index	BLCA Pearson	LUAD Pearson	BRACS AUC	BRACS ACC
w/o Intra-region Diffusion	0.6524 ± 0.0549	0.6812 ± 0.0474	0.7058 ± 0.0323	0.6088 ± 0.0284	0.8883 ± 0.0142	0.6431 ± 0.0218
Random Concepts	0.6477 ± 0.0570	0.6728 ± 0.0621	0.7016 ± 0.0297	0.6044 ± 0.0307	0.8873 ± 0.0119	0.6344 ± 0.0095
w/o Pseudo-time Evolution	0.6421 ± 0.0468	0.6746 ± 0.0540	0.7022 ± 0.0267	0.6061 ± 0.0298	0.8885 ± 0.0085	0.6470 ± 0.0243
w/o diversity loss	0.6538 ± 0.0460	0.6817 ± 0.0471	0.7104 ± 0.0263	0.6125 ± 0.0361	0.8894 ± 0.0084	0.6513 ± 0.0196
Full TMEvolve	0.6589 ± 0.0570	0.6952 ± 0.0418	0.7158 ± 0.0326	0.6226 ± 0.0288	0.8940 ± 0.0057	0.6544 ± 0.0095

Random concepts retain probes of the same dimensionality and a comparable parameter budget, supporting the role of language-derived semantic priors rather than merely extra parameters. However, Table 3 does not separately remove boundary visual flux, concept differences, and directional gating, so it cannot establish which boundary component contributes most.

Key Findings

• Random concepts visibly affect LUAD survival and both gene Pearson columns in Table 3. For BLCA survival, the lowest mean instead occurs without pseudo-time evolution; there is no universally dominant component across tasks.
• Tables 4 and 7 support task-dependent settings: BRACS uses 4 regions and 2 rounds, LUAD survival uses 6 regions and 3 rounds, and BLCA survival uses 8 regions and 3 rounds. More regions or rounds do not yield monotonic improvements, so deeper evolution is not automatically better.
• Appendix C Table 8 reports 5.18M trainable parameters and 6.48 ms per slide for TMEvolve, compared with 1.84M and 0.59 ms for ABMIL, and 3.59M and 4.80 ms for TransMIL. TMEvolve is not the fastest comparator.
• Timing uses pre-extracted patch features and batch size 1 on a server equipped with eight NVIDIA RTX 4090 GPUs, excluding offline feature extraction. Foundation models using precomputed slide embeddings are also excluded from the runtime comparison. These measurements are not raw-WSI end-to-end costs and do not establish a cheap complete pipeline.
• Figures 4 and 6 show regions, concept responses, and boundary maps; Figure 5 shows five-fold Kaplan–Meier curves. The text cache does not contain complete numerical values from these plots, so risk-group thresholds, log-rank values, and spatial-coherence metrics are not supplied.

Highlights & Insights

• Treating regions as updatable states rather than fixed pooling windows allows tissue structure and patch features to refine each other. The transferable idea is to recompute grouping and propagation conditions each round, not simply rename a graph network as “evolution.”
• Concept semantics enter boundary interaction rules rather than only the final classifier or query attention. The concept-response centroid converts semantic response strength into a spatial direction cue, but the direction remains model-derived.
• Distinct conductivities for intra-region coordination and heterogeneous interfaces avoid reducing spatial modeling to uniform neighborhood smoothing. Other spatial weak-supervision tasks can compare this dual mechanism with fixed graphs without assuming biological mechanistic validity.

Limitations & Future Work

• Appendix G explicitly acknowledges that pseudo-time is not real time and learned regions are not exact pathological segmentations. Without expert region annotations or spatial molecular measurements, semantically plausible visualizations offer interpretability clues rather than biological causal or clinical validation.
• Experts review the concept bank, but rare-pattern coverage can remain limited and task-specific text can introduce semantic preferences. Further evaluation could perturb concepts, use synonymous descriptions, and test cross-cohort transfer rather than only replacing embeddings with random probes.
• Zero conductivity denominators, zero-polarity normalization, low-mass region merging, and specific task losses are insufficiently documented. Code or supplementary implementation should resolve these issues; this note does not invent fallback policies.
• Patient-grouped K-fold constraints are unspecified, GBC remains private, and external hospital/scanner validation is insufficient. Explicit patient-level splits, external tests, and configuration-selection procedures would better separate methodological gains from data-pipeline effects.
• Appendix A’s energy analogy does not prove strict stability of the complete system with state-dependent directed conductivities and learnable transformations. Theory should analyze the implemented update operator rather than only generic graph diffusion.
• This is research on pathology representation learning, not concrete diagnostic or treatment guidance. Appendix H identifies dataset bias, domain shift, and over-interpretation risks; region maps and predictions should not be treated as independent clinical conclusions.

Related Work & Insights

• vs ABMIL / TransMIL: These methods primarily improve patch-to-slide aggregation, whereas TMEvolve updates soft regions and interface interactions before aggregation. Its gains are not simply a replacement attention pool, and its iterative computation is more involved.
• vs Patch-GCN / HIPT: Graphs or hierarchies provide spatial context, while this paper makes region memberships and conductivities depend on current latent states. Not all spatial methods appear directly in the main comparison tables, so architectural differences should not be equated with experimentally established advantages against each one.
• vs ConcepPath / GECKO / CITE: Concept methods use language to enhance alignment, aggregation, or prediction; TMEvolve extends semantics to local propagation directions and boundary reactions. A research question worth testing is whether the propagation rule still outperforms concept-query pooling after separately ablating polarity and concept differences.
• vs GRAND / PDE-inspired GNNs: These approaches share diffusion-style graph updates as structural inspiration; this paper adds pathological soft regions and concept-conditioned boundaries. The reusable element is a spatial inductive bias, not a claim that latent network evolution simulates actual disease.

Rating

• Novelty: 4/5 — The combination of soft region refresh and concept-directed boundary interactions is distinctive.
• Experimental Thoroughness: 3/5 — Tasks, cohorts, and ablations are broad, but patient splitting, external validation, and component-level ablations remain insufficient.
• Writing Quality: 3/5 — The main pipeline is clear, while gene-count inconsistencies and several numerical implementation details need clarification.
• Value: 4/5 — Useful inspiration for spatial weakly supervised pathology modeling, not an already validated clinical tool.