A Dual-space Patch-driven Complementary Learning Framework for Semi-supervised Multi-organ Segmentation

Conference: ECCV 2026
Paper: Official ECCV page (3317)
PDF: Full paper
Area: Medical Imaging
Keywords: Semi-supervised multi-organ segmentation, class imbalance, local enhancement, co-occurrence decoupling, contrastive learning

TL;DR

DPCL combines rarity- and uncertainty-guided cropping, full-image–patch pseudo-label fusion, and complementary contrastive objectives to reduce confusion between neighboring organs, reaching 68.89% mean Dice on Synapse with two labeled training cases.

Background & Motivation

Semi-supervised medical segmentation uses a small annotated set to learn from a larger pool of unlabeled scans, but multi-organ segmentation introduces more than additional output classes. Large structures supply many training pixels, while small organs such as the pancreas and gallbladder offer fewer examples and less reliable boundaries. With very few labeled cases, consistency training can expand supervision without ensuring that this supervision actually covers the difficult minority structures.

Organ co-occurrence makes the problem subtler: nearby organs repeatedly appear together, allowing a network to use a neighboring structure as evidence for its target. The resulting false activations may be confident, so filtering low-confidence predictions does not necessarily remove them. The paper connects this representation problem to imbalance: minority organs can be both underexposed and entangled with dominant organs, something loss reweighting alone does not directly address.

DPCL changes where the model looks in image space and what it must distinguish in feature space. Targeted patches increase exposure to difficult structures, while class prototypes and contrastive objectives separate locally co-occurring organs without abandoning global class boundaries. Core idea: select local views using both class rarity and uncertainty, then couple focused discrimination of neighboring organs with global class discrimination rather than treating more confident pixels as inherently better supervision.

Method

Overall Architecture

Inputs are labeled and unlabeled CT/MRI images, processed as two-dimensional slices in the experiments. Each full image is accompanied by several sampled patches; a student learns segmentation, and a teacher supplies predictions for unlabeled inputs. The backbone is U-Net with separate normalization statistics for full-image and cropped inputs, while the shared representation benefits from both views.

The image-space path uses Dual-prior Cropping, Dual Normalization, and Pseudo-label Fusion to obtain useful local views and combine their predictions with global context. The feature-space path uses Complementary Co-occurrence Contrast, constructing reliable class prototypes from labeled data and supervising both patch-level class representations and full-image pixel features. These are training-time interactions: inference uses full images only, without repeating patch extraction and prediction fusion.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Labeled and unlabeled<br/>full slices"] --> B["Dual-prior Cropping"]
    B --> C["Dual Normalization<br/>full-image and patch encoding"]
    C -->|"Unlabeled predictions"| D["Pseudo-label Fusion"]
    C -->|"Labeled features"| E["Complementary<br/>Co-occurrence Contrast"]
    D --> F["Joint segmentation<br/>and contrastive training"]
    E --> F
    F --> G["Full-image-only inference<br/>organ segmentation"]

Key Designs

1. Dual-prior Cropping: cover both minority classes and ambiguous regions

Uniform cropping can repeatedly select background or easy parts of large organs. DPCL instead samples crop centers from a pixel-weight distribution combining class frequency and predictive uncertainty. The class prior is computed from foreground pixel counts within the current image, with logarithmic reweighting favoring rarer classes. The uncertainty prior is prediction entropy: it highlights locations where several classes remain plausible, but alone cannot tell a small organ from an uncertain part of a large one. Multiplying the two priors preferentially exposes regions that are both underrepresented and difficult.

Equation (5) multiplies one plus pixel entropy by the corresponding class weight and adds a small positive constant to preserve nonzero sampling probability. Entropy is \(\mathcal H(P_j)=-\sum_{k=0}^{K-1}P_j(k)\log P_j(k)\), where \(P_j(k)\) is the predicted probability of class \(k\) at pixel \(j\). For labeled images, frequencies come from ground-truth masks and uncertainty from student predictions; for unlabeled images, the teacher's full-image pseudo-labels and probabilities replace these inputs. The method samples centers probabilistically and records crop coordinates so labels and predictions can be mapped between local and full-image positions. The default is four \(64\times64\) patches per image, not a fixed partition into nonoverlapping tiles.

2. Dual Normalization: share recognition features without mixing view statistics

A crop is not simply a smaller full image: its background fraction, organ proportions, and intensity distribution can differ substantially. A single set of normalization statistics can therefore create a conflict between the two input domains, offsetting the intended benefit of local enhancement. DPCL replaces U-Net normalization layers with Dual Normalization (DN), maintaining separate means, variances, and affine parameters for full-image and patch inputs. Each domain updates its running statistics with its own exponential moving average; this is not equivalent to training two independent segmentation backbones.

Shared convolutional features still learn from both views, but normalization accounts for the distribution that produced an activation. The model can absorb fine local appearance without applying patch-specific intensity statistics indiscriminately to full images. Only the full-image path is required at inference, although additional crop processing still incurs training cost.

3. Pseudo-label Fusion: select the more certain view at each pixel

The teacher predicts weakly augmented full images and patches, while the student processes strongly augmented versions of unlabeled inputs. Patch predictions are first placed back into full-image coordinates. Where several patches overlap, fusion selects the prediction with the lowest entropy rather than averaging their probabilities. The merged local prediction is then compared with the full-image prediction: covered pixels use the lower-entropy view, and uncovered pixels retain the full-image prediction. This allows local detail to correct a coarse boundary while letting global context override an uncertain crop.

The selected class forms the unified pseudo-label, and its maximum class probability is subsequently filtered with threshold \(\gamma_1\). Entropy thus decides which view to use, whereas the maximum-probability threshold decides whether the resulting pixel should provide supervision. The same fused target supervises the student's full-image and spatially corresponding patch predictions through an unsupervised Dice objective. Lower entropy is not a guarantee of correctness, which is why the separate representation objective remains necessary for confident co-occurrence errors.

4. Complementary Co-occurrence Contrast: separate neighbors locally and retain global class boundaries

A projection head maps student features into the contrastive representation space. Class prototypes are constructed only from labeled inputs: ground truth selects the class, and predicted class probability of at least \(\gamma_2=0.9\) filters unreliable pixels before feature averaging. Full-image and crop representations jointly update a class-indexed momentum prototype bank, using momentum 0.99. Crops containing small organs provide targeted updates for classes whose prototypes would otherwise update infrequently. Prototype construction should not be confused with patch queries: the latter average same-class patch features under ground-truth masks, while the stated reliability filter specifically applies to prototype construction.

Patch-based contrastive loss (PBC) uses a patch's class representation as a query and the corresponding global class prototype as its positive. Its negative classes are restricted to other classes present in that patch, concentrating separation pressure on structures that actually co-occur locally. Global-based contrastive loss (GBC) instead uses labeled full-image pixel features as queries and compares them with all class prototypes, pulling each toward its ground-truth class. In particular, Equation (18) is a pixel-to-prototype objective, not merely an alignment loss between two prototypes.

Both objectives access prototypes updated from full images and crops, connecting global semantics with local detail. PBC alone does not directly separate classes that never meet in a patch; GBC alone can dilute difficult neighboring-organ relationships within a broader objective. Their combination preserves targeted local discrimination and global class consistency instead of adding two interchangeable contrastive penalties.

Loss & Training

Labeled full images and patches receive cross-entropy and Dice supervision, while unlabeled inputs receive confidence-filtered pseudo-label supervision. Equation (19) combines these objectives as follows:

\[ \mathcal L_{\mathrm{total}} =\mathcal L_{\mathrm{sup}}+\lambda_u\mathcal L_{\mathrm{unsup}} +\lambda_m(\mathcal L_{\mathrm{pbc}}+\mathcal L_{\mathrm{gbc}}). \]

The paper sets \(\lambda_u=0.25\) and uses Gaussian warm-up, \(\lambda_m(t)=0.1\exp[-5(1-t/t_{\max})^2]\), for the contrastive terms. Gradually increasing their influence avoids making initially unstable representations carry strong contrastive supervision immediately. Implementation uses 256-dimensional projected features, SGD with momentum 0.9, weight decay 0.0001, an initial learning rate of 0.01 with polynomial decay, and 30,000 training iterations. Labeled and unlabeled batch sizes are each six; experiments use an NVIDIA 3090, with volumes converted into \(256\times256\) two-dimensional slices. The main text does not clearly specify the teacher parameter-update rule, so a conventional Mean Teacher EMA configuration should not be treated as a verified implementation detail here.

Key Experimental Results

Main Results

The selected results below come from Tables 1 and 2; Dice is higher-is-better, and average surface distance (ASD), measured in voxels, is lower-is-better. Each row compares the same dataset and labeling setup, selecting the comparison method with the highest tabulated mean Dice, not necessarily the best ASD. Gains are computed as absolute Dice percentage-point differences rather than repeating questionable relative improvements in the paper's prose.

Dataset and labeled training cases	Comparator	Comparator Dice / ASD	DPCL Dice / ASD	Dice gain (pp)
Synapse, 1/18 (called 5%)	ABD	53.57 / 32.84	61.48 / 23.78	+7.91
Synapse, 2/18 (called 10%)	AD-MT	62.73 / 22.25	68.89 / 14.84	+6.16
FeTA2021, 3/56 (called 5%)	AD-MT	68.31 / 4.60	70.04 / 4.58	+1.73
SegTHOR, 3/28 (called 10%)	FixMatch	76.65 / 4.20	78.18 / 3.26	+1.53
ACDC, 7/70 (called 10%)	CGS	89.83 / 0.68	90.47 / 0.28	+0.64

Section 4.1 specifies 18 training and 12 test cases for Synapse, and train/validation/test splits of 56/8/16 for FeTA2021, 28/4/8 for SegTHOR, and 70/10/20 for ACDC. The paper uses the same labeled/unlabeled data settings across methods and U-Net as the main comparison backbone, but DPCL adds crop processing and contrastive computation; equal training compute is not established. For ACDC, Table 2 lists both a reproduced M3HL Dice of 88.83 and its originally reported 90.47, which DPCL matches rather than exceeds. On Synapse at 10% labeling, SKCDF achieves ASD 12.38 versus DPCL's 14.84, so superiority on every metric is also unsupported.

Ablation Study

These selected rows from Table 3 use Synapse with two labeled training cases and the same 12-case test split. Local Enhancement includes crop-related and pseudo-label training; the first row also retains DN and is not a supervised-only baseline.

Configuration	Dice (%) ↑	ASD (voxels) ↓
Local Enhancement + DN, neither PBC nor GBC	64.55	20.85
Local Enhancement + DN + PBC	66.17	16.11
Local Enhancement + DN + GBC	67.50	15.88
Local Enhancement + PBC + GBC, without DN	64.84	20.48
Full DPCL	68.89	14.84

The complete system exceeds the configuration without PBC by 1.39 Dice points, without GBC by 2.72 points, and without DN by 4.05 points. This supports complementary contrastive objectives and shows that normalization conflicts matter when full images and patches share a network. However, the Local Enhancement configuration already uses unlabeled learning, so its entire gain over supervised-only training cannot be attributed to cropping alone.

Key Findings

• Table 4(c) reports Dice 65.57 for uniform sampling, 67.54 for uncertainty-only sampling, 66.10 for class-frequency sampling, and 68.89 for the combined strategy. Both priors contribute useful information rather than one entirely replacing the other.
• Within the tested settings in Table 4(a–b), four patches and crop size 64 perform best; increasing the patch count does not monotonically improve accuracy. These are Synapse-specific findings, not universal settings for every organ dataset.
• On Synapse with 10% labeling, Table 1 reports pancreatic Dice 54.90 versus liver Dice 91.21. Difficult-class performance improves, but substantial inter-organ disparities remain.

Highlights & Insights

• Exposure and discrimination are designed together. Sampling determines what the model repeatedly sees, while PBC determines which neighboring structures it must distinguish; more minority-class patches alone do not guarantee disentanglement.
• Prediction reliability and representation separability are different problems. Entropy selection and confidence filtering improve pseudo-label selection, while contrastive supervision addresses feature confusion that can survive those filters.
• Most added machinery stays in training. Patch learning and domain-specific normalization transfer local information into a full-image inference path, without establishing that training cost or measured inference latency is unchanged.

Limitations & Future Work

• Evidence boundary: The main text has no dedicated limitations discussion and reports no multi-seed means, standard deviations, or significance tests. Sensitivity to which one or two cases are labeled is a reader concern, not a quantified author finding.
• Reader assessment: High-confidence labeled pixels reduce prototype noise, but very difficult classes may have too few reliable prototype contributors. Targeted sampling improves exposure without creating missing annotated appearances.
• Scope: Evaluation uses two-dimensional slices, without establishing three-dimensional consistency, external cross-center performance, or clinical deployment readiness. Results across four datasets are not a clinical generalization guarantee.
• Reporting inconsistencies: Section 4.4 gives full-model Dice as 68.67, whereas Tables 1, 3, and 4 consistently give 68.89; several stated ablation improvements also disagree with the tables. This note uses table values and explicitly computed percentage-point differences.
• Available material: The local source contains 18 pages of main text and references, but not the extra hyperparameter appendix mentioned in the paper. Unverified temperature, pseudo-label threshold, and teacher-update settings are not supplied here, and no code URL is provided.

Related Work & Insights

• vs BCP / ABD: These methods improve labeled/unlabeled data use through bidirectional copy-paste or displacement; DPCL additionally targets crop locations using class and uncertainty priors and explicitly separates co-occurring organ representations.
• vs PH-Net: PH-Net applies hard-patch contrastive learning to breast lesion segmentation. DPCL addresses multiclass segmentation by making locally co-occurring classes a targeted negative set.
• vs CGS / SKCDF: Their emphases are generalist–specialist collaboration and semantic knowledge complementarity, respectively. DPCL focuses on view-specific normalization and complementary local/global feature constraints rather than simply adding segmentation heads.

Rating

• Novelty: 3.5/5. A well-motivated combination of class-aware cropping, domain-specific normalization, and contrastive objectives, although the underlying mechanisms are established.
• Experimental Thoroughness: 3.5/5. Four datasets and component ablations are useful, but repeated-run statistics, compute measurements, and external generalization tests are missing.
• Writing Quality: 3/5. The problem and architecture are understandable, but numerical inconsistencies and underspecified implementation details reduce precision.
• Value: 4/5. An interpretable training modification for label-scarce multi-organ segmentation with a comparatively simple inference workflow.