A²LC: Active and Automated Label Correction for Semantic Segmentation¶

Conference: AAAI 2026 arXiv: 2506.11599
Code: Unavailable (authors claim to release upon acceptance)
Area: Semantic Segmentation Keywords: Active label correction, automated label correction, pseudo labels, class imbalance, foundation models

TL;DR¶

This paper proposes the A²LC framework, which augments conventional active label correction (ALC) — where annotators manually fix errors one by one — with an automated correction stage via a Label Correction Module (LCM). The LCM leverages annotator feedback to automatically rectify similar erroneous masks, while an Adaptively Balanced acquisition function (ABC) is designed to mitigate class imbalance. On Cityscapes, A²LC surpasses the previous SOTA using only 20% of the budget, achieving a 27.23% mIoU improvement under equal budget conditions.

Background & Motivation¶

Semantic segmentation relies on large-scale pixel-wise annotated datasets, yet manual annotation is prohibitively expensive and prone to label noise. Active Label Correction (ALC) addresses this by iteratively selecting likely mislabeled samples for human review to construct clean datasets. Kim et al. (ICML 2024) recently incorporated foundation models (Grounded SAM) into the ALC pipeline: pseudo labels are first generated by the foundation model, and uncertain pixels are then presented to annotators for correction.

Nevertheless, two core issues remain in existing pipelines: (1) Redundant queries — foundation models tend to produce systematic biases on fine-grained categories (e.g., traffic lights vs. traffic signs), causing a large number of feature-similar masks to be repeatedly selected and corrected one by one, wasting the annotation budget; (2) Class imbalance — conventional acquisition functions are biased toward head categories, leaving tail categories persistently under-corrected.

Core Problem¶

How can the utility of human annotation feedback be maximized? Specifically: after an annotator corrects a mask where a traffic light was mislabeled as a traffic sign, can that signal be automatically used to fix all similar errors in the dataset, rather than waiting for the annotator to correct them individually? Additionally, how can tail-category correction be adequately addressed within a limited budget?

Method¶

Overall Architecture¶

Input: An unannotated image set. Grounded SAM is first applied to generate initial pseudo labels and a mask collection. The pipeline then iterates for \(R\) rounds: in each round, a segmentation model scores all masks via an acquisition function, and the top-\(B\) masks are sent to annotators for ground-truth correction (manual correction stage). A Label Correction Module (LCM) is then trained on these human-corrected masks to automatically rectify the remaining uncorrected masks (automated correction stage). After both stages, the segmentation model is updated and the next round begins.

Key Designs¶

Label Correction Module (LCM): This is the central contribution of the paper. The LCM is a lightweight four-layer fully connected network (\(256 \to 128 \to 64 \to\) number of classes), taking the mean feature vector of each mask (extracted from the main segmentation model) as input and outputting a class prediction. In each round, the LCM is trained on annotator-corrected clean masks as supervision, and then applied to infer labels for uncorrected masks. To prevent error propagation, three conservative selection criteria are enforced: (a) prediction confidence exceeds threshold \(\tau\) (initialized at 0.99 and incremented each round); (b) the predicted class is not a tail category; (c) the original pseudo label is also not a tail category. Automated correction is executed only when all three conditions are satisfied. This allows the LCM to correct a large number of additional masks at zero human cost — e.g., 22,105 additional masks are corrected in one round on Cityscapes, with accuracy improving from 19% to 61%.
Adaptively Balanced Acquisition Function (ABC): An adaptive class weight \(w(x)\) is multiplied onto the base CIL (Confidence in Label) acquisition function. This weight comprises two components: a class rarity score (computed from the frequency ratios of pseudo-label categories, assigning higher weights to rarer classes) and a dataset imbalance score (measuring the KL divergence between the dataset distribution and the uniform distribution, amplifying the weight effect when imbalance is severe). The final weight is \(w(x) = \hat{w}(x)^{\mathrm{KL}^3}\), which substantially up-weights tail categories on heavily imbalanced datasets while automatically suppressing itself on near-balanced ones, requiring no manual tuning. ABC and LCM are synergistic: ABC guides annotators to prioritize tail-category masks, and this corrective feedback in turn helps LCM learn correction patterns for tail categories.
Mask-level Correction + Non-redundant Correction: The correction unit is elevated from the pixel level to the mask level (SAM-predicted masks as atomic units), avoiding erroneous label propagation inherent in pixel-level correction. Already-queried masks are excluded to prevent redundant queries; as rounds progress, the pool of uncorrected masks shrinks and efficiency continues to improve.

Loss & Training¶

LCM is trained with a weighted cross-entropy loss \(\mathcal{L}_{\mathrm{LCM}} = \lambda_{y(m)} \cdot \mathrm{CE}(y(m), \psi_r(m))\), where class weight \(\lambda_k = (N/N_k) / \sum(N/N_c)\) handles class imbalance.
The main segmentation model is DeepLab-v3+ with ResNet-101 (ImageNet pre-trained), trained for 30K iterations per round with SGD and polynomial LR decay.
The automated correction confidence threshold \(\tau\) starts at 0.99 and increases each round, reflecting a strategy of increasing conservatism over time.
Tail categories are defined with \(\alpha = 0.5\), i.e., the bottom 50% of classes ranked by frequency.

Key Experimental Results¶

Dataset	Metric	Ours (A²LC)	Prev. SOTA (ALC)	Gain
Cityscapes (100k, 1 round)	Model mIoU	72.37%	70.71%	+1.66
Cityscapes (10k×5 rounds)	Data mIoU	85.26%	67.01%	+18.25
Cityscapes (10k×5 rounds)	Model mIoU	70.51%	58.59%	+11.92
PASCAL (1k×5 rounds)	Data mIoU	88.08%	77.06%	+11.02
PASCAL (1k×5 rounds)	Model mIoU	68.42%	65.48%	+2.94
Cityscapes (50k, acquisition fn.)	Data mIoU	84.92% (ABC)	78.98% (SIM)	+5.94

Efficiency highlight: A²LC surpasses the final performance of ALC using only 20% of its budget on Cityscapes, and only 60% on PASCAL VOC.

Ablation Study¶

LCM contributes most: Adding LCM improves 5-round Data mIoU from 66.86% to 80.59% (+13.73), making it the most critical component.
ABC also contributes independently: Adding ABC alone improves Data mIoU from 66.86% to 81.58% (+14.72).
Synergy between LCM and ABC is significant: The combination reaches 84.15%, exceeding the sum of individual contributions, indicating that ABC's tail-category-guided sampling provides higher-quality training data for LCM.
Mask-level correction over pixel-level also yields consistent gains.
In a single automated correction pass, LCM corrects 22,105 masks on Cityscapes with accuracy improving from 19.13% to 60.99%; on PASCAL VOC, 1,629 masks are corrected with accuracy rising from 4.85% to 82.26%.

Highlights & Insights¶

"Annotate once, correct many" core idea: The most elegant contribution of this paper is the transformation of per-instance manual correction into a generalizable correction capability. The LCM, trained on clean annotations, automatically corrects feature-similar uncorrected masks, dramatically improving annotation efficiency.
Adaptive design of ABC weights: The cube of KL divergence serves as a modulating factor — the more imbalanced the dataset, the more aggressively the weighting favors tail categories; for balanced datasets, the function automatically degenerates to the standard acquisition function, requiring no manual hyperparameter tuning.
Three conservative selection criteria: LCM automated correction is only executed under high confidence on non-tail categories, effectively preventing error propagation — a practically robust design choice.
20% budget surpasses 100% baseline: The efficiency gain is substantial, demonstrating that automated correction genuinely reduces annotation costs.

Limitations & Future Work¶

Human involvement remains necessary: Automated correction is supplementary and cannot fully replace manual annotation; each round of the pipeline still requires annotator participation.
LCM architecture is overly simplistic: The four-layer FC network uses only mean features, discarding spatial information, which may limit its effectiveness on shape- or position-sensitive errors.
Limited dataset scale: Experiments are conducted only on Cityscapes (2,975 images) and PASCAL VOC (1,464 images); performance on large-scale benchmarks (e.g., ADE20K) or more complex scenes remains unvalidated.
Pseudo-label ceiling effect: Initial pseudo-label quality is bounded by Grounded SAM's capacity, and accuracy improvements saturate in later rounds; the paper itself acknowledges saturation after 10 rounds.
Underutilization of inter-class confusion patterns: LCM implicitly learns class confusion patterns but does not explicitly model a confusion matrix, leaving potential room for further improvement.
Hyperparameter sensitivity: Settings such as \(\tau\) starting at 0.99 and incrementing, \(\alpha = 0.5\), and the cubic KL divergence exponent are all empirically determined; their robustness has not been thoroughly validated.

vs. ALC (Kim et al., ICML 2024): ALC is the direct predecessor, also using Grounded SAM for pseudo-label generation followed by active pixel-level correction. The key distinction of A²LC is the addition of an automated correction stage (LCM) and the adaptive balanced acquisition function (ABC), shifting from "manual correction only" to "manual correction + automated extension." A²LC comprehensively outperforms ALC across all experiments.
vs. superpixel-based active learning methods (Spx/MerSpx/MulSpx): These methods focus on "which regions to annotate" but do not exploit annotation feedback for automated correction and do not explicitly handle label noise. A²LC achieves 5.77–8.60 mIoU higher under equivalent budgets.
vs. active learning in general: Conventional active learning focuses on "selecting the most informative unlabeled samples," whereas A²LC focuses on "correcting erroneous pseudo labels already generated," constituting a fundamentally different problem formulation.
LCM's "annotation feedback generalization" idea is transferable: For instance, in medical image segmentation, after an expert corrects an organ boundary in one slice, could similar corrections be automatically propagated to adjacent slices in the same CT volume? This paradigm is highly inspiring.
Adaptive weighting via KL divergence is worth borrowing: In any sampling or weighting strategy involving class imbalance, using distributional divergence to automatically modulate weight intensity is an elegant design choice.
This work points toward a broader research direction: given pseudo labels generated by foundation models, how can minimal human intervention be used to bring pseudo-label quality close to ground-truth annotation — a question of value to any semi-supervised or weakly supervised method that relies on pseudo labels.

Rating¶

Novelty: ⭐⭐⭐⭐ Both the LCM automated correction stage and the ABC acquisition function are novel contributions, though the overall framework represents incremental improvement over ALC.
Experimental Thoroughness: ⭐⭐⭐⭐ Two standard benchmarks, multi-round comparisons, ablation studies, and class-wise analyses are comprehensive, though large-scale dataset validation is absent.
Writing Quality: ⭐⭐⭐⭐ The paper is logically clear with rich figures, complete mathematical derivations, and a detailed appendix.
Value: ⭐⭐⭐⭐ Practically meaningful for reducing semantic segmentation annotation costs, though applicability is constrained to settings where foundation models can generate pseudo labels.