Residual-Guided Expert Specialization for Incomplete Multimodal Learning

Conference: ECCV 2026
arXiv: 2606.30355
Code: https://github.com/seunghub/MARS
Area: Multimodal VLM / Semantic Segmentation
Keywords: Incomplete Multimodal Learning, Mixture of Experts, Residual Routing, Modality Missing, Representation Shift

TL;DR

MARS proposes a Mixture-of-Experts (MoE) framework that captures the representation shift patterns caused by missing modalities by computing the residual between full and incomplete modality representations to guide expert specialization. Concurrently, it introduces dual-router distillation and discrepancy-aware noise regularization to tackle training-inference routing inconsistency, comprehensively outperforming state-of-the-art (SOTA) methods on four incomplete multimodal classification and segmentation datasets.

Background & Motivation

Background: Multimodal learning enhances classification and segmentation performance by fusing multi-source information such as RGB, depth, infrared, and text. However, in practical deployment, some modalities are often missing due to sensor failures or acquisition costs, making Incomplete Multimodal Learning (IML) a key challenge for practical application. Existing methods generally fall into three categories: imputation-based methods attempt to reconstruct missing modalities from available ones but are prone to hallucination and incur high computational overhead; representation-learning-based methods (e.g., LCR, RFNet, DMRNet) learn representations robust to missing modalities but operate strictly on incomplete inputs, failing to explicitly model the impact of the missing modalities; and expert-based methods (e.g., MoMKE, SimMLM) introduce multi-expert structures but hard-bind expert roles to modality combinations (requiring \(2^M\) experts) or employ two-stage training, which limits flexibility.

Key Challenge: No matter how exquisitely a model is designed, a systematic discrepancy inevitably exists between the task representation \(z^{\text{partial}}\) produced by incomplete inputs and the representation \(z^{\text{full}}\) under full modalities — the missing modalities strip away complementary evidence, irreversibly "reshaping" the representations. However, all prior methods solely make decisions on the degraded representations, and none explicitly represent or utilize the patterns of this "reshaping". A natural follow-up question is: does this representation shift (i.e., the residual \(z^{\text{full}} - z^{\text{partial}}\)) itself contain valuable information for the model?

Core Idea: Use the difference between the complete and incomplete representations as a "privileged residual signal," enabling the MoE router to assign experts to samples based on the representation shift patterns characterized by the residual, so that different experts specialize in different missing patterns. During inference, a feature router that depends solely on incomplete inputs distills the behavior of the residual router, realizing implicit inference of missing patterns and expert selection.

Method

Overall Architecture

MARS addresses two problems: first, how to use residual signals during training to teach experts to specialize according to "missing patterns"; and second, how to maintain the same specialized behavior during inference when full modalities are unavailable (and residuals cannot be computed). The core of the method is a sparse MoE structure, but the router does not directly inspect the incomplete input \(z^{\text{partial}}\). Instead, it looks at "how much this incomplete input deviates from the complete input" — namely, the residual \(z^{\text{res}} = z^{\text{full}} - z^{\text{partial}}\). During training, the residual router uses \(z^{\text{res}}\) for routing allocation, while the feature router uses \(z^{\text{partial}}\) to mimic its behavior; during inference, the feature router operates independently. Meanwhile, discrepancy-aware noise regularization directs the residual router to inject larger noise into experts with high routing discrepancy, making these experts more robust to routing mismatches. Discrepancy-guided sampling prioritizes training modality combinations with high routing discrepancy. The entire framework forms a closed loop of "privileged-information-guided specialization \(\rightarrow\) distillation transfer to the deployable router \(\rightarrow\) noise-based mitigation of distillation error".

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Full + Incomplete Modality Inputs"] --> B["Encoding & Fusion to obtain<br/>z_full, z_partial"]
    B --> C["Residual Computation<br/>z_res = z_full − z_partial"]
    C --> D["Residual Routing<br/>R_res allocates experts by shift"]
    B --> E["Feature Routing<br/>R_fea mimics R_res"]
    E -->|"KL Distillation"| D
    D --> F["MoE Experts<br/>Weighted aggregation of predictions"]
    E --> F
    F --> G["Task Output"]
    D --> H["Discrepancy-Aware Noise<br/>Scale noise by routing discrepancy"]
    E --> H
    H --> I["Discrepancy-Guided Sampling<br/>Prioritize high-discrepancy combinations"]
    I -.->|Next epoch training| A

Key Designs

1. Residual-Guided Expert Routing Specialization: Routing with Representation Shifts Instead of Degraded Representations

Traditional MoE routers directly inspect the incomplete input \(z^{\text{partial}}\) to determine which experts to activate. However, \(z^{\text{partial}}\) itself is distorted due to missing modalities, and the information the router can extract regarding "what is missing, and to what extent" is limited and implicit. Instead, MARS utilizes the residual \(z^{\text{res}} = z^{\text{full}} - z^{\text{partial}}\) as the routing input. This residual directly characterizes the "perturbation direction and magnitude" of the missing modalities on the task representation, which reflects the structural features of missing patterns much better than the degraded representation itself.

The residual router \(\mathcal{R}_{\text{res}}\) receives \(z^{\text{res}}\) and outputs clean logits \(l_i^{\text{res}} \in \mathbb{R}^N\) and noise standard deviation \(\sigma_i \in \mathbb{R}^N\) (where \(N\) is the number of experts). It adopts a noisy Top-K routing mechanism: Gaussian noise \(\tilde{l}_i^{\text{res}} = l_i^{\text{res}} + \epsilon \sigma_i\) (where \(\epsilon \sim \mathcal{N}(0, I)\)) is injected into the logits, selecting the top-\(K\) experts to compute the routing probability \(p_i^{\text{res}}\) via \(\text{Softmax}_K\). Each expert \(E_j\) processes \(z^{\text{partial}}\), and the outputs are weighted by \(p_i^{\text{res}}\), aggregated, and fed into the task head \(h\): \(\hat{y}_i^{\text{partial}} = h(\sum_{j=1}^N (p_i^{\text{res}})_j E_j(z_i^{\text{partial}}))\). During training, for each sample within the same batch, both the complete input and a randomly masked incomplete input are constructed simultaneously, allowing \(z^{\text{res}}\) to be computed. In experiments, Oracle routing (which uses the residual router even during inference) achieves an ACER of 0.94, demonstrating that the experts have fully specialized according to the missing patterns.

2. Distillation Learning of the Feature Router: Enabling the Deployable Router to Mimic the Privileged Router

The core limitation of the residual router is its unavailability during inference — when modalities are missing, \(z^{\text{full}}\) cannot be obtained, making \(z^{\text{res}}\) uncomputable. The solution proposed by MARS is to train a feature router \(\mathcal{R}_{\text{fea}}\), which takes only the incomplete input \(z^{\text{partial}}\) as input and learns to mimic the routing decisions of the residual router via knowledge distillation.

Specifically, \(\mathcal{R}_{\text{fea}}\) outputs clean logits \(l_i^{\text{fea}}\) for \(z^{\text{partial}}\). These are not used directly for the task loss (which is still dominated by the residual router) but are aligned with the residual router's logits using KL divergence: \(\mathcal{L}_{\text{distill}} = D_{\text{KL}}(\text{Softmax}(\text{GradStop}(l_i^{\text{res}})) \parallel \text{Softmax}(l_i^{\text{fea}}))\). Here, \(\text{GradStop}\) prevents the residual router from being backpropagated by the distillation loss, ensuring that the distillation is unidirectional — the residual router maintains optimal routing, while the feature router aligns toward it. When all modalities are available, \(z^{\text{res}} = 0\), and routing is fully handled by the feature router. This design decouples "specialization" (completed by the residual router during training) and "deployment" (executed by the feature router during inference), representing the core architectural decision of the paper.

3. Discrepancy-Aware Noise Regularization: Bridging the Training-Inference Routing Gap with Noise

Even with the distillation loss, the feature router cannot perfectly replicate the behavior of the residual router due to the lack of complete information provided by \(z^{\text{full}}\). This inevitable routing bias causes samples allocated to a certain expert during training to be allocated to a different expert during inference. Since the target expert has not seen these samples, performance degrades.

The solution in MARS is to inject noise "directionally": for experts where the two routers have larger divergence, the residual router injects a larger noise variance, forcing the respective expert to be exposed to more uncertainty during training and making it more robust to routing shifts during inference. Specifically, for each sample, given the top-K expert sets \(\mathcal{T}_i^{\text{res}}\) and \(\mathcal{T}_i^{\text{fea}}\) from both routers, for each expert \(j\) in their union \(\mathcal{U}_i\), the routing discrepancy is calculated as \((m_i)_j = |\text{Softmax}(l_i^{\text{res}})_j - \text{Softmax}(l_i^{\text{fea}})_j|\). The experts are sorted in descending order of \(m_i\) to obtain a permutation \(\pi_i\). A Softplus loss is then used to penalize cases where the order of noise variance is inconsistent with the order of discrepancy:

\[\mathcal{L}_{\text{noise}} = \frac{1}{|\mathcal{U}_i|-1}\sum_{j=1}^{|\mathcal{U}_i|-1}\text{Softplus}\big(\log(\sigma_i)_{\pi_i(j+1)}^2 - \log(\sigma_i)_{\pi_i(j)}^2\big)\]

This loss softly enforces \((\sigma_i^2)_{\pi_i(1)} \geq \dots \geq (\sigma_i^2)_{\pi_i(|\mathcal{U}_i|)}\), indicating that experts with larger discrepancy receive larger noise. Compared to standard load-balancing noise in traditional MoE, this design adds a layer of "purposefulness" — noise is not just for expert diversification, but specifically to bridge the training-inference routing gap.

4. Discrepancy-Guided Modality Sampling: Concentrating Training on Modality Missing Patterns Most Difficult to Align

Different modality combinations present different training difficulty levels. Under certain combinations (e.g., IR-only), the discrepancy between the residual router and the feature router is exceptionally large, indicating that the feature router struggles to replicate the decisions of the residual router under these conditions. MARS proposes dynamically adjusting the sampling probability of each modality combination based on the Top-K overlap rate of both routers after a warm-up of several epochs.

For each incomplete modality combination \(c_j\), the routing discrepancy over all samples in the current epoch is computed as \(d_j = 1 - \frac{1}{|\mathcal{D}_{c_j}|}\sum_{i \in \mathcal{D}_{c_j}} \frac{|\mathcal{T}_i^{\text{res}} \cap \mathcal{T}_i^{\text{fea}}|}{K}\). Larger discrepancy indicates that the feature router struggles more to align under this combination. The sampling probability for the next epoch is then normalized using softmax: \(q_j^{(t+1)} = \frac{\exp(d_j/\tau)}{\sum_k \exp(d_k/\tau)}\). In experiments, the IR-only combination receives the highest sampling probability (approx. 0.43), closely aligning with the fact that all prior methods perform poorest under this setting — MARS effectively allocates most training resources to the most challenging scenarios.

A Complete Example: Inference Process on CASIA-SURF with Only the IR Modality

Taking the face anti-spoofing dataset CASIA-SURF as an example, the inputs comprise three modalities: RGB, Depth, and IR. During inference, if only IR is available (RGB and Depth sensor failures), the pipeline is as follows: The IR image is processed by the modality encoder to produce \(e_i^{\text{IR}}\), while RGB and Depth are zeroed out due to \(\delta = 0\). After concatenation and processing via the fusion layer, \(z^{\text{partial}}\) is obtained. Feature router \(\mathcal{R}_{\text{fea}}\) maps \(z^{\text{partial}}\) to logits over 16 experts. After Top-5 activation, expert E0 receives the highest routing probability. E0 has been repeatedly assigned "IR-only" samples by the residual router during training (since the residual \(z^{\text{full}} - z^{\text{partial}}\) presents a specific shift direction under the IR-only setting). E0 processes \(z^{\text{partial}}\) and outputs features, which are weighted and aggregated with the outputs of 4 other experts, before being fed into the classification head to determine real face vs. attack. The entire process requires no complete modalities, and MARS reduces the ACER from DMRNet's 8.98 down to 3.96 in this most challenging scenario.

Loss & Training

The total loss is a weighted sum of four terms:

\[\mathcal{L}_{\text{total}} = \lambda_{\text{task}}\mathcal{L}_{\text{task}} + \lambda_{\text{LB}}\mathcal{L}_{\text{LB}} + \lambda_{\text{distill}}\mathcal{L}_{\text{distill}} + \lambda_{\text{noise}}\mathcal{L}_{\text{noise}}\]

where \(\mathcal{L}_{\text{task}} = \ell(\hat{y}_i^{\text{full}}, y_i) + \ell(\hat{y}_i^{\text{partial}}, y_i)\) simultaneously supervises the task losses (cross-entropy) for complete and incomplete inputs; \(\mathcal{L}_{\text{LB}} = \text{CV}^2(\text{importance}) + \text{CV}^2(\text{load})\) is the load-balancing loss, measuring expert utilization uniformity using the squared coefficient of variation; \(\mathcal{L}_{\text{distill}}\) and \(\mathcal{L}_{\text{noise}}\) are defined as above. During training, both complete and incomplete inputs are constructed for each sample (masks for incomplete inputs are randomly sampled), and the complete input is always retained during each iteration to provide residual supervision. On CASIA-SURF, using a ResNet-18 backbone, SGD optimizer, \(N=16\) experts, and Top-\(K=5\), the loss weights are set to \(\lambda_{\text{task}}=1, \lambda_{\text{LB}}=0.05, \lambda_{\text{distill}}=1, \lambda_{\text{noise}}=0.01\), and discrepancy-guided sampling is activated from the 20th epoch.

Key Experimental Results

Main Results

CASIA-SURF Face Anti-Spoofing Classification (ACER ↓, lower is better):

Modality Combination (RGB/Depth/IR)	ResNet-18	HeMIS	DMRNet	SimMLM	MARS
●○○ (RGB Only)	11.75	14.36	8.23	10.04	6.92
○●○ (Depth Only)	5.87	4.70	2.01	1.95	1.87
○○● (IR Only)	16.62	16.21	8.98	12.57	3.96
●●○	4.61	3.23	1.21	1.20	1.06
●○●	6.68	6.27	3.00	5.09	2.09
○●●	4.95	3.68	0.80	1.06	0.66
●●● (Full)	2.21	1.97	0.66	0.79	0.45
Average	7.52	7.18	3.58	4.67	2.43

MARS achieves the lowest ACER under all 7 modality combinations, reducing the average ACER by 1.15 compared to the strongest baseline DMRNet. In the most challenging setting where only IR is available, MARS reduces the error rate from 8.98 to 3.96, a decrease of over 50%.

MCubeS Material Segmentation (mIoU ↑, higher is better):

Modality Combination	DeepLab v3+	DMRNet	SimMLM	MARS
RGB Only	0.4249	0.4647	0.4462	0.4730
RGB+AoLP	0.4238	0.4682	0.4473	0.4767
RGB+DoLP	0.4249	0.4670	0.4420	0.4753
RGB+NIR	0.4247	0.4672	0.4355	0.4769
RGB+AoLP+DoLP	0.4269	0.4694	0.4427	0.4777
RGB+AoLP+NIR	0.4269	0.4701	0.4286	0.4808
RGB+DoLP+NIR	0.4261	0.4691	0.4224	0.4776
Full Four Modalities	0.4271	0.4712	0.4220	0.4808
Average	0.4257	0.4683	0.4367	0.4773

On the segmentation task, MARS similarly performs best across all combinations, reaching an average mIoU of 0.4773. On CREMA-D emotion recognition, the average accuracy is 65.52% (compared to DMRNet's 61.35%), and on UPMC Food-101 food classification, the average accuracy is 91.59% (compared to SimMLM's 84.81%), validating the generalizability of the method across different domains and tasks.

Ablation Study

Configuration	CASIA-SURF ACER ↓	MCubeS mIoU ↑	Description
Baseline (ResNet-18 / DeepLab v3+)	7.52	0.4257	Vanilla model without MoE
+ MoE (Without Residual Routing)	4.12	0.4327	Standard MoE only, routing based on \(z^{\text{partial}}\)
+ Residual Routing	3.78	0.4661	Introduces residual router, core contribution
+ Noise Regularization	2.72	0.4750	Introduces discrepancy-aware noise to bridge the routing gap
+ Discrepancy-Guided Sampling (Full MARS)	2.43	0.4773	Introduces adaptive modality sampling
Oracle Routing (Residual router also used during inference)	0.94	0.4768	Theoretical upper bound: complete modalities available during inference

The introduction of residual routing yields a significant mIoU improvement of 0.0334 on the segmentation task, representing the largest individual gain and confirming the critical role of residual signals in expert specialization. On the segmentation task, MARS even slightly outperforms the Oracle (0.4773 vs. 0.4768), implying that noise regularization not only bridges the routing gap but also encourages experts to learn complementary decision boundaries.

Expert deactivation experiments further validate the structural nature of specialization: deactivating expert E0 (the most critical expert for this combination) under the IR-only setting increases the ACER from 3.96 to 7.58 (+3.62), whereas deactivating E0 under the RGB-only setting has no impact. This indicates that a stable "expertise correspondence" has formed between experts and modality combinations.

Key Findings

• Residual Routing is the Main Driver: Moving from standard MoE to introducing residual routing drops the CASIA-SURF ACER from 4.12 to 3.78, and elevates MCubeS mIoU from 0.4327 to 0.4661 (+0.0334). This constitutes the largest single-step improvement compared to the vanilla baseline, verifying that "looking at the residual instead of the complete representation" is the correct approach.
• Noise Regularization Offers Significant and Unexpected Gains: Noise regularization contributes to a 1.06 ACER reduction in classification. More interestingly, on segmentation, it allows MARS to outperform the Oracle. Noise not only prevents over-reliance on residual routing but also guides experts to learn more effective complementary decision boundaries under feature routing.
• Expert Specialization is Highly Interpretable: Visualization of the routing distribution reveals that MARS's expert activation adaptively changes with the modality combinations structurally (e.g., E12 corresponds to RGB-only, E0 to IR-only, and E5 to full modalities), showing minimal expert overlap across different combinations. In contrast, standard MoE routing systematically biases toward the same few experts under all combinations.
• Outstanding Parameter Efficiency: MARS features only 244M parameters / 716G FLOPs, which is close to DMRNet (238M / 639G), and significantly lower than Flex-MoE (672M / 716G) and SimMLM (247M / 955G), indicating that the extra overhead introduced by residual routing is negligible.

Highlights & Insights

• Converting "Missingness" Itself into a Signal: While traditional IML methods view missing modalities as noise to be tolerated or blanks to be filled, MARS takes the opposite direction — the representation shift caused by the missing modalities itself serves as a valuable routing signal. This paradigm shift serves as upstream inspiration for system design and is highly transferable to other incomplete input scenarios (such as reasoning under occlusion or partially observable reinforcement learning).
• Elegant Decoupling of Privileged Information + Distillation: Splitting "how to specialize with complete information" and "how to maintain specialization under incomplete information" into two routers connected via distillation presents a highly general design pattern. Any scenario involving "extra information during training but not during inference" (e.g., teacher forcing, oracle features) can adopt this dual-router distillation framework.
• Making Noise Directional and Purposeful: While noise in standard MoEs only serves load balancing, MARS links noise intensity to routing discrepancy — experts with larger discrepancy are injected with more noise, actively countering training-inference inconsistency. This "purpose-driven randomness" offers a fresh concept in MoE design, not limited to multimodal settings.
• Grad-CAM Visualization Confirms Essential Improvements: The attention maps of MARS consistently focus on stable facial regions (such as the nose), whereas standard MoE shifts focus to shallow textures like the mouth corners or forehead as the modality combinations vary. This demonstrates that residual routing aids the model in capturing cross-modality invariant discriminative clues rather than relying on surface patterns of specific modalities.

Limitations & Future Work

• Dependency on Complete Modalities During Training: MARS assumes all modalities are fully available in the training dataset, making the quality of the residual signals directly dependent on the reliability of the complete representation. In scenarios where training data itself is inherently incomplete (e.g., naturally missing examinations in medical datasets), residuals cannot be calculated directly, requiring alternative solutions (such as aggregating approximate complete representations from multiple samples as anchors).
• Limited Expressiveness of Single-Layer Residuals: \(z^{\text{res}} = z^{\text{full}} - z^{\text{partial}}\) is a linear residual at the post-fusion representation level. Although the authors validate in the appendix that subtraction outperforms alternatives like addition, product, concatenation, and attention (Table A1），this still fundamentally assumes that a linear difference in the representation space suffices to encapsulate the effects of missing modalities. When missing-induced representation shifts lie on non-linear manifolds, multi-layer or multi-scale residual modeling might be required.
• Potential Instability of Top-K Routing Under Extreme Missingness: When most modalities are missing, causing \(z^{\text{partial}}\) to be highly unreliable, the mimicry quality of the feature router will degrade further. While noise regularization alleviates this, it cannot entirely eradicate the issue. A potential direction is to introduce uncertainty estimation for the feature router, triggering rollback strategies (such as activating more experts or using average aggregation) during highly uncertain routing decisions.
• Sampling Challenges Under Exploding Multimodal Combinations: Discrepancy-guided sampling is currently effective when the combination space is small (3-4 modalities). As the number of modalities \(M\) scales up, estimating sampling probabilities for the \(2^M-1\) combinations becomes sparse. One solution is to model the "missing probability" of each individual modality rather than combinations, or to introduce similarity priors across combinations to share sampling statistics.

Related Work & Insights

• vs DMRNet: DMRNet enhances robustness to missing modalities through probabilistic uncertainty modeling but still operates strictly on \(z^{\text{partial}}\), failing to explicitly utilize the complete representation. The residual perspective of MARS is a fundamental paradigm shift — moving from "modeling uncertainty on degraded representations" to "using complete representations to supervise the shift patterns of degraded representations." DMRNet's uncertainty modeling could be integrated with MARS's feature router, permitting the inference router to concurrently output both "which expert to trust" and "how confident it is."
• vs MoMKE / SimMLM: These two methods strictly bind experts to modalities (one expert per modality), limiting the flexibility of the expert count and requiring two-stage training. In contrast, MARS's residual routing allows experts to specialize naturally according to "representation shift patterns," where the number of experts is independent of the number of modalities, and a single expert can serve multiple modality combinations (if they exhibit similar shift patterns). This philosophy of "specialization by demand rather than by modality" is much more flexible.
• vs Flex-MoE: Flex-MoE defines experts based on modality combinations (requiring \(2^M-1\) experts), which essentially replaces routing learning with look-up tables. MARS demonstrates that different combinations can share similar shift patterns (e.g., RGB-only and RGB+IR may have similar shift directions in certain dimensions), making learning these patterns through residual routing more efficient than predefined combinations.
• vs Learning Using Privileged Information (LUPI): The dual-router design in MARS is highly aligned with the LUPI paradigm — using privileged information (\(z^{\text{full}}\)) to compute residuals for guiding specialization during training, while relying solely on regular information (\(z^{\text{partial}}\)) during inference. The core innovation lies in applying privileged information to routing instead of directly to prediction, showcasing a novel integration within the MoE framework.

Rating

• Novelty: ⭐⭐⭐⭐⭐ — Re-framing the IML problem from "robust modeling against missing modalities" to "representation shift-guided expert specialization," and introducing the residual signal along with the dual-router design, represents an original shift in perspective.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ — Four datasets cover classification and segmentation across vision, audio, and text. The ablation study validates each module systematically; expert deactivation experiments verify the specialized structure; and routing distribution and Grad-CAM visualizations provide qualitative analysis. Furthermore, the appendix evaluates various alternative options for residuals.
• Writing Quality: ⭐⭐⭐⭐ — The logic of the method is clear, and the motivational diagram in Fig. 1 effectively conveys the intuition of the residual perspective. The appendix provides an in-depth justification of the residual's rationality (analogizing with Word2Vec/GAN, etc.). A minor shortcoming is that equation numbers/formatting are slightly untidy in some sections, and some symbol representations assume reader familiarity with MoEs.
• Value: ⭐⭐⭐⭐⭐ — The combination of residual routing + dual-router distillation + noise regularization forms a transferable, general framework suitable for any scenario characterized by "complete information during training but incomplete during inference." The paper also demonstrates that this framework yields significant performance gains under minuscule parameter and computational overhead (comparable to DMRNet), showcasing strong practical deployment value.

Related Papers

• [CVPR 2025] SGMA: Semantic-Guided Modality-Aware Segmentation for Remote Sensing with Incomplete Multimodal Data
• [CVPR 2026] ConceptPrism: Concept Disentanglement in Personalized Diffusion Models via Residual Token Optimization
• [NeurIPS 2025] SAM-R1: Leveraging SAM for Reward Feedback in Multimodal Segmentation via Reinforcement Learning
• [ICCV 2025] HiMTok: Learning Hierarchical Mask Tokens for Image Segmentation with Large Multimodal Model
• [CVPR 2026] PixDLM: A Dual-Path Multimodal Language Model for UAV Reasoning Segmentation