Skip to content

Boosting Domain Incremental Learning: Selecting the Optimal Parameters Is All You Need

Conference: CVPR 2025
arXiv: 2505.23744
Code: None
Area: Object Detection
Keywords: domain incremental learning, parameter selection, continual learning, object detection

TL;DR

Discovers that selecting the optimal subset of parameters is more effective than fine-tuning all parameters in domain incremental learning, and proposes a parameter selection strategy to resolve catastrophic forgetting in domain incremental object detection.

Background & Motivation

Background

Background: The field of Boosting Domain Incremental Learning has made significant progress in recent years, but key challenges remain.

Limitations of Prior Work: Existing methods have deficiencies in generalization, efficiency, or robustness, limiting practical applications. Specifically, most methods work under specific assumptions and struggle to handle real-world diversity.

Key Challenge: The trade-off between performance and efficiency/generalization is the core challenge. There is a need to improve the practicality of the model while maintaining high performance.

Goal: To design a more efficient, robust, and generalized solution to overcome the aforementioned limitations.

Key Insight: Analyzing the contributions of parameters in different layers to domain adaptation and forgetting, and selectively freezing or updating parameters.

Core Idea: Discovering that selecting the optimal subset of parameters is more effective than fine-tuning all parameters in domain incremental learning.

Method

Overall Architecture

Analyzes the contributions of parameters in different layers to domain adaptation and forgetting, and selectively freezes or updates parameters. This is combined with lightweight domain-specific adapters to ensure adaptation to new domains.

Key Designs

  1. Core Module

    • Function: Implements the core functionality of the method
    • Mechanism: Analyzes the contributions of parameters in different layers to domain adaptation and forgetting, and selectively freezes or updates parameters
    • Design Motivation: Addresses the core limitations of existing methods

  2. Auxiliary Module

    • Function: Enhances the effectiveness of the core module
    • Mechanism: Improves performance through additional constraints or information
    • Design Motivation: Compensates for the deficiencies of the core module when used alone

  3. Optimization Strategy

    • Function: Improves training stability and convergence speed
    • Mechanism: Employs appropriate learning rate scheduling, gradient clipping, and regularization strategies
    • Design Motivation: Ensures training efficiency of the model on large-scale data

Implementation Details

  • The framework is implemented based on PyTorch.
  • Standard data augmentation strategies are used to improve generalization.
  • Both training and inference are executed efficiently on GPUs.

Loss & Training

  • Integrates loss functions from multiple objectives to balance various aspects of performance.

Key Experimental Results

Main Results

Method Key Metrics Description
Baseline Method Lower Has limitations
Ours Higher Achieves better performance on multiple domain incremental detection benchmarks with fewer trainable parameters

Ablation Study

Component Effect
Core Module Major contribution
Auxiliary Module Additional improvement
Full Best

Key Findings

  • Achieves better performance on multiple domain incremental detection benchmarks with fewer trainable parameters.
  • The components are complementary and indispensable.

Highlights & Insights

  • The design insight that selecting the optimal subset of parameters is more effective than fine-tuning all parameters in domain incremental learning is novel.
  • Demonstrates potential for application in practical scenarios.
  • The framework is generalizable and can be extended to related tasks.

Limitations & Future Work

  • Validation on more datasets and scenarios.
  • Computational efficiency can be further optimized.
  • The complementarity with other methods is worth exploring.
  • Compared with existing representative methods, the proposed method has distinct advantages in key metrics.
  • The proposed ideas can inspire research in related fields.

Rating

  • Novelty: ⭐⭐⭐⭐ Innovative core idea
  • Experimental Thoroughness: ⭐⭐⭐⭐ Multi-benchmark evaluation
  • Writing Quality: ⭐⭐⭐⭐ Clear structure
  • Value: ⭐⭐⭐⭐ Has practical application prospects