ECCV2026 Model Compression AI paper notes paper summaries Knowledge Distillation Compression Continual Learning Adversarial Robustness Multimodal/VLM

📦 Model Compression¶

🎞️ ECCV2026 · 89 paper notes

📌 Same area in other venues: 📷 CVPR2026 (108) · 🔬 ICLR2026 (240) · 💬 ACL2026 (59) · 🧪 ICML2026 (117) · 🤖 AAAI2026 (60) · 🧠 NeurIPS2025 (143)

🔥 Top topics: Knowledge Distillation ×9 · Model Compression ×9 · Compression ×7 · Continual Learning ×7 · Adversarial Robustness ×7

GenRecal: Generation after Recalibration from Large to Small Vision-Language Models: GenRecal uses a training-time Recalibrator to connect a small vision-language model's hidden features to the teacher's language head, bypassing direct cross-tokenizer correspondence and improving InternVL2.5-8B from 56.0 to 68.1 on MMMU, while removing all auxiliary modules for deployment and retaining the student's original architecture.
Towards Consistent and Efficient Dataset Distillation via Diffusion-Driven Selection: This work turns a pre-trained diffusion model from an image generator into a real-image patch selector, combining class-conditioned localization with DIFT clustering to build reusable compressed training sets and achieving \(61.0\pm0.3\%\) classification accuracy on ImageNet-1K at IPC=50 in the main experiment.
Enhancing Pretrained Model-based Continual Representation Learning via Guided Random Projection: SCL-MGSM uses first-session labels and residuals to select blocks of random projection bases, then freezes the representation and recursively updates a ridge-regression classifier, improving expressivity and numerical stability in exemplar-free class-incremental learning and reaching 77.12% final accuracy on ImageNet-R B-0 Inc-10.
Quick ViTs: Speeding up Vision Transformers through Equivariance: Quick ViTs turns rotation and reflection equivariance into structured sparsity and weight sharing in Fourier-domain linear layers, allowing H8(ViT-H/14) to reach 85.0% rather than 84.6% supervised ImageNet-1K Top-1 accuracy with 102.3G rather than 167.8G FLOPs, while explaining why theoretical FLOP savings do not imply an equal runtime speedup.
Audio-Visual Continual Test-Time Adaptation without Forgetting: AVReCAP retrieves historical fusion-layer parameters using input statistics instead of continually overwriting one parameter state, achieving 43.51% mean accuracy on VGGSound-2C, 6.28 percentage points above the source model, while substantially reducing source-domain forgetting.
LANCE: Low Rank Activation Compression for Efficient On-Device Continual Learning: LANCE runs a single higher-order singular value decomposition (HOSVD) before fine-tuning to calibrate a fixed low-rank projection subspace per layer activation; during training it stores only the low-rank core tensor and reprojects it in the backward pass with the same orthonormal factors, cutting peak activation memory to 1/25–1/380 of full backpropagation, and the same fixed subspaces can be partitioned across tasks into mutually orthogonal null spaces at offline calibration time, enabling on-device continual learning without replay buffers or resident task-specific matrices.
Pathryoshka: Compressing Pathology Foundation Models via Multi-Teacher Knowledge Distillation with Nested Embeddings: Pathryoshka compresses global and spatial knowledge from three large pathology foundation models into a prefix-truncatable student, achieving a median accuracy of 85.6 across ten classification benchmarks with 86M parameters while reducing encoding and feature-storage costs.
Shared LoRA Subspaces for almost Strict Continual Learning: Share integrates incoming data or LoRA adapters into an evolving shared basis and replaces separate adapters with lightweight task coefficients, reaching 94.20% final accuracy in replay-free CIFAR-100 class-incremental learning; however, its general formulation retains task coefficients and Share-full permits old-data access, so not all results represent strict continual learning.
Stay Unique, Stay Efficient: Preserving Model Personality in Multi-Task Merging: Rather than forcing every task to use identical weights, DTS stores reconstructable task residuals through low-rank decomposition and four-group thresholding, achieving 90.17% average accuracy in eight-task ViT-B/32 merging with 0.98% additional storage per task and combining residuals by task semantic similarity for unseen tasks.
C2E: Boosting Ego-Only 3D Object Detection via Multi-Teacher Contrastive Knowledge Distillation: C2E uses the M2S multi-teacher distillation framework to transfer dense knowledge from multi-agent training observations into a student receiving only ego LiDAR, raising AttFuse's V2XSet test [email protected] from 58.06 to 66.70 without inter-agent communication at deployment.
CL-Anomaly: Layer-Adaptive Mixture-of-Experts with Multimodal Large Language Model for Continual Learning in Anomaly Detection: CL-Anomaly adds parallel task-private LoRA and shared experts to a frozen multimodal large language model, then merges old and new shared parameters according to layer-wise task relevance, achieving 85.61% final average accuracy and −0.96 percentage points of backward transfer on an eight-task anomaly detection sequence; its primary contribution is reducing forgetting rather than introducing a new anomaly localizer.
Cross-Resolution Distribution Matching for Diffusion Distillation: The paper proposes RMD, which treats resolution reduction as a second acceleration axis alongside step reduction: it splits the denoising trajectory with logSNR curves and assigns each segment a resolution, projects low-resolution generator samples into the teacher's high-resolution space through a differentiable upsampling transform, and performs DMD-style distribution matching there, with predicted-noise re-injection for stable training — reaching up to 33.4× speedup on SDXL at 2+2 steps and 25.6× on Wan2.1-14B at 3+3 steps while preserving fidelity.
DiffPro: Joint Timestep and Layer-Wise Precision Optimization for Efficient Diffusion Inference: DiffPro uses calibration data to identify quantization-sensitive layers and denoising steps, then jointly selects layer precision, dynamic activation quantization, and retained timesteps, reducing DiT-XL/2 from 2575 MB to 327 MB and A30 latency from 6.88 to 2.55 seconds/image while FID increases from 2.27 to 5.89.
FastSTAR: Spatiotemporal Token Pruning for Efficient Autoregressive Video Synthesis: FastSTAR jointly uses spatial and temporal feature changes to select tokens for computation during InfinityStar's high-resolution refinement, while preventing residual additions at skipped positions, reducing end-to-end 720p text-to-video latency from 81.7 to 40.6 seconds as the VBench total score changes from 83.89 to 83.18.
From Draft to Draft-Free: One-Step Video Object Removal via Privileged Distillation and Fast Planting: D2DF first trains a multi-step teacher to refine video removal drafts, distills a one-step student using a ground-truth prior visible only to the teacher, and finally replaces the external inpainter with lightweight latent pseudo-drafts; its draft-free version achieves 31.56 dB PSNR on RORD with 1.05 seconds for prior extraction plus denoising, excluding latent encoding and decoding.
Improving Knowledge Distillation Under Unknown Covariate Shift Through Confidence-Guided Data Augmentation: ConfiG combines inter-class diffusion editing with teacher-student confidence disagreement to generate examples the student has not mastered, improving distillation without knowing the spurious attributes and raising CelebA-HQ worst-group accuracy from 53.44% with real data alone to 88.15%.
Learning to Recover Task Experts from a Multi-Task Merged Model: ReTeX treats model-merging interference as a learnable parameter offset, identifies tasks through SVD subspaces, and recovers experts on demand, reaching 92.2% average accuracy on eight ViT-B/32 tasks without storing full expert checkpoints at inference.
MambaRaw: Selective State Space Modeling for Efficient 4K Raw Image Reconstruction: MambaRaw introduces State Space Models (SSM/Mamba) into the entropy parameter estimation of the JPEG-guided metadata RAW image reconstruction framework. Through two lightweight modules—energy-guided tile selective scanning (TileMambaBlock) and energy-aware feature refinement (EAR)—the approach simultaneously improves reconstruction quality (PSNR increases by 1.2–1.4 dB) and reduces coding latency (by approximately 9%) at 4K resolution.
Neutralizing Token Aggregation via Information Augmentation for Efficient Test-Time Adaptation: The paper formalizes plugging token aggregation into Test-Time Adaptation (TTA) to save compute as Efficient Test-Time Adaptation (ETTA), uses a mutual-information analysis to show that aggregation is a many-to-one mapping whose information loss cannot be recovered by the LayerNorm tuning that TTA usually relies on, and proposes NAVIA: two learnable information carriers — an input-level [CLS] embedding augmentation and a feature-level [CLS] bias augmentation — are attached to the [CLS] token before aggregation. With only a handful of extra trainable parameters (5.3K more than pure LayerNorm tuning on ViT-B/16), NAVIA beats the TTA state of the art (FOA) by up to 3.1% across four OOD benchmarks while achieving 14%–26% wall-clock speedup.
ProGVC: Progressive-based Generative Video Compression via Auto-Regressive Context Modeling: ProGVC ports the next-scale prediction of Visual Auto-Regressive (VAR) models to video compression: a multi-scale residual quantizer turns video latents into a coarse-to-fine token pyramid, the encoder transmits all intra scales plus the first k inter scales, and the truncated high-frequency scales are generated on the fly by the same autoregressive context model — yielding a scalable bitstream, adaptive rate control, and competitive perceptual quality in the 300–2000 kbps regime.
Puppet-CNN: Continuous Parameter Dynamics for Input-Adaptive Convolutional Networks: Puppet-CNN treats the convolution kernels of an entire CNN backbone as samples along one continuous trajectory in parameter space, evolved by a tiny "puppeteer" neural ODE over a normalized coordinate \(s\in[0,1]\), so that network depth becomes the sampling resolution \(\lfloor 1/\Delta s\rfloor\) and the trajectory's initial state and step size are both driven by an entropy-based input complexity; on CIFAR-10 it reaches 72.51% Top-1 with 1.08 trainable parameters (the paper labels the column "MB", but the magnitudes indicate a parameter count in millions, ⚠️ refer to the original paper), while sample-wise parameter and depth adaptation fall out of the dynamical system itself rather than from an add-on controller.
Rapidly Deploying On-Device Eye Tracking by Distilling Visual Foundation Models: DistillGaze first self-distills DINOv3 into a domain-specialized 86M teacher using labeled synthetic eye images plus unlabeled real infrared images, then distills that teacher into a 256K on-device student, using no real gaze labels at all; on the Project Aria crowd-sourced benchmark it cuts median gaze error from 3.48° to 1.44° (-58.6%) and tail error E90U90 from 14.84° to 8.45° (-43.1%) with no increase in model size.
Robust Trajectory Distillation: Hybrid Reweighting Meets Teacher-Inspired Targets: On top of a trajectory-matching dataset distillation backbone (DATM), this paper adds two modules — Selective Guidance Reweighting (SGR), which fuses dynamic KNN neighborhood consistency with static second-split-forgetting timestamps through a curriculum-style convex interpolation to purify the teacher's parameter trajectory, and Teacher-Inspired Auxiliary Targets (TIAT), which derive an auxiliary alignment target from a denoised teacher checkpoint fine-tuned on a high-confidence subset — yielding distilled datasets that stay robust under symmetric, asymmetric, and real-world label noise, with gains of up to 4.7–6.6 points over DATM on CIFAR-10 at 40% symmetric noise.
Structural Assessment for Understanding and Guiding Dataset Distillation in Discrete Token Space: This work proposes a Structural Score to evaluate the quality of distilled datasets through statistical analysis in the discrete visual token space. Based on this, a Token-Guided Dataset Distillation (TGDD) framework is developed, which utilizes the Structural Score to guide diffusion models for generating high-quality distilled data.
Towards Memory-Efficient Autoregressive Video Generation via Instance-Specific Parametric Absorption: ISPA fits the difference between local and full attention during a short video-generation warmup and absorbs historical effects into output projections, allowing LongLive ISPA-0.5 to retain only 54.2% of its KV cache with subject consistency changing from 96.10 to 95.61 in the 30-second evaluation, without implying strictly lossless behavior across all models and metrics.
Training-Free Task Classification for Multi-Task Model Merging: SiM identifies a test input's task using low-rank feature subspaces built from small task support sets, then selects a compressed task vector, reaching 92.7% average accuracy on eight tasks with CLIP ViT-B/32 versus 92.8% for individual experts, without router training or test-time task IDs.
ZipDepth: Bringing Lightweight Zero-Shot Monocular Depth Anywhere, on Any Device: ZipDepth distills the multi-domain depth knowledge of Depth Anything v2-Large into a convolutional student of about 6.1M parameters, pairing a half-resolution detail path with hardware-adaptive upsampling to preserve boundaries; on a Jetson Orin NX at 15 W in FP32 it reaches 34.4 FPS at 396.6 mJ per frame, though accuracy still trails the teacher and the output remains relative depth only.
AVQ-Attention: Adaptive Vector-Quantized Attention: AVQ-Attention starts with coarse codeword attention and selectively activates learned children for the parents receiving the most attention within each query tile; fused precomputation and incremental correction improve the speed–quality trade-off, reaching 43.33% mIoU in a controlled ADE20K comparison, still below exact attention's 49.0%.
Boosting Text-Driven Video Segmentation via Geometry-Aware Distillation: Built on SAMWISE, GeoLaV first pretrains on continuous novel views synthesized from single images, then distills semantic and 3D geometric teachers on real videos, reaching 70.5 J&F on Ref-YouTube-VOS—1.3 points above the equally sized 202M-parameter SAMWISE—without retaining teachers or projection heads at inference.
Breaking the Model Forgetting Cycle in Long-Incremental 3D Object Detection: LDMR turns changes in historical-scene recall into decisions about which samples need more review now and which deserve storage for the next stage, reaching 19.38 [email protected] on the ten-stage SUN RGB-D protocol with TR3D, 3.84 points above the equal-budget SDCoT++ random-memory baseline.
CloSeR: Unified Relational Distillation from Closed-Set Teachers for Category Discovery: Instead of asking a closed-set teacher to name unknown categories, CloSeR transfers two kinds of structure—how samples relate to known classes and how samples relate to one another—raising SelEx's average fine-grained All ACC from 63.1 to 72.0 with DINO while supporting both classifier-based and clustering-based students.
Concept-to-Pixel: Prompt-Free Universal Medical Image Segmentation: C2P distills model-generated medical semantics into semantic tokens, supervises geometric tokens with mask attributes, and uses them to generate image-specific convolution kernels and geometry-consistency fusion weights, achieving 88.22% average Dice across eight datasets with one model and no manual prompts or reference images, but without robust transfer to every structural category.
Condensing Large-Scale Datasets Directly with Minimal Information Loss: This paper points out that the "data -> model -> image" dual compression process in mainstream large-scale dataset distillation (SRe2L-based family) causes severe information loss and shifts distilled images away from the real distribution, thereby undermining Relabeling. To address this, the authors propose CIM, which uses a computable "effective information gap" metric to directly minimize the information discrepancy between the synthetic and real sets on the original images, bypassing the expensive recovery stage. Consequently, CIM achieves 48.7% Top-1 accuracy (ResNet-18) on ImageNet-1K with IPC=10 in just 80 minutes on a single GPU.
Cross-Space Distillation: Teaching One-Step Students with Modern Diffusion Teachers: Proposes the Cross-Space Distillation framework with a lightweight Bridge module that combines a frozen student VAE decoder prefix as a spatial prior with a SwinIR projector, overcoming latent resolution and VAE mismatches to enable compact SD 1.5 to achieve teacher-level generation quality in a single step.
DA-MergeLoRA: Hypernetwork-Based LoRA Merging for Few-Shot Test-Time Domain Adaptation: Addressing the strict constraints of few-shot test-time domain adaptation where target data is extremely scarce and gradient backpropagation is prohibited, DA-MergeLoRA fine-tunes independent source-domain LoRA modules on frozen CLIP and uses a meta-trained cross-attention hypernetwork to dynamically generate column-wise merge weights, synthesizing a specialized target model in parameter space with zero test-time gradient updates.
DeCoFlow: Structural Decomposition of Normalizing Flows for Continual Anomaly Detection: DeCoFlow decomposes the scale and shift subnets of normalizing flows into a frozen base and independent task adapters, then compensates for restricted adaptation through input alignment, auxiliary coupling, and tail weighting, achieving image-level AUROC of 98.40% / 93.00% on MVTec-AD / VisA in replay-free sequential class learning, although its zero-forgetting property still requires correct routing.
Denoised Variance-Based Pruning with Optimal Brain Bias Compensation: DVBP + OB2C selects MLP neurons using a mixture of denoised and raw activation variance, then solves for weight and bias compensation using centered covariance; with 50% of MLP hidden neurons pruned and no fine-tuning, it raises Swin-S ImageNet-1K Top-1 accuracy from VBP's 69.91% to 77.24%.
Describe-Then-Act: Proactive Agent Steering via Distilled Language-Action World Models: Addressing the prohibitive latency of conventional visual world models in real-time control, this paper introduces DILLO, a vision-free Language-Action World Model that predicts semantic outcome descriptions and binary safety verdicts directly from policy latents and proposed action chunks via cross-modal distillation, delivering a 14× speedup and substantial proactive steering gains.
Distill on a Diet: Efficient Knowledge Distillation via Learnable Data Pruning: This paper proposes the IF-Beta framework, which uses influence functions (IF) as a retraining-free sample importance scorer. Combined with a learnable Beta distribution sampling strategy, it efficiently searches for the optimal pruning subset via bi-level optimization in the feature space, enabling the student to outperform full-dataset distillation with less data and computation.
Distill Once, Adapt Life-Long: Exploring Dataset Distillation for Continual Test-Time Adaptation: Before deployment, DO-ALL uses dataset distillation (DD) to compress the source domain into a small set of synthetic anchors (each containing synthetic samples, soft labels from the source model, and latent features). During testing, it matches the semantically closest anchor for each target sample via feature space cosine similarity. It then stabilizes model updates using anchor replay, target-anchor MixUp regularization, and multi-layer MMD feature alignment, supplemented by a harm-adaptive blending mechanism that selectively rolls back to the source model initialization based on the gradient harm level of parameter groups. In a plug-and-play manner, DO-ALL consistently improves the long-term robustness of various CTTA methods, such as EATA, RMT, ROID, and ASR, on the ImageNet-C and CCC benchmarks.
Ex-Sim(3)-Reg: 2D-3D Correspondence Pruning via Extended Sim(3) Registration: Addressing the vulnerability of depth-prior-based 2D-3D correspondence pruning to scale noise, this paper reformulates pruning as an extended Sim(3) registration problem with correspondence-specific scale perturbations, and introduces Ex-Sim(3)-Reg to convert it into a high-inlier SE(3) registration via scale-parameterized compatibility graphs, achieving up to 24.7% improvement in registration recall.
Fisher-Routed Mixture of Experts for Federated Class-Incremental Learning: FedFMX selects sample-specific expert subsets during training using forgetting risk and learning gain, then distills a label-free inference gate; with ResNet-18, fine-grained increments, and Dirichlet label imbalance, it achieves 50.32% accuracy on CIFAR-100 versus 46.27% for FedCBDR.
HoloTetSphere: Unified TetSphere Mesh Reconstruction for Physical Simulations: Addressing the error-prone nature of decoupled two-stage surface extraction and tetrahedralization pipelines, as well as the topological inflexibility and disjoint fragmentation caused by homeomorphic constraints in existing Lagrangian volumetric primitives, HoloTetSphere introduces a tightly coupled tetrahedral-Gaussian representation that achieves end-to-end differentiable element pruning via a vertex-shared continuous opacity field and alternates geometric refinement with two-stage HC-Laplacian smoothing to reconstruct high-fidelity, single-connected tetrahedral meshes ready for physical simulations.
IDeaL: Data-Free Multi-Teacher Distillation via Improved Dead Leaves: Addressing the challenge of multi-teacher distillation in the absence of real training data, IDeaL optimizes procedural Dead Leaves noise via patch- and image-level decorrelation losses through frozen ViT teachers, substantially narrowing the performance gap with real-image distillation without requiring any real samples or class labels.
Learning Semantic-Robust Change Detection via Semantic-Invariant Self-Distillation: To alleviate false alarms and missed detections caused by non-semantic environmental disturbances (e.g., illumination, shadows, and weather) in bitemporal remote sensing change detection, this paper introduces SCDistill, which simulates realistic appearance variations via diffusion models and guides a vision foundation model encoder via semantic-invariant self-distillation, achieving state-of-the-art results across semantic change detection, binary change detection, and change captioning tasks.
Learning to Tessellate: Point Cloud Generation via Recursive Spectral Partitioning: PointRSP reformulates 3D point cloud generation as a topology-preserving spatial tessellation process via recursive spectral partitioning, building a non-balanced binary tree geometric blueprint and a dual-stream cascaded generator that significantly eliminates fragmentation artifacts and sets new SOTA generation benchmarks.
Low-Rank Ternary Adaptation for Fine-Tuning Transformers: To address the fundamental inability of existing low-bit LoRA methods to adapt ternary weights without breaking discrete domain constraints upon merging, this paper proposes a low-rank ternary multiplicative adapter via Kronecker factorization that performs discrete keep/zero/flip updates directly within \(\{-1, 0, 1\}\), achieving dequantization-free merging into pure 1.58-bit models.
MixTTA: Low-Rank Cross-Channel Mixing for Reliable Test-Time Adaptation: On top of the channel-wise affine transformations of normalization layers, MixTTA inserts a low-rank residual cross-channel mixing module to enable test-time adaptation (TTA) to correct variations in the inter-channel correlation structure caused by distribution shifts. Combined with two regularization methods—decoupled projection and spectral projection—to prevent diagonal leakage and rank-one collapse, it provides a plug-and-play boost to the accuracy and stability of five major baselines (Tent, EATA, SAR, DeYO, and ReCAP) in both standard and wild TTA scenarios.
Moebius: 0.2B Lightweight Image Inpainting Framework with 10B-Level Performance: Moebius combines a lightweight LλMI backbone with adaptive multi-granularity latent distillation to build a 0.226B-parameter inpainting specialist, achieving FID 0.92 and LPIPS 0.091 on Places2 with small masks; its approximately 15.5-fold total-time advantage over FLUX.1-Fill-Dev comes from both faster individual steps and fewer default sampling steps.
MVPruner: Dynamic Token Pruning for Accelerating Multi-view Vision-Language Models in Autonomous Driving: MVPruner prunes multi-view vision-language models in two stages—first preserving rich scene information, then focusing on the current instruction—and, at 90% pruning with DriveMM on DriveLM, retains an aggregate score of 58.2 versus 59.1 while reducing prefilling latency from 1968 ms to 395.6 ms; this does not imply a 4.97-fold end-to-end speedup.
On the Vulnerability of Parameter-Level Defenses to Model Merging: This paper exposes a fundamental vulnerability of parameter-level defenses that use linear transformations to make models "unmergeable"—namely, the protected task vector is 2 to 3 orders of magnitude smaller than the pre-trained anchor. Consequently, this work proposes the Anchor-Guided Attack (AGA). By treating the public pre-trained model as a static anchor, AGA analytically reconstructs the hidden transformation matrix using least squares and the Hungarian algorithm, recovering the protected model almost losslessly. To counter this, an attack-resistant defense named Anchor-Repulsive Fine-tuning (ARF) is introduced as a remedy.
RaysUp: Ultra-light Universal Feature Upsampling via Geometry-Aware Ray Representation: An ultra-lightweight, task-agnostic, and VFM-agnostic universal feature upsampling framework named RaysUp with only 0.14M parameters is proposed. Through three key designs—spatial decoupled guidance encoder, Ray Positioning Encoding (RayPE), and geometry-aware neighborhood cross-attention—it lifts feature reconstruction from the 2D pixel plane to the 3D ray domain. It achieves state-of-the-art (SOTA) or near-SOTA performance across various dense prediction tasks such as semantic segmentation, depth/normal estimation, video object segmentation, and open-vocabulary segmentation, while offering an inference speed approximately 7 times faster than AnyUp, the only prior VFM-agnostic method.
Self-Improving Diffusion Classifiers with Minority Preference Optimization: MiPO introduces a self-improving framework that broadens diffusion classifiers' perceptual coverage over low-density manifold regions using only arbitrary text prompts and Tweedie-based minority rewards, optimized via selective Group Relative Policy Optimization and LoRA without requiring downstream images or external reward models.
SFKD: Spatial–Frequency Joint-Aware Heterogeneous Knowledge Distillation via Multi-Level Wavelet Spectral Interaction: To resolve representational spatial distribution discrepancies caused by divergent inductive biases across heterogeneous architectures (CNN, Transformer, MLP-Mixer), SFKD couples multi-level discrete wavelet spatial decoupling, dual-stream dual-stage spectral refinement, and Gaussian-filtered Fourier frequency contrastive loss to preserve and transfer global structures and local details without discarding spatial information.
Structured Hyperedge Adaptation for Parameter-Efficient Fine-Tuning of Vision Transformers: This work shifts the adapter fine-tuning of ViTs from "individual token-wise updates" to a "hyperedge space." First, patch tokens are softly routed into several groups (hyperedges) using learnable prototypes. Then, low-rank bottleneck adaptation is conducted on these hyperedges, and finally, the updates are diffused back to the tokens. This achieves a significant accuracy boost in structured reasoning tasks on VTAB-1K with a minimal parameter footprint (<0.5% of the backbone).
TextDS: Parameter-Efficient Representation Alignment for Scene Text Detection under Distribution Shifts: TextDS adopts a dual-encoder architecture featuring SAM2 and DINOv3, achieving robust scene text detection without large-scale scene text pre-training through Step-Wise Low-Rank Adaptation (SWLoRA) and Common Subspace Fusion (CSF). With only 4.9M parameters, it achieves leading performance under domain shifts and image degradations such as rain, fog, exposure, and low resolution.
VICAL: Vicinal Consistency Alignment for Long-Tailed Visual Recognition: Challenging the prevailing dogma that multi-expert long-tailed recognition fundamentally requires expert diversity, VICAL introduces frequency-decoupled vicinal consistency alignment—smoothing tail-class loss landscapes via intra-expert interpolated self-consistency while enforcing low-frequency semantic consensus and conflicting knowledge filtering across experts, reducing prediction variance without optimization conflicts.
WinTok: A Win-Win Hybrid Tokenizer via Decomposing Visual Understanding and Generation with Transferable Tokens: WinTok presents a hybrid visual tokenizer that incorporates learnable semantic tokens into a unified ViT encoder alongside asymmetric distillation from foundation models, fundamentally decoupling visual understanding from generation to attain an 82.0% ImageNet-1K classification accuracy and an rFID of 0.41 using only 50M training images.
Accurate Zero-shot Quantization via Hierarchical Teacher-Assistant Distillation: Zest connects a full-precision CNN to a low-bit student without original training images through an intermediate-precision teacher assistant, quantization-aware image recalibration, and block-wise reconstruction, reaching 70.50% ImageNet-1K Top-1 accuracy for ResNet-18 W4A4, 0.51 percentage points below the reported full-precision baseline.
Benchmarking Federated Learning & Knowledge Distillation for Point Cloud Classification: A benchmark of federated teachers and distillation objectives shows that labeled proxy data can produce a 92.94% student from an 8.50% teacher, so a compact student's high accuracy does not necessarily demonstrate successful transfer from federated training.
CaPCL: Caption-Preserved Continual Learning for Text-to-Image Retrieval: When learning a new image-text retrieval task, CaPCL requires the model to reproduce auxiliary captions generated for current images before the update, protecting prior semantics omitted by new queries; averaged over four BLIP-2 task orders, it reduces ModX's forgetting measure FM from 0.0890 to 0.0654 while raising AA from 0.6387 to 0.6942.
CARA: Collision-Aware Resolution Adaptation for Multiresolution Hash Encoding Based Image Fitting: Addressing the capacity bottlenecks and parameter inefficiencies caused by data-agnostic geometric resolution schedules and hash collisions in Instant-NGP, CARA quantifies band-wise effective information using empirical entropy coupled with a hash collision attenuation factor, solves for a min-max balanced resolution schedule via dynamic programming, and incorporates an invertible pixel-shuffle transform to lower the load factor, matching SOTA fidelity with only 27.76% of the parameters and gaining up to 6.11 dB PSNR under comparable budgets.
Causal Yet Future-Aware: Dual-Path Temporal Modeling for Online Action Segmentation: DPTM explicitly separates historical and future dependencies in an offline teacher, then trains a branch that reads only recent history to imitate future representations, reducing fragmented online action predictions without accessing future inputs; with ASFormer, GTEA Edit rises from 73.2 for the causal baseline to 80.0.
Don’t Teach Instability, Teach Robustness: Selective Sensitivity Gating for Adversarial Distillation: Addressing the "stability punishment" phenomenon where standard Adversarial Robustness Distillation blindly mimics the teacher's jagged decision boundaries, this paper proposes AEGIS to decouple supervision via semantic logit gating and sensitivity calibration gating, boosting ResNet-18 AutoAttack accuracy to 53.22% on CIFAR-10 (a +3.12% gain over ARD) while preserving ~50% accuracy under severe \(\epsilon=16/255\) perturbations.
EAGS: Error-Aware Gaussian Splatting with Dual-Confidence-Guided Modeling for Uncalibrated Driving Scenes: EAGS reformulates feed-forward 3D Gaussian Splatting reconstruction from uncalibrated driving images as residual-driven generation and geometry-aware retention, using distillation to constrain the pruned representation and achieving 28.02 dB PSNR with 49,556 Gaussians per view on Waymo, versus DGGT's 27.41 dB and 152,292.
EffiDINO: Task-Specific Model Pruning via Gram Anchoring Subspace Consistency: Addressing the issues of manifold collapse caused by rigid point-to-point token alignment and the lack of task specificity in generic pruning pipelines, Cut-ViT (EffiDINO) decouples spatial and semantic topologies via Gram anchoring, enforces basis-invariant and residual subspace constraints, and introduces spectral entropy weighting to dynamically tailor pruning to downstream tasks—obtaining SOTA subnetworks across diverse sparsity levels in approximately one minute on a single A100 GPU.
Enhanced Neural Video Representation Compression with High Scalability: This paper introduces NVRC++, an implicit neural representation framework for video compression that employs high-resolution multi-scale feature grids, an in-parameter hierarchical B-frame coding structure with unconditional quantization, feature grid annealing masks, and a multi-prior entropy model, achieving constant decoding complexity across wide bitrate ranges and up to 7.6x faster decoding speed than NVRC.
Flash-DD: An Ultra Parameter-Efficient Approach to Dataset Distillation: To resolve the excessive storage, slow online inference, and suboptimal generalization caused by full-sized teacher models in large-scale dataset distillation, Flash-DD introduces dataset-distillation-oriented model parameter reduction and adaptive ensemble label generation, cutting extra storage down to 0.03% with an 843.81× speedup, while improving accuracy by up to 13.4% under a 1.8% storage budget.
From Predictions to Embeddings: Dual Knowledge Distillation for Instance-Dependent Partial Label Learning: Addressing candidate label semantic confusion and dark knowledge distortion in instance-dependent partial label learning (IDPLL), DKDP introduces a dual distillation framework that structurally rectifies teacher predictions via minimal-perturbation closed-form constrained projection and extends reliable supervision into the embedding space using online class prototypes.
GCMRD: Global Consistency Multi-teacher Robustness Distillation: To tackle supervision underutilization and task conflict in dual-teacher adversarial robustness distillation, GCMRD refines the paradigm via collaborative soft-label attack generation, bidirectional full-repulsion regularization, and natural-teacher-anchored robust teacher fine-tuning, pushing the Pareto frontier of lightweight student models.
Identifiable Gated Residual Personalization for Federated Parameter-Efficient Fine-Tuning: Addressing the intrinsic scale non-identifiability in gated residual mixing for federated PEFT, FedSDG introduces structure-decoupled dual-path LoRA, projection-level scalar gating, and backbone-anchored Dynamic Alignment to enable depth-aware, scale-identifiable personalization without additional communication.
Interference-Aware Continual Vision–Language Learning via Instance-Level Expert Routing: IAR addresses catastrophic forgetting in continual vision-language learning by turning cross-task parameter conflict into an instance-level signal via Fisher-whitened projection energy, routing inputs to least-interfering experts with subspace packing and expanding rank along residual principal directions.
Learning Egocentric Cues from Exocentric Video using Privileged Egocentric Supervision: Ego2ExoVLM teaches an exocentric-only student to reproduce an egocentric teacher's answers and uses adaptive visual tokens to extract interaction cues that would otherwise be overwhelmed by the background. It improves VideoLLaMA3's average accuracy on Ego-in-Exo Perception from 71.2% to 74.2%, without requiring an egocentric camera at test time.
Mitigate Modality-Asymmetric Forgetting via Stabilizing Visual Representations in CLIP-Based Class-Incremental Learning: To tackle modality-asymmetric forgetting in CLIP-based class-incremental learning—where the visual branch degrades far more severely than the text branch—this paper introduces Visual Anchor-based Structural Transfer (VAST), which builds stable visual anchors from intermediate cross-layer self-similarity and aligns cross-task manifold distributions via matrix Rényi entropy distillation, achieving SOTA performance without replay data.
MLVC: Multi-Platform Learned Video Codec for Real-world Deployment: This work changes the scale parameter required for entropy coding from "computed in real-time by the network" to "deterministically transmitted via hyperprior". This enables neural video codecs to perform "encode on device A, decode on device B" without crashing on heterogeneous NPUs like Apple, Intel, and Qualcomm for the first time. Simultaneously, a series of improvements including gated memory, ReGLU, and long-term reference frame recovery recover the coding efficiency loss caused by cross-platform constraints, achieving a >70% BD-rate(MOS) improvement over hardware HEVC on video conferencing benchmarks, with an average of 100 FPS across three platforms.
MoE-KD: Your Teacher Model is Worth Mixture-of-Experts for Knowledge Distillation: MoE-KD reformulates knowledge distillation from a Mixture-of-Experts perspective by introducing teacher predictions as latent variables, reusing the teacher classifier for routing, and training specialized prototype experts via an Expectation-Maximization algorithm maximizing the evidence lower bound.
Multi-Head Normalization for Wide Vision Transformers: Addressing training collapse and severe gradient attenuation in wide Vision Transformers caused by outlier dimensions dominating global normalization, this paper introduces Multi-Head RMSNorm (MH-RMSNorm) to partition high-dimensional features into head-wise chunks for independent normalization, unlocking stable scaling up to width 4096.
Open-Vocabulary Long Term Action Anticipation: Addressing the limitation of conventional egocentric action anticipation confined to closed vocabularies and disjoint multi-horizon predictions, this paper introduces the first open-vocabulary long-term action anticipation evaluation framework and proposes a Multi-Horizon Cross-Attention Decoder (MHCAD) integrated with 4D Rotary Position Embeddings and lightweight text bottleneck adapters, substantially outperforming existing adapted models and foundation baselines in zero-shot cross-dataset transfer.
Perceptual Projection Pruning: Diversity-Aware Video Token Pruning for Multimodal Large Language Models: P3 compresses the criteria for selecting video tokens into three scores for motion, regional contrast, and local complexity, then clusters within temporal segments and retains original tokens through stratified selection; on Qwen3-VL-8B, it achieves 97.3% average relative performance at 25% retention and a 4.05-fold speedup for pruning plus prefill in a specified dense-video setting.
R2M: Real-Aware Residual Model Merging for Robust and Generalizable Deepfake Detection: Addressing decision-boundary shifts and conflicting forgery cues caused by naive model merging in multi-generator deepfake detection, R2M proposes a training-free merging framework that suppresses real-sensitive modes and extracts forgery residuals via offline spectral decomposition, then adaptively routes the best-matching residual online through single-backward gradient alignment.
RUTaL: Residual Upcycling with Task Ladder for Efficient Multi-Task Learning: Addressing the prohibitive resource overhead of full fine-tuning and the representational competition inherent in coupled PEFT methods where shared and task-specific parameters contest a fixed backbone capacity, RUTaL introduces a structurally decoupled multi-task adaptation framework: it expands the shared backbone into a task-general MoE via low-rank residual upcycling (TGRU) while extracting task-specialized representations through a non-interfering side ladder branch (TSLA), achieving a state-of-the-art +7.79% overall improvement (\(\Delta m\)) on PASCAL-Context with merely 2.93M trainable parameters.
Scaling Dense Prediction with Latent Decoding: Addressing the severe efficiency wall and visual degradation of conventional pyramidal pixel decoders at ultra-high resolutions, L-DPT decouples dense prediction modeling capacity from pixel rendering by performing attention-based reasoning entirely on a compact token grid in latent space, reading out to pixels via a lightweight PixelShuffle operator to achieve stable, detail-preserving inference on photographs up to 100 megapixels.
Sequential Visual Place Recognition: Exploiting Trajectory Priors for Robust Localization: Addressing the brittleness of single-image visual place recognition and the impractical reliance of prior sequence methods on ordered reference databases, this paper proposes TRAIL, a lightweight Conditional Random Field framework that leverages query trajectory priors and camera motion consistency to localize the final query against unordered databases, boosting Recall@10 by up to 8.3–10.4 percentage points.
SurvMILKD: A Weakly Supervised Survival Analysis Framework for Multi-Teacher Knowledge Distillation using Pathology Foundation Models: To address the foundation model selection dilemma and the prohibitive inference cost of multi-model ensembles in whole slide image (WSI) survival analysis, SurvMILKD introduces the first multi-teacher knowledge distillation (MKD) framework tailored for WSI survival modeling. By coupling weakly supervised multiple instance learning adapters with a risk-aware soft ordinal distillation loss and adaptive batch-wise teacher weighting, it compresses complementary representations from multiple frozen billion-parameter teachers into a single compact student, surpassing both individual backbones and heavy ensemble baselines without any additional inference overhead.
Task Alignment: A simple and effective proxy for model merging in computer vision: Addresses the computational bottleneck where model merging with trainable decoders requires expensive downstream fine-tuning for hyperparameter search, proposing an unsupervised Task Alignment Proxy (TAP) that speeds up hyperparameter selection by orders of magnitude while preserving near-optimal performance across diverse 2D and 3D tasks.
Topology-Weighted Effective Rank: A Zero-Cost Proxy for Training Dynamics Stability in Deep Vision Networks: To overcome the failure of conventional zero-cost proxies to capture optimization dynamics and architectural topological heterogeneity, this paper introduces ER-Score based on the effective rank of feature correlation matrices and its topology-weighted variant TER-Score, achieving state-of-the-art ranking consistency and ultra-low search overhead across diverse NAS benchmarks.
Towards Reconfigurable Visual Feature Compression: Addressing transmission redundancy and inference inefficiency when deploying vision foundation models (VFMs) for arbitrary user-requested multi-task combinations in end-cloud collaboration, this paper proposes the Reconfigurable Feature Compression (RFC) framework based on a "factorization-consolidation-recomposition" paradigm, achieving demand-aware minimal bitrates and once-forward cloud multi-task inference.
VD-LoRA: Adaptive Reuse of Low-Rank Directions for Continual Learning: Addressing capacity collapse and plasticity degradation in rehearsal-free class-incremental learning caused by rigid orthogonality in LoRA, VD-LoRA introduces recursive direction-wise variational Bayesian modeling that leverages posterior variance to adaptively calibrate protection strength and guide low-risk direction reuse under strict rank constraints.
When the Teacher Has More Bits: Self-Teacher Latent Distillation for Learned Image Compression: Addressing the disconnected optimization across operating points in learned image compression (LIC), this paper proposes a self-teacher latent distillation framework where a high-rate instance of the identical codec supervises lower-rate encoders, introducing variance-normalized latent distillation (VNLD) to inverse-weight channel supervision and achieve consistent rate-distortion gains with zero inference overhead and standard-compliant fixed decoders.