📦 Model Compression

🎞️ ECCV2026 · 14 paper notes

📌 Same area in other venues: 📷 CVPR2026 (108) · 🔬 ICLR2026 (240) · 💬 ACL2026 (59) · 🧪 ICML2026 (117) · 🤖 AAAI2026 (60) · 🧠 NeurIPS2025 (143)

🔥 Top topics: Model Compression ×3

Accurate Zero-shot Quantization via Hierarchical Teacher-Assistant Distillation

Zest connects a full-precision CNN to a low-bit student without original training images through an intermediate-precision teacher assistant, quantization-aware image recalibration, and block-wise reconstruction, reaching 70.50% ImageNet-1K Top-1 accuracy for ResNet-18 W4A4, 0.51 percentage points below the reported full-precision baseline.

Audio-Visual Continual Test-Time Adaptation without Forgetting

AVReCAP retrieves historical fusion-layer parameters using input statistics instead of continually overwriting one parameter state, achieving 43.51% mean accuracy on VGGSound-2C, 6.28 percentage points above the source model, while substantially reducing source-domain forgetting.

AVQ-Attention: Adaptive Vector-Quantized Attention

AVQ-Attention starts with coarse codeword attention and selectively activates learned children for the parents receiving the most attention within each query tile; fused precomputation and incremental correction improve the speed–quality trade-off, reaching 43.33% mIoU in a controlled ADE20K comparison, still below exact attention's 49.0%.

Benchmarking Federated Learning & Knowledge Distillation for Point Cloud Classification

A benchmark of federated teachers and distillation objectives shows that labeled proxy data can produce a 92.94% student from an 8.50% teacher, so a compact student's high accuracy does not necessarily demonstrate successful transfer from federated training.

Boosting Text-Driven Video Segmentation via Geometry-Aware Distillation

Built on SAMWISE, GeoLaV first pretrains on continuous novel views synthesized from single images, then distills semantic and 3D geometric teachers on real videos, reaching 70.5 J&F on Ref-YouTube-VOS—1.3 points above the equally sized 202M-parameter SAMWISE—without retaining teachers or projection heads at inference.

CaPCL: Caption-Preserved Continual Learning for Text-to-Image Retrieval

CaPCL requires the model to continue generating auxiliary captions written for current images prior to the update when learning new text-image retrieval tasks, thereby protecting old semantics not covered by the new query. Across the average scores of four task sequences using BLIP-2, it reduces the forgetting metric (FM) of ModX from 0.0890 to 0.0654, while increasing the average accuracy (AA) from 0.6387 to 0.6942.

Condensing Large-Scale Datasets Directly with Minimal Information Loss

This paper points out that the "data -> model -> image" dual compression process in mainstream large-scale dataset distillation (SRe2L-based family) causes severe information loss and shifts distilled images away from the real distribution, thereby undermining Relabeling. To address this, the authors propose CIM, which uses a computable "effective information gap" metric to directly minimize the information discrepancy between the synthetic and real sets on the original images, bypassing the expensive recovery stage. Consequently, CIM achieves 48.7% Top-1 accuracy (ResNet-18) on ImageNet-1K with IPC=10 in just 80 minutes on a single GPU.

FastSTAR: Spatiotemporal Token Pruning for Efficient Autoregressive Video Synthesis

FastSTAR jointly uses spatial and temporal feature changes to select tokens for computation during InfinityStar's high-resolution refinement, while preventing residual additions at skipped positions, reducing end-to-end 720p text-to-video latency from 81.7 to 40.6 seconds as the VBench total score changes from 83.89 to 83.18.

MambaRaw: Selective State Space Modeling for Efficient 4K Raw Image Reconstruction

MambaRaw introduces State Space Models (SSM/Mamba) into the entropy parameter estimation of the JPEG-guided metadata RAW image reconstruction framework. Through two lightweight modules—energy-guided tile selective scanning (TileMambaBlock) and energy-aware feature refinement (EAR)—the approach simultaneously improves reconstruction quality (PSNR increases by 1.2–1.4 dB) and reduces coding latency (by approximately 9%) at 4K resolution.

MLVC: Multi-Platform Learned Video Codec for Real-World Deployment

By changing the scale parameters required for entropy coding from "computed in real-time by the network" to "deterministically transmitted through the hyperprior," this work enables neural video codecs to perform "encode on A, decode on B" across heterogeneous NPUs (such as Apple, Intel, and Qualcomm) for the first time without crashing. Concurrently, a series of improvements including gated memory, ReGLU, and long-term reference frame recovery are introduced to recoup the rate-distortion loss caused by cross-platform constraints, achieving a >70% BD-rate (MOS) reduction relative to hardware HEVC on video conferencing benchmarks, with an average of 100 FPS across three platforms.

Moebius: 0.2B Lightweight Image Inpainting Framework with 10B-Level Performance

Moebius combines a lightweight LλMI backbone with adaptive multi-granularity latent distillation to build a 0.226B-parameter inpainting specialist, achieving FID 0.92 and LPIPS 0.091 on Places2 with small masks; its approximately 15.5-fold total-time advantage over FLUX.1-Fill-Dev comes from both faster individual steps and fewer default sampling steps.

On the Vulnerability of Parameter-Level Defenses to Model Merging

This paper exposes a fundamental vulnerability of parameter-level defenses that use linear transformations to make models "unmergeable"—namely, the protected task vector is 2 to 3 orders of magnitude smaller than the pre-trained anchor. Consequently, this work proposes the Anchor-Guided Attack (AGA). By treating the public pre-trained model as a static anchor, AGA analytically reconstructs the hidden transformation matrix using least squares and the Hungarian algorithm, recovering the protected model almost losslessly. To counter this, an attack-resistant defense named Anchor-Repulsive Fine-tuning (ARF) is introduced as a remedy.

RaysUp: Ultra-light Universal Feature Upsampling via Geometry-Aware Ray Representation

An ultra-lightweight, task-agnostic, and VFM-agnostic universal feature upsampling framework named RaysUp with only 0.14M parameters is proposed. Through three key designs—spatial decoupled guidance encoder, Ray Positioning Encoding (RayPE), and geometry-aware neighborhood cross-attention—it lifts feature reconstruction from the 2D pixel plane to the 3D ray domain. It achieves state-of-the-art (SOTA) or near-SOTA performance across various dense prediction tasks such as semantic segmentation, depth/normal estimation, video object segmentation, and open-vocabulary segmentation, while offering an inference speed approximately 7 times faster than AnyUp, the only prior VFM-agnostic method.

Structural Assessment for Understanding and Guiding Dataset Distillation in Discrete Token Space

This work proposes a Structural Score to evaluate the quality of distilled datasets through statistical analysis in the discrete visual token space. Based on this, a Token-Guided Dataset Distillation (TGDD) framework is developed, which utilizes the Structural Score to guide diffusion models for generating high-quality distilled data.