⚡ VLM Efficiency¶
🎞️ ECCV2026 · 18 paper notes
📌 Same area in other venues: 📷 CVPR2026 (63) · 🔬 ICLR2026 (18) · 💬 ACL2026 (6) · 🧪 ICML2026 (4) · 🤖 AAAI2026 (5) · 🧠 NeurIPS2025 (8)
🔥 Top topics: Model Compression ×6 · Diffusion Models ×4 · Compression ×3
- 3DZip: Spatial-Aware Feature Diversity-Guided Token Compression for 3D Question Answering
-
3DZip compresses LLaVA-3D visual tokens before the language model through coarse voxelization, feature-diversity DPP anchor selection, and spatially constrained merging, reducing the reported 1410-token configuration to 128 tokens while retaining a paper-reported 94.7% relative performance and reducing latency from 342 ms to 178 ms per sample on a single RTX 4090.
- Accelerating Diffusion Models via Equal-Risk Caching
-
ERC probes how a single cache-window length affects final outputs at different timesteps, accumulates the resulting one-dimensional risk curve, and partitions it to allocate refreshes at lower offline cost than exhaustive interval evaluation, achieving, for example, 3.23x acceleration and LPIPS 0.1954 on FLUX.1 [dev] with a refresh budget of 15.
- Activation Quantization of Vision Encoders Needs Prefixing Registers
-
RegCache extracts reusable middle-layer register KV caches from reference images and combines prefix insertion with residual outlier token deletion to improve post-training quantization, raising CLIP-B/16 ImageNet-1k accuracy under ERQ W4A4 from 1.56% to 46.07%, without universally recovering full-precision performance.
- AnchorPrune: Relevance-Anchored Contextual Expansion for Visual Token Pruning
-
AnchorPrune separates visual token selection into the ordered stages of protecting query-relevant evidence and adding important, non-redundant context, requiring no training and retaining 97.6% of full-model average relative performance on LLaVA-NeXT-7B with only 160/2880 tokens.
- ARVAR: Accelerating Visual Autoregressive Model via Attention Retrospect
-
ARVAR uses low-resolution attention and activations to allocate high-resolution computation, recomputing important tokens, reconstructing other positions, and reusing the conditional branch at selected late steps to achieve 1.56x HART and 3.01x Infinity-2B speedups on one RTX3090 while preserving the original model's images better than FastVAR.
- AViTS: Adaptive Spatiotemporal Token Selection for Efficient Dynamic-Resolution Generation
-
AViTS jointly asks which latent tokens are relevant to the text and which are still changing during denoising to decide where to upsample early, reducing FLUX.1-dev latency from 25.78 s to 3.65 s in its 14-step configuration while ImageReward changes from 0.9719 to 0.9723, although more aggressive acceleration still sacrifices quality.
- BATQuant: Outlier-Resilient MXFP4 Quantization via Learnable Block-wise Optimization
-
BATQuant confines learnable non-orthogonal transformations to MXFP4's 32-element quantization blocks, then uses shared factorization and local clipping to reduce deployment overhead and residual outlier effects, achieving 96.43% multimodal performance recovery on Qwen3-VL-8B-Instruct under W4A4KV16.
- Bayesian Self-Attention with Local Pixel Correlations for Lightweight Denoising Transformers
-
NLformer extends lightweight denoising Transformers from finding and averaging similar patches to also exploiting within-patch dependencies through explicit inverse-based processing, reaching 39.88 / 39.92 dB on SIDD / DND with 2.44M parameters and outperforming the lightweight models compared in the paper, without minimizing every resource metric.
- BLOB-Q: Boosting Low Bit ViT Quantization via Global Optimization on Model Distortion
-
BLOB-Q turns mixed-precision allocation for ViT weights and activations into an additive approximation of model-output distortion, then uses dynamic programming to find the best combination within a bit budget, reaching 76.85% ImageNet top-1 on ViT-S at 4-bit average precision; its global-optimality guarantee should be restricted to the approximated discrete problem, not unconditional optimal accuracy for the original network.
- CASA: Cross-Attention over Self-Attention for Efficient Vision-Language Fusion
-
CASA revisits cross-attention through controlled experiments, keeping image representations outside the language backbone and replacing window-local visual KV sources for efficient fusion; vanilla CA reaches 6.8 FPS versus 1.2 FPS for token insertion in the controlled Helium1 streaming setup, but complex infographic understanding still shows a substantial performance gap.
- CMuon: Accelerating and Stabilizing Diffusion Transformer Training via Chunked Momentum Orthogonalization
-
CMuon independently orthogonalizes momentum for the functional QKV, FFN gate/up, and AdaLN components stored in fused DiT tensors, enabling a 675M DiT-XL to reach FID 1.18 on ImageNet-1K 256x256 in 200 epochs, better than AdamW's 1.21 after 400 epochs, with gains extending beyond early training.
- Combating Textual Noise and Redundancy: Entropy-Aware Dense Visual Token Pruning
-
EADP filters dispersed text guidance using the spatial entropy of text-to-image responses, combines local and global semantics, smooths the relevance map, and selects visual tokens by coverage gain, allowing LLaVA-NeXT-7B to match the full model's 66.3 average while retaining only 640/2880 tokens.
- CoverPrune: Coverage-Driven Token Pruning for 3D VLMs via Optimal Transport
-
CoverPrune defines how retained tokens can represent the original evidence through joint feature, spatial, and temporal costs, using an optimal-transport approximation to select representative prototypes: on GS-Reasoner, it scores 56.83 on VSI-Bench with only 10% of visual tokens, versus 51.66 for DTC at the same budget, while Lite replaces repeated transport solving with local grouping to reduce pruning overhead from 2.53 to 0.41 seconds.
- CUST : Clustered Unit-level Similarity Transformer for Lightweight Image Super-Resolution
-
CUST groups distant but similar patches using pooled window representatives, applies attention within the resulting groups, and guides local refinement with downsample-upsample feature residuals: CUST-Base improves average PSNR over CATANet by 0.094 dB across five ×4 benchmarks, while the separate Base+ configuration demonstrates a practical memory-latency trade-off.
- DARL: Efficient Document-to-Markup Generation via Look-Ahead Diffusion Trajectory Sampling
-
DARL preserves a verifiable autoregressive prefix while refining candidate tokens in a short parallel window, then uses online sampling and look-ahead rewards to learn trajectories that are easier to accept consecutively, achieving a 2.31x speedup and an overall score of 87.69 with dots.ocr-3B on OmniDocBench-1.5 without improving every recognition submetric over the original AR model.
- DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression
-
DASH treats discontinuities between adjacent audio representations as semantic chunking cues and combines boundary, representational uniqueness, and attention scores to retain tokens, allowing Qwen2.5-Omni-7B to reach 44.9% WorldSense accuracy at 25% target retention and 14.9T FLOPs, compared with OmniZip's 44.7% at 35% retention and 21.4T FLOPs.
- DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Space
-
DC-Gen aligns input and output interfaces before using LoRA to adapt a pretrained diffusion backbone to a deeply compressed latent space with fewer tokens, retaining comparable quality across generation tasks; the paper reports up to 53.8 times acceleration for FLUX at 4K, but times only the Transformer backbone.
- Dense Video Understanding with Inter-tokenization Acceleration
-
The paper defines high-FPS video understanding through DIVE-Bench, where nearly every frame carries evidence, and proposes Gated Residual Tokenization (GRT), which encodes only changed patches before merging semantically redundant scenes, reducing tokenization latency by 53.6% at 1 FPS while improving answer quality on dense-text and high-motion tasks.