⚡ VLM Efficiency

🎞️ ECCV2026 · 57 paper notes

📌 Same area in other venues: 🧠 NeurIPS2026 (1) · 📷 CVPR2026 (63) · 🔬 ICLR2026 (18) · 💬 ACL2026 (6) · 🧪 ICML2026 (4) · 🤖 AAAI2026 (5)

🔥 Top topics: Model Compression ×21 · Multimodal/VLM ×11 · Diffusion Models ×10 · Compression ×9 · LLM ×3

FlashVLM: Text-Guided Visual Token Selection for Large Multimodal Models

At the vision encoder–language model interface, FlashVLM selects important regions using fused query relevance and visual saliency, then adds non-redundant background tokens; reducing LLaVA-1.5-7B from 576 to 128 visual tokens yields a reported 100.6% relative performance, but does not preserve every task's performance.

ResilPhase: Plug-and-Play Phase Mapping and Noise-Resilient Macro-Trajectory Extrapolation for Diffusion Acceleration

ResilPhase reformulates training-free acceleration for DiT from a layer-wise derivative extrapolation paradigm to macro-trajectory extrapolation in the ODE space. It eliminates layer-wise error cascades using an end-to-end Global Drift, bypasses chaotic derivative noise using derivative-free Barycentric Lagrange interpolation, and suppresses the Runge phenomenon by projecting discrete timesteps into a bounded phase space via Chebyshev/Balanced phase mapping. It maintains state-of-the-art generation quality on FLUX.1-dev even at an approximate 5x acceleration.

Fast-dVLA: Accelerating Discrete Diffusion VLA to Real-Time Performance

Fast-dVLA adapts discrete diffusion vision-language-action models to cache completed prefixes and denoise multiple blocks concurrently through lightweight asymmetric distillation, increasing Dream-VLA decoding throughput from 98.8 to 313.1 tokens/s on LIBERO and reporting 30 Hz execution in real-world tasks.

Why and When Visual Token Pruning Fails? A Study on Relevant Visual Information Shift in MLLMs Decoding

The paper finds that static visual token pruning cannot keep pace with shifting visual evidence during multi-step reasoning and introduces DSTP, a training-free add-on that detects attention shifts and temporarily recovers relevant tokens during decoding, raising Qwen3-VL-4B's MathVerse accuracy at 33.3% initial retention from FastV's 32.23% to 52.54%, still below the full model's 61.29%.

DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Space

DC-Gen aligns input and output interfaces before using LoRA to adapt a pretrained diffusion backbone to a deeply compressed latent space with fewer tokens, retaining comparable quality across generation tasks; the paper reports up to 53.8 times acceleration for FLUX at 4K, but times only the Transformer backbone.

Spatial-TTT: Streaming Visual-based Spatial Intelligence with Test-Time Training

Spatial-TTT combines online fast-weight updates, local sliding-window attention, and a small set of full-attention layers, using dense 3D scene descriptions to teach spatial evidence retention; its 2B model scores 64.4 on VSI-Bench while using less memory and computation than Qwen3-VL-2B at 1024 input frames.

ToDRE: Effective Visual Token Pruning via Token Diversity and Task Relevance

ToDRE greedily selects diverse visual tokens before the large language model (LLM), then removes all remaining visual tokens at a deeper layer where cross-modal interaction has subsided, preserving 95.0% mean relative performance across eight image tasks on LLaVA-NeXT-7B with only 10% of input visual tokens retained and achieving about 2.6x speedup in the four-task ablation timing without training.

DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression

DASH treats discontinuities between adjacent audio representations as semantic chunking cues and combines boundary, representational uniqueness, and attention scores to retain tokens, allowing Qwen2.5-Omni-7B to reach 44.9% WorldSense accuracy at 25% target retention and 14.9T FLOPs, compared with OmniZip's 44.7% at 35% retention and 21.4T FLOPs.

MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptation

MMLoP factorizes deep prompts in both CLIP encoders into a shared low-rank factor and modality-specific factors, then protects generalization with text drift correction and zero-shot consistency, achieving 79.70% base-to-novel harmonic mean accuracy with about 11.5K trainable parameters without claiming the highest absolute accuracy.

BATQuant: Outlier-Resilient MXFP4 Quantization via Learnable Block-wise Optimization

BATQuant confines learnable non-orthogonal transformations to MXFP4's 32-element quantization blocks, then uses shared factorization and local clipping to reduce deployment overhead and residual outlier effects, achieving 96.43% multimodal performance recovery on Qwen3-VL-8B-Instruct under W4A4KV16.

Keeping the Evidence Chain: Semantic Evidence Allocation for Training-Free Token Pruning in Video Temporal Grounding

SemVID reformulates video token pruning as semantic evidence allocation, retaining object evidence alongside state transitions and context anchors, allowing Qwen3-VL-4B to achieve 38.49 mIoU on ActivityNet-Grounding with 12.5% of visual tokens, or 95.4% of its full-input mIoU.

Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Models

VISE proposes a purely unsupervised, single-model self-evolution framework. By utilizing geometric invariance and semantic invariance as reward signals, it directly regularizes the decoder's visual conditioning strategy (rather than optimizing answer consistency). On Qwen3-VL-2B, it improves COCO CIDEr by +16.85, TextCaps CIDEr by +19.66, and reduces the hallucination metric Chair-I by 5.0 points. Its effectiveness is consistent across 4 model scales (2B/4B/8B/32B) and 4 architectural backbones, without any trade-offs between tasks.

HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decoding

This paper proposes Hierarchical Speculative Decoding (HSD), a training-free end-to-end acceleration method for document parsing. It first utilizes a lightweight pipeline to generate rough drafts for each region, followed by a two-stage verification process: Stage 1 performs parallel local verification on cropped regions to achieve high throughput, while Stage 2 conducts global verification on the full page to restore cross-region coherence. HSD achieves close to zero loss in accuracy with a 2.78x end-to-end speedup on HunyuanOCR, and up to a 7.04x speedup in long-document scenarios.

AnchorPrune: Relevance-Anchored Contextual Expansion for Visual Token Pruning

AnchorPrune separates visual token selection into the ordered stages of protecting query-relevant evidence and adding important, non-redundant context, requiring no training and retaining 97.6% of full-model average relative performance on LLaVA-NeXT-7B with only 160/2880 tokens.

Fourier Compressor: Frequency-Domain Visual Token Compression for Vision-Language Models

Fourier Compressor transforms visual feature grids into the frequency domain, retains low frequencies, and reconstructs a smaller grid; with adaptation training, compressing LLaVA-v1.5-7B from 576 to 36 visual tokens changes its eight-benchmark average from 64.6 to 62.1 and reduces FLOPs from 8.54 T to 1.38 T.

Accelerating Multimodal Large Language Models with Prior-Corrected Token Reduction

PriorTR is a training-free visual token pruning method. By estimating the model's intrinsic attention prior using a null token (delimiter) and replacing raw attention scores with V-usable information \(P \cdot \log(P/Q)\) for token ranking, it performs prior correction and physical pruning in a single forward pass, significantly improving the accuracy-efficiency trade-off of MLLMs under aggressive token budgets.

Activation Quantization of Vision Encoders Needs Prefixing Registers

RegCache extracts reusable middle-layer register KV caches from reference images and combines prefix insertion with residual outlier token deletion to improve post-training quantization, raising CLIP-B/16 ImageNet-1k accuracy under ERQ W4A4 from 1.56% to 46.07%, without universally recovering full-precision performance.

CASA: Cross-Attention over Self-Attention for Efficient Vision-Language Fusion

CASA revisits cross-attention through controlled experiments, keeping image representations outside the language backbone and replacing window-local visual KV sources for efficient fusion; vanilla CA reaches 6.8 FPS versus 1.2 FPS for token insertion in the controlled Helium1 streaming setup, but complex infographic understanding still shows a substantial performance gap.

CMuon: Accelerating and Stabilizing Diffusion Transformer Training via Chunked Momentum Orthogonalization

CMuon independently orthogonalizes momentum for the functional QKV, FFN gate/up, and AdaLN components stored in fused DiT tensors, enabling a 675M DiT-XL to reach FID 1.18 on ImageNet-1K 256x256 in 200 epochs, better than AdamW's 1.21 after 400 epochs, with gains extending beyond early training.

MoE-ViE: Mixture of Experts Vision Encoder for Efficient Image and Video Understanding

MoE-ViE replaces the MLPs of a CLIP vision tower with a fine-grained MoE — 32 experts per MoE layer, each one quarter the width of the original MLP, plus always-on shared experts, Sigmoid gating, and an auxiliary-loss-free balancer whose bias update is scaled by the degree of imbalance — and pairs it with a custom Triton grouped-GEMM / kernel-fusion implementation and a frame-level distillation plus expert-freezing video fine-tuning stage, yielding a unified image/video encoder: the largest model holds 3.5B total parameters but activates only 1.1B, matches the zero-shot performance of the 1.7x larger dense SOTA encoder PEcoreG, and runs at roughly 76% of its latency.

P-MTP: Efficient Document Parsing via Multi-Token Prediction with Progressive Depth Scaling

P-MTP trains a document parser to establish reliable short-range predictions before taking on more distant targets and adjusts speculative length by confidence, reducing native decoding latency from 26.71 to 5.10 for a 5.24× speedup with Qwen3-VL-2B on PubTabNet; this figure must not be substituted for vLLM deployment gains or used to characterize accuracy changes across all tasks.

Semantic-Geometric Dual Compression: Training-Free Visual Token Reduction for Ultra-High-Resolution Remote Sensing Understanding

DualComp dynamically combines local semantic aggregation with structural path completion according to a question's object-semantic and scene-geometric demands, raising reported average accuracy from 51.5% to 53.1% while reducing inference time from 8.15 to 3.87 seconds/image in the GeoLLaVA-8K experiment on XLRS-Bench; its compression modules are training-free, but its offline router is not.

ViQ: Text-Aligned Visual Quantized Representations at Any Resolution

ViQ proposes a two-stage visual quantization framework—first performing text-aligned pre-training to enrich continuous features with semantics, then progressively compressing and using the FSQ quantizer for low-dimensional discretization—enabling discrete visual encoders to reach or even surpass the performance of continuous encoders for the first time across 9 multimodal benchmarks, while delivering 20%-70% training acceleration and a storage compression ratio of nearly 1/96.

3DZip: Spatial-Aware Feature Diversity-Guided Token Compression for 3D Question Answering

3DZip compresses LLaVA-3D visual tokens before the language model through coarse voxelization, feature-diversity DPP anchor selection, and spatially constrained merging, reducing the reported 1410-token configuration to 128 tokens while retaining a paper-reported 94.7% relative performance and reducing latency from 342 ms to 178 ms per sample on a single RTX 4090.

Combating Textual Noise and Redundancy: Entropy-Aware Dense Visual Token Pruning

EADP filters dispersed text guidance using the spatial entropy of text-to-image responses, combines local and global semantics, smooths the relevance map, and selects visual tokens by coverage gain, allowing LLaVA-NeXT-7B to match the full model's 66.3 average while retaining only 640/2880 tokens.

CoverPrune: Coverage-Driven Token Pruning for 3D VLMs via Optimal Transport

CoverPrune defines how retained tokens can represent the original evidence through joint feature, spatial, and temporal costs, using an optimal-transport approximation to select representative prototypes: on GS-Reasoner, it scores 56.83 on VSI-Bench with only 10% of visual tokens, versus 51.66 for DTC at the same budget, while Lite replaces repeated transport solving with local grouping to reduce pruning overhead from 2.53 to 0.41 seconds.

CUST : Clustered Unit-level Similarity Transformer for Lightweight Image Super-Resolution

CUST groups distant but similar patches using pooled window representatives, applies attention within the resulting groups, and guides local refinement with downsample-upsample feature residuals: CUST-Base improves average PSNR over CATANet by 0.094 dB across five ×4 benchmarks, while the separate Base+ configuration demonstrates a practical memory-latency trade-off.

FlashBEV: Fast and Memory-Efficient Exact BEV Transformation with IO-Awareness

FlashBEV recasts sampling-based view transformation (Sampling-VT) into a gather reduction operator, eliminating large intermediate 3D tensors via a single fused CUDA kernel with thread-local reduction and dynamic recomputation, slashing peak memory by over 37× and achieving 5.2× speedup while preserving exact mathematical equivalence.

HQ-DM: Single Hadamard Transformation-Based Quantization-Aware Training for Low-Bit Diffusion Models

To resolve the catastrophic quality degradation in low-bit diffusion models caused by activation outliers, HQ-DM applies a Single Hadamard Transformation exclusively to activations, smoothing activation peaks while preventing weight outlier amplification and enabling hardware-friendly integer convolution and linear operations, boosting LDM-4 Inception Score by over 467% under W4A3.

LinCa: Accelerating Diffusion Models via Learnable Decomposed Feature Caching

Addressing the cumulative degradation caused by cross-timestep and cross-dimension dynamics mismatch in existing feature caching, LinCa introduces a lightweight learnable invertible network to decompose cached features into sub-components with distinct temporal continuity and applies differentiated-order polynomial extrapolation, achieving 5-7× near-lossless acceleration on FLUX, Qwen-Image, and HunyuanVideo with under 0.2% additional parameters.

Rethinking Token Reduction for Diffusion Models via Output-Similarity-Awareness

Addressing the severe recovery error caused by conventional input-similarity token matching in diffusion models, DiTo introduces an output-similarity-aware token reduction framework that leverages temporal consistency across adjacent timesteps to reuse prior-output matching correspondences, paired with PMR-guided interval scheduling and a frequency penalty, yielding 1.6–3.9 dB PSNR improvements and up to 18.6% latency reduction on Flux and SD3.

SMART: When is it Actually Worth Expanding a Speculative Tree?

SMART proposes a hardware-aware marginal analysis framework that reformulates speculative tree construction in speculative decoding as a sequential decision-making problem aimed at maximizing the end-to-end speedup ratio. A node is expanded only when its marginal benefit-to-cost ratio exceeds the overall speedup ratio of the current tree. This avoids the "negative acceleration" paradox caused by the superlinear growth of verification overheads under large batch sizes, yielding an average of 20.0% (for MLLMs) and 15.4% (for LLMs) extra speedup without requiring retraining.

Structured Redundancy Modeling for Efficient Visual Token Pruning in High-Resolution MLLMs

To overcome the severe latency bottleneck caused by iterative greedy searches in subset-optimization visual token pruning for high-resolution MLLMs, this paper proposes Single-Forward Pruner (SFPruner), which attenuates global covariance-level redundancy via semantics-guided ridge leverage scoring and eliminates local spatial overlap through parallel ranking-based directional masking, slashing token selection time from 112.4 ms to 2.5 ms while preserving up to 99.0% relative performance.

When Token Compression Breaks: Structural Pruning vs. Token Reduction for Robust ViT Segmentation under High Compression

This paper establishes a unified benchmark comparing token compression and structural pruning for Vision Transformers across mild-to-aggressive compression regimes, uncovering token compression collapse under severe reduction and proposing a robust "prune-then-merge" pipeline that achieves superior clean accuracy and corruption robustness.

Accelerating Diffusion Models via Equal-Risk Caching

ERC probes how a single cache-window length affects final outputs at different timesteps, accumulates the resulting one-dimensional risk curve, and partitions it to allocate refreshes at lower offline cost than exhaustive interval evaluation, achieving, for example, 3.23x acceleration and LPIPS 0.1954 on FLUX.1 [dev] with a refresh budget of 15.

ARVAR: Accelerating Visual Autoregressive Model via Attention Retrospect

ARVAR uses low-resolution attention and activations to allocate high-resolution computation, recomputing important tokens, reconstructing other positions, and reusing the conditional branch at selected late steps to achieve 1.56x HART and 3.01x Infinity-2B speedups on one RTX3090 while preserving the original model's images better than FastVAR.

AViTS: Adaptive Spatiotemporal Token Selection for Efficient Dynamic-Resolution Generation

AViTS jointly asks which latent tokens are relevant to the text and which are still changing during denoising to decide where to upsample early, reducing FLUX.1-dev latency from 25.78 s to 3.65 s in its 14-step configuration while ImageReward changes from 0.9719 to 0.9723, although more aggressive acceleration still sacrifices quality.

Bayesian Self-Attention with Local Pixel Correlations for Lightweight Denoising Transformers

NLformer extends lightweight denoising Transformers from finding and averaging similar patches to also exploiting within-patch dependencies through explicit inverse-based processing, reaching 39.88 / 39.92 dB on SIDD / DND with 2.44M parameters and outperforming the lightweight models compared in the paper, without minimizing every resource metric.

BLOB-Q: Boosting Low Bit ViT Quantization via Global Optimization on Model Distortion

BLOB-Q turns mixed-precision allocation for ViT weights and activations into an additive approximation of model-output distortion, then uses dynamic programming to find the best combination within a bit budget, reaching 76.85% ImageNet top-1 on ViT-S at 4-bit average precision; its global-optimality guarantee should be restricted to the approximated discrete problem, not unconditional optimal accuracy for the original network.

CURE: Cumulative Knowledge Reuse for Efficient Device-Server Hybrid Inference in Vision-Language Models

Rather than discarding server classification results after use, CURE accumulates them in an on-device Gaussian knowledge reservoir and combines two-stage uncertainty routing with prediction fusion, achieving 85.75% average accuracy at a 33.26% average delegation frequency across 13 image classification datasets, matching or exceeding the aggregate performance of always querying the server.

DARL: Efficient Document-to-Markup Generation via Look-Ahead Diffusion Trajectory Sampling

DARL preserves a verifiable autoregressive prefix while refining candidate tokens in a short parallel window, then uses online sampling and look-ahead rewards to learn trajectories that are easier to accept consecutively, achieving a 2.31x speedup and an overall score of 87.69 with dots.ocr-3B on OmniDocBench-1.5 without improving every recognition submetric over the original AR model.

Dense Video Understanding with Inter-tokenization Acceleration

The paper defines high-FPS video understanding through DIVE-Bench, where nearly every frame carries evidence, and proposes Gated Residual Tokenization (GRT), which encodes only changed patches before merging semantically redundant scenes, reducing tokenization latency by 53.6% at 1 FPS while improving answer quality on dense-text and high-motion tasks.

DIVA: Instruction-Aware Vision Token Pruning via Dual-Probe Attention Discrepancy

Instead of directly using absolute attention values as vision token importance, DIVA compares the discrepancy in attention distributions over vision tokens between two sets of text probes—one that "sees the instruction" and one that "cannot see the instruction"—and then calibrates the bypassed tokens that skip self-attention. On LLaVA-NeXT-7B, retaining only 80 out of 2880 vision tokens preserves 92.6% of the baseline performance, while reducing the KV cache size by a factor of 10.8.

Efficient Quantization-Aware Adaptation for Visual Foundation Models

Addressing the severe memory explosion caused by cached intermediate activations and activation quantizers when fine-tuning visual foundation models, EQuA decouples a lightweight sub-network side adapter branch from the backbone based on activation-scaled weight importance, cuts training memory by over 70%, and merges the adapter back into the backbone for zero-overhead quantized deployment.

GoodQ: Zero-Shot Quantization for Object Detectors Using Off-the-Shelf Generative Models

GoodQ leverages off-the-shelf diffusion models to generate diverse training images, and systematically addresses three major challenges in zero-shot quantization for object detection using three components: Information-Dense Prompting, Inherent Distribution-Aware Selection, and Teacher-guided Adaptive Noise Reduction. Under extremely low bit-widths such as W4A4 and W3A3, it significantly outperforms previous noise-optimization-based methods.

GTR: Guide-Then-Refine Token Compression for Training-Free Acceleration of Video-LLMs

Proposes Guide-Then-Refine (GTR), a training-free two-stage visual token compression framework that dynamically allocates frame-level budgets using cross-frame global uniqueness and text-query relevance, followed by local spatial refinement within each frame, maintaining 95.8% of full performance with only 15% visual tokens.

Improved Immiscible Diffusion: Accelerating Diffusion Training by Reducing Miscibility

Reveals the theoretical bottleneck of crossed diffusion trajectories causing denoising supervision collapse in noisy layers, proves that vanilla denoising holds an intrinsic local bijection between generated images and noise origins, and introduces \(O(n)\) KNN noise selection and pairing-free image scaling to accelerate diverse diffusion models by 2.5× to 4.5× without sacrificing diversity.

Lightweight Online Reinforcement Learning for Block Decomposition of CAD Models

Addressing the bottleneck of offline dataset dependency and heavy pre-training in 3D CAD block decomposition, this paper proposes the first pure online reinforcement learning framework that leverages a frozen topological graph encoder and a continuous-to-discrete action projection mechanism to autonomously explore and derive sweepable decomposition strategies on a single target model from scratch.

MobileSAM2: Lightweight Segment Anything in Images and Videos via Hypergraphical Knowledge Distillation

To tackle the excessive parameter scale and compute cost of SAM2 on resource-constrained edge devices, MobileSAM2 introduces Hypergraphical Knowledge Distillation (HyperKD), which explicitly captures high-order temporal associations and multi-granularity visual hierarchies via hypergraphs and pairs with progressive architecture contraction, yielding state-of-the-art efficiency and segmentation quality using only ~10% training data.

ORFC: Orthogonal Reparameterization for Low-Bitrate ViT Feature Coding

Addressing the severe task degradation and low-bitrate collapse in edge-cloud split ViT inference caused by feature-space MSE misalignment and highly uneven channel subspace task sensitivity, ORFC incorporates a differentiable Cayley-parameterized orthogonal rotation prior to fixed-group product quantization and optimizes an entropy-constrained rate-distortion objective using frozen-tail output deviation as a proxy distortion, substantially eliminating task performance collapse under 0.1 BPFP.

SpecEyes: Accelerating Agentic Multimodal LLM via Speculative Planning and Perception

SpecEyes lifts speculative acceleration from the token level to the agentic level by utilizing a lightweight non-agentic MLLM to draft tool-free answers, regulated by a scale-invariant cognitive gate based on top-\(K\) logit answer separability and served through a heterogeneous parallel funnel, delivering a 1.73x average speedup (up to 3.35x) and a 2.87% accuracy boost across V*, HR-Bench, and POPE.

Spectral Evolution-Guided Token Pruning in Multimodal Large Language Models

This paper proposes CLSE (Cross-Layer Spectral Evolution), a training-free visual token pruning framework. Instead of evaluating token importance using single-layer attention or feature magnitudes, CLSE transforms the visual tokens of each layer into the frequency domain, applies high-pass filtering to isolate high-frequency structures, and measures the relative variation of this structural energy between adjacent LLM layers. Tokens with highly dynamic variations are classified as semantically active and are retained. Under aggressive compression (retaining only 11% to 33% of the tokens) across various MLLMs such as LLaVA, Qwen2-VL, and Video-LLaVA, CLSE's accuracy retention significantly outperforms FastV, SparseVLM, PDrop, and other baselines.

Stabilizing Ultra-Low-Bit Quantization of Multimodal LLMs via Global Bit Allocation

To prevent severe accuracy collapse in multimodal LLMs at ultra-low bit-widths (2–3 bits), GloBitQ uncovers a two-level output-channel anisotropy and outlier-sensitivity resonance at the Transformer block level, deriving a training-free global bit allocation and affine smoothing framework that matches full-precision performance within 2.1% at 3.01 bits while remaining completely stable at 2.01 bits.

STAT: Soft Tail-dropping for Adaptive Visual Tokenization

STAT presents a discrete 1D visual tokenizer with a soft tail-dropping mechanism that predicts token-wise keep probabilities guided by content-adaptive and decreasing importance priors, aligning allocated token counts with image perceptual complexity to achieve SOTA reconstruction and scalable causal autoregressive generation.

SWIFT: Spatial-Window Integrated Frequency-aware Token Pruning for Efficient MLLMs on Edge Devices

Addressing the issues of high-frequency detail loss and spatial collapse caused by conventional attention-based token pruning, SWIFT extracts frequency-domain residuals via matrix factorization and enforces local spatial window regularization, achieving near-lossless multimodal reasoning with up to 75% compression and a 2.66× decoding speedup on edge devices.

TAQ: Static-Deployable Temporal-Aware Quantization for Real-World Video Super-Resolution

To tackle high runtime overhead from dynamic quantization and severe temporal drift in static baselines for edge-deployed video super-resolution, TAQ introduces an offline-only temporal-aware static post-training quantization framework that combines sequence-wise histogram initialization, consecutive frame-difference refinement, and quantizer bound ensembling, achieving a 3.56× practical speedup on Jetson Orin Nano while substantially eliminating temporal flickering.

VQT: Vector Quantization Tuning for Efficient Fine-tuning and Compression of Pre-trained Vision Transformers

To tackle Coupled Vector Quantization Noise (CVQN) arising from the interaction between LoRA and Straight-Through Estimators (STE) during low-bit vector quantization tuning, VQT introduces stochastic sub-vector replacement alongside progressive alignment, enabling efficient fine-tuning, extreme compression, and sub-1-bit cloud multi-task storage.