Revisiting Parameter Redundancy in Vision-Language-Action Models: Insights from VLM-to-VLA Adaptation¶
Conference: ECCV 2026
arXiv: 2606.31382
Code: https://github.com/Niannnnnn/VLA_Parameter_Redundancy_VLM2VLA
Area: Robotics / Embodied AI / Model Compression
Keywords: VLA models, parameter redundancy, pruning, VLM-to-VLA adaptation, module heterogeneity, recovery-free pruning
TL;DR¶
This paper revisits the parameter redundancy problem in Vision-Language-Action (VLA) models, finding that the existing "prune-then-fine-tune" paradigm masks the mistaken deletion of critical parameters. The authors propose using the weight divergence (\(\Delta W\)) during the VLM-to-VLA adaptation process as a redundancy metric. Through controlled pruning diagnostic experiments, they reveal that the \(\Delta W\) signals exhibit strong heterogeneity across different modules. Based on these findings, they design a multi-module joint pruning scheme that achieves a 12%--30% parameter reduction on OpenVLA and \(\pi_{0.5}\) while maintaining approximately 90% of the original performance without any post-pruning recovery.
Background & Motivation¶
VLA models have made significant progress in embodied AI by adapting large-scale pre-trained VLMs to robotic control tasks, thereby inheriting the cross-modal representation capabilities of VLMs. However, VLA models have massive parameter scales, leading to severe computational and storage overheads. Furthermore, they exhibit extreme sensitivity to parameter pruning, where even moderate pruning ratios cause a "cliff-like drop" in task success rate, rendering the model almost completely dysfunctional.
Existing mainstream strategies adopt a pragmatic approach: they treat post-pruning performance degradation as an unavoidable side effect and "repair" it through fine-tuning or low-rank adaptation. A pruning method is considered "successful" as long as it restores performance to an acceptable level during the recovery phase. This paper fundamentally questions this premise: if a pruned VLA model must relearn to recover its functionality, are the removed parameters truly redundant? The authors argue that the widespread reliance on performance recovery conceals a more critical issue: existing pruning criteria actually "indiscriminately kill" critical parameters rather than genuinely identifying redundancy.
The key insight is that parameter redundancy in VLAs is not equivalent to that in traditional CNNs or LLMs. VLAs are not trained end-to-end on a single task distribution; instead, they are constructed through a structured adaptation process from general vision-language understanding to robotic control. During this process, parameters are selectively reused, reweighted, or stabilized to support action-oriented computing paths. Therefore, the parameter divergence (\(\Delta W\)) in the VLM-to-VLA adaptation process is not random noise, but rather carries structured signals regarding the functional importance of parameters—a signal that can serve as the key to precisely identifying redundancy.
Method¶
Overall Architecture¶
The methodological core of this paper is not directly proposing a pruning algorithm, but rather reconceptualizing VLA parameter pruning from an optimization problem into an analytical issue. The authors construct a complete diagnostic framework: first, they quantify the spatial distribution patterns of parameter divergence (\(\Delta W\)) during the VLM-to-VLA adaptation process; then, they introduce controlled pruning as diagnostic probes—by comparing the direct impact (without any fine-tuning) of removing different parameter subsets characterized by various \(\Delta W\) properties on VLA performance, they establish a causal relationship between the adaptation signals and functional contributions; finally, they design a multi-module joint pruning scheme based on the discovered module heterogeneity. The entire framework unfolds around the step-by-step verification of four hypotheses.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["⟨VLM, VLA⟩ Model Pairs<br/>Prismatic→OpenVLA<br/>PaLI-Gemma→π0.5"] --> B["Calculate ΔW Parameter Divergence<br/>Module-wise/Head-wise/Channel-wise"]
B --> C["Visualize ΔW Spatial Distribution<br/>Discover Structured Heterogeneity"]
C --> D["Controlled Pruning Diagnostics<br/>Highest-diff vs Lowest-diff"]
D --> E["Establish Causal Mapping Between<br/>ΔW Signals & Functional Importance"]
E --> F["Multi-Module Joint Pruning<br/>Light/Moderate/Aggressive"]
F --> G["Direct Inference Evaluation Without Recovery"]
Key Designs¶
1. Controlled Pruning as Diagnostic Probes: From "How to Recover" to "Is It Truly Redundant"
Existing pruning methods treat performance recovery as a standard component of the pruning pipeline, formalized as \(f(\cdot; \mathcal{T}(\mathcal{P}(W; M)))\)—the final evaluated model is no longer supported by its post-pruning parameters alone, but is rather the result of relearning and self-reconstruction under structural perturbations. The authors point out a fundamental problem: if a pruning intervention must rely on explicit performance recovery to succeed, the removed parameters cannot be truly redundant.
Therefore, the authors redefine pruning as an analytical tool rather than an optimization objective: for a target module, they construct two complementary masks at a given pruning ratio \(r\)—\(M^{\text{high}}(r)\) retains parameters with the largest \(|\Delta W|\) (Highest-diff pruning), and \(M^{\text{low}}(r)\) retains parameters with the smallest \(|\Delta W|\) (Lowest-diff pruning). They then directly evaluate the inference behavior of the pruned model without any fine-tuning. If there is a significant performance gap between the two strategies, it indicates that the \(\Delta W\) signal indeed contains valid information about parameter importance. If Highest-diff pruning maintains high performance while Lowest-diff pruning leads to collapse, it shows that highly changed parameters are the functional core—contradicting the traditional intuition that "small change = redundancy." In the algorithm, for multi-matrix modules like LLM FFNs, channel dimensions must be aligned across gate/up/down projections, and for Attention, aggregation is performed at the head level before ranking, ensuring that the pruning granularity consists of structural units (heads/channels) rather than individual weights.
2. \(\Delta W\) as a Structured Signal: Discovering Module Heterogeneity
The authors visualize the spatial distribution of \(\Delta W_{\text{rel}} = \|W^{\text{VLA}} - W^{\text{VLM}}\|_2 / \|W^{\text{VLM}}\|_2\) on two representative model pairs. In OpenVLA (Prismatic-based), the parameter updates of the Llama 2 language model present a clear "three-stage" vertical distribution: the initial layers (L0) are densely calibrated to process token injections after multi-modal fusion, the middle layers (L1-L23) are relatively stable, and the end layers (L24-L31) are active again to map to the action space. In the FFN modules, obvious inter-layer fluctuations and stripe-like sparsity in the channel dimension are observed, indicating that embodied knowledge is encoded in specific sub-channels rather than uniformly distributed. In the vision backbone, updates of DINOv2 are concentrated in the shallow layers (low-level visual cues such as grasping and obstacle avoidance), while SigLIP responds more strongly in the deep layers (supplementary semantic information aligned with the language model). The FFN structure of the Projector shows a monotonic increase, and the updates of the mapping layers (fc2, fc3) closer to the language model entrance are much larger than those of the initial layer (fc1).
In \(\pi_{0.5}\) (PaLI-Gemma-based), the modular design makes the evolutionary patterns more regular: the Multi-Query Attention of the Gemma language model exhibits highly discriminative signals in the head dimension, with the largest divergence occurring in the middle layers (L1-L9) (functional redirection to embodied task attention) and high layers tending to stabilize; the FFN of the vision tower shows strong divergence signals in the middle and high layers. These systematic observations point to a key conclusion: parameter updates in VLM-to-VLA adaptation are not random noise, but rather exhibit strong structured heterogeneity in selective reorganization of Attention heads and local activation of FFN channels—which is the empirical foundation for overcoming the "prune-and-collapse" bottleneck.
3. Differentiated Pruning Logic for Diverse Module Functional Roles
Controlled pruning experiments reveal three typical module types, each requiring a fundamentally different pruning strategy:
- Attention and FFN of the Language Backbone: Highest-diff pruning maintains high performance (Llama2 Attention 84.3% SR, Gemma FFN 95.0% SR), while Lowest-diff pruning leads to performance collapse (0.0% and 5.0%). This indicates that in the language backbone, the parameters that change the most during adaptation are precisely the functional carriers of action decision-making—hence, retain high \(\Delta W\) and prune low \(\Delta W\).
- DINOv2 Visual Encoder: Exhibits a "sensitivity reversal"—Highest-diff pruning in Attention heads causes collapse (1.6%) while Lowest-diff maintains performance (76.7%), whereas the FFN channels show the opposite behavior (Highest-diff 82.0%, Lowest-diff 0.0%). This demonstrates that within the same backbone, the functional importance of Attention and FFN relies on different computational paths, and the stable attention heads are actually the critical ones. Therefore, DINOv2's Attention should be pruned for high \(\Delta W\), and FFN should be pruned for low \(\Delta W\).
- Projector and SigLIP: For the Projector, both strategies collapse at a pruning ratio of 0.3 (0.0%/54.0%), highlighting it as a "fragile and non-selective" cross-modal alignment bottleneck that must be strictly protected. SigLIP maintains a 47%--70% SR even under a pruning ratio of 1.0 (complete removal), demonstrating extreme robustness—it acts merely as an auxiliary semantic supplement and can be aggressively pruned.
An Illustrative Example: Explaining Multi-Module Joint Pruning on OpenVLA¶
Take the Moderate configuration of OpenVLA on LIBERO-Spatial as an example: the total parameter count is reduced from 7.5B to 6.2B, and GPU memory usage drops from 14.9GB to 12.4GB. The pruning configurations are: Llama2 Attention: prune Highest-diff by 12.5% (retaining attention heads with the largest adaptation changes, which carry the core action decision-making); Llama2 FFN: prune Highest-diff by 20% (retaining high-\(\Delta W\) channels, which are carriers of embodied knowledge encoding); DINOv2 Attention: prune Lowest-diff by 6.25% (retaining stable heads, which are key to major structural representation); DINOv2 FFN: prune Highest-diff by 10% (retaining functional channels activated during adaptation); SigLIP Attention: prune Highest-diff by 12.5% (auxiliary semantic supplement, aggressively prunable); SigLIP FFN: prune Lowest-diff by 10% (leveraging its robustness); Projector: 0% pruning (fragile bottleneck, strictly protected). Direct inference is performed immediately after all pruning is completed, without any LoRA fine-tuning or RL recovery, resulting in an average SR of 62.3%. Compared to the baseline of 76.5%, it drops by only 14.2 percentage points, whereas LLM-Pruner under the same conditions achieves only 1.0% with the same parameter count.
Loss & Training¶
This paper does not introduce additional training objectives. The core experiments evaluate direct inference after recovery-free pruning without involving any fine-tuning. In the control experiments validating Hypothesis I, LoRA fine-tuning adopts the FSDP distributed strategy, 10k steps, and a learning rate of 1e-4, which is solely used to demonstrate the existence of the "recovery compensation effect" and is not a proposed component of the method.
Key Experimental Results¶
Main Results¶
Table 1: OpenVLA Multi-Module Joint Pruning vs. Existing Pruning Methods (LIBERO benchmark, direct inference without recovery)
| Model | Parameters (B) | GPU Memory (GB) | Spatial | Object | Goal | Long | Average SR | vs. Baseline |
|---|---|---|---|---|---|---|---|---|
| OpenVLA (Baseline) | 7.5 | 14.9 | 84.7 | 88.4 | 79.2 | 53.7 | 76.5 | +0.0 |
| LLM-Pruner | 6.2 | 12.4 | 23.4 | - | - | 1.0 | - | - |
| FLAP | 6.3 | 12.5 | 0.2 | - | - | 0.0 | - | - |
| Wanda (Full Sparse) | - | 10.2 | 0.0 | 13.4 | 0.8 | 0.0 | 7.1 | -69.4 |
| Wanda (Sparse Lang.) | - | 10.6 | 31.2 | 50.8 | 20.0 | 12.4 | 28.6 | -47.9 |
| Ours-Light | 6.6 | 13.0 | 78.3 | 82.5 | 74.0 | 46.8 | 70.4 | -6.1 |
| Ours-Moderate | 6.2 | 12.4 | 70.5 | 74.9 | 64.7 | 39.0 | 62.3 | -14.2 |
| Ours-Aggressive | 5.7 | 11.3 | 59.0 | 65.7 | 56.0 | 29.5 | 52.5 | -24.0 |
Key findings: Under a matched comparison of approximately 6.2B parameters and 12.4GB of GPU memory, the Moderate configuration maintains an average SR of 62.3%, whereas LLM-Pruner achieves only 1.0%. This gap of approximately 60 percentage points indicates that traditional pruning metrics fail completely in the VLA scenario because they unconditionally remove critical parameters currently undergoing functional reorganization. Wanda, even under a shallower sparsification level (12.0GB), yields only 28.6%, and its structured variants suffer catastrophic collapses.
Table 2: \(\pi_{0.5}\) Multi-Module Joint Pruning Results (LIBERO benchmark, direct inference without recovery)
| Model | Parameters (B) | GPU Memory (GB) | Spatial | Object | Goal | Long | Average SR | vs. Baseline |
|---|---|---|---|---|---|---|---|---|
| \(\pi_{0.5}\) (Baseline) | 3.6 | 7.3 | 98.8 | 98.2 | 98.0 | 92.4 | 96.9 | +0.0 |
| Ours-Light | 3.0 | 6.1 | 95.5 | 94.0 | 95.0 | 88.5 | 93.3 | -3.6 |
| Ours-Moderate | 2.8 | 5.6 | 90.7 | 90.1 | 91.3 | 84.0 | 89.0 | -7.9 |
| Ours-Aggressive | 2.5 | 5.0 | 83.6 | 83.0 | 84.5 | 77.1 | 82.1 | -14.8 |
The Moderate scheme for \(\pi_{0.5}\) reduces GPU memory from 7.3GB to 5.6GB, and the average SR only shifts from 96.9% to 89.0%, proving the generalizability of the module-differentiated pruning logic across different VLA architectures.
Ablation Study¶
Table 3: Verification of Hypothesis I—The "Strong Compensation" Effect of Post-Pruning Fine-tuning Recovery (OpenVLA, Llama2 FFN, LIBERO-Spatial, Baseline SR=84.7%)
| Pruning Strategy | Ratio (%) | Post-Pruning SR (%) | Post-Fine-tuning SR (%) |
|---|---|---|---|
| Lowest-diff | 20 | 1.5 | 86.5 |
| Lowest-diff | 50 | 0.0 | 81.0 |
| Lowest-diff | 80 | 0.0 | 76.4 |
| Highest-diff | 20 | 76.3 | 85.8 |
| Highest-diff | 50 | 20.5 | 84.1 |
| Highest-diff | 80 | 0.0 | 80.7 |
| Random | 20 | 12.2 | 86.0 |
| Random | 50 | 0.0 | 82.2 |
| Random | 80 | 0.0 | 77.6 |
Core findings: Removing low-\(\Delta W\) channels (Lowest-diff) causes a complete collapse in direct evaluation (1.5%/0.0%/0.0%), but LoRA fine-tuning restores all configurations to or even beyond the baseline level—including those recovered from 0.0% SR. This "strong compensation" effect proves that the existing paradigm indeed repairs the structural damage caused by "mistaken parameter deletion." Furthermore, the convergence steps required for recovery increase with the pruning ratio (Fig. 3), indicating that higher pruning ratios introduce deeper structural perturbations. This directly supports Hypothesis I: relying on recovery is equivalent to repairing mistakenly deleted parameters, and effective redundancy identification must maintain core functions without recovery.
Table 4: Verification of Hypotheses II & III—Effectiveness of the \(\Delta W\) Signal and Its Cross-Module Heterogeneity (LIBERO-Spatial)
| Model | Module | Sub-module | Pruning Ratio | High-diff SR (%) | Low-diff SR (%) |
|---|---|---|---|---|---|
| OpenVLA | Vision | DINOv2 Attn(head) | 0.125 | 1.6 | 76.7 |
| OpenVLA | Vision | DINOv2 FFN(channel) | 0.20 | 82.0 | 0.0 |
| OpenVLA | Vision | SigLIP Attn(head) | 0.125 | 83.4 | 80.7 |
| OpenVLA | Vision | SigLIP FFN(channel) | 0.20 | 75.1 | 81.7 |
| OpenVLA | Language | Llama2 Attn(head) | 0.125 | 84.3 | 0.0 |
| OpenVLA | Language | Llama2 FFN(channel) | 0.20 | 72.0 | 2.7 |
| OpenVLA | Projector | FFN(channel) | 0.30 | 0.0 | 54.0 |
| \(\pi_{0.5}\) | Language | Gemma Attn(head) | 0.20 | 85.0 | 0.0 |
| \(\pi_{0.5}\) | Language | Gemma FFN(channel) | 0.50 | 95.0 | 5.0 |
| \(\pi_{0.5}\) | Vision | SigLIP Attn(head) | 0.20 | 90.0 | 55.0 |
| \(\pi_{0.5}\) | Vision | SigLIP FFN(channel) | 0.50 | 95.0 | 15.0 |
The "sensitivity reversal" of DINOv2 (Attn High-diff collapsed to 1.6% / FFN Low-diff collapsed to 0.0%) is the most critical finding—the functional importance logic of different computational paths within the same backbone is completely different. The language modules of Llama2 and Gemma perform consistently across both models: Low-diff pruning always leads to collapse (0.0%), while High-diff pruning maintains high performance (84.3%/95.0%). The discriminative power of the \(\Delta W\) signal in SigLIP is weaker (High/Low gap of only 2.7--10 percentage points), consistent with its role as a semantic supplement. The Projector collapses completely (0.0%) under High-diff pruning at a 0.3 pruning ratio, while barely maintaining performance (54.0%) under Low-diff pruning, proving it is a "fragile and non-selective" alignment bottleneck. Cross-model generalization validation shows that High-\(\Delta W\) pruning on \(\pi_0\) across five RoboTwin 2.0 tasks (average 47.4) is far superior to Low-\(\Delta W\) pruning (average 10.6), further confirming the cross-benchmark effectiveness of the \(\Delta W\) signal.
Key Findings¶
- The \(\Delta W\) signal is not a global importance metric but is highly module-specific: In the language backbone, high \(\Delta W\) = important (retain high \(\Delta W\)); in DINOv2 Attention, low \(\Delta W\) = important (retain low \(\Delta W\)); in DINOv2 FFN, high \(\Delta W\) = important (retain high \(\Delta W\)); in SigLIP, \(\Delta W\) has almost no discriminative power; in the Projector, any pruning leads to collapse. There is no unified rule such as "high change = redundant" or "low change = redundant."
- The "strong compensation" effect of the recovery paradigm covers up all pruning errors: Even starting from 0.0% SR, LoRA can restore performance to above the baseline. This means that evaluating pruning quality based on recovered performance is entirely unreliable; it measures the model's relearning capability rather than the accuracy of redundancy identification.
- SigLIP exhibits extreme pruning robustness, while DINOv2 is the primary structural representation module: Complete removal of SigLIP (pruning ratio 1.0) still preserves a 47%--70% SR for OpenVLA, validating the prior characterization of SigLIP as an auxiliary semantic supplement. Slight pruning of DINOv2 causes failure, confirming it as the primary structural representation module.
- The Projector is the most fragile bottleneck and must be protected with zero pruning: Once this cross-modal alignment interface is compromised, performance degrades regardless of which parameters are removed.
Highlights & Insights¶
- Reconceptualizing pruning from an optimization problem to a causal analysis tool: This is the deepest insight of the paper. Instead of simply "proposing a better pruning metric," it first establishes a causal mapping between the \(\Delta W\) signal and functional importance via controlled diagnostic experiments before designing the pruning strategy based on this mapping. This "diagnose-then-design" methodology has higher transfer value than the specific algorithm itself—any model compression involving an adaptation process (such as LoRA merging or multi-task fine-tuning model consolidation) can leverage the same diagnostic framework.
- The "sensitivity reversal" in DINOv2 Attention is the most inspiring finding: Within the same backbone, stable heads in Attention are critical (Low-diff pruning maintains 76.7%), while changing channels in FFN are core (Low-diff pruning drops directly to 0.0%). This shows that Attention and FFN assume fundamentally different computational roles in visual representations—Attention may be responsible for cross-layer structural information transfer (where stable heads are information channels), while FFN handles content-level feature transformations (where channels activated during adaptation are task-specialized paths). This fine-grained functional division of labor goes far beyond the existing coarse understanding of "vision module is primary, language module is secondary."
- The methodology used to reveal the "strong compensation" effect is worth emulating: Instead of directly arguing about the unreliability of recovery, the authors compare Lowest-diff pruning (0.0% SR) after fine-tuning recovery (86.5%) with Highest-diff pruning (76.3% SR) after fine-tuning recovery (85.8%). This demonstrates that the recovery mechanism equally repairs all damage, whether it stems from actual redundancy or mistaken deletion of critical parameters. Exposing methodological flaws with such comparative experiments is much more powerful than pure theoretical arguments.
- Transferable diagnostic framework design: The entire workflow—selecting adaptation pairs \(\rightarrow\) calculating \(\Delta W\) \(\rightarrow\) visualizing distribution \(\rightarrow\) controlled pruning diagnostics \(\rightarrow\) establishing causal mapping \(\rightarrow\) developing differentiated schemes—can serve as a standard analysis paradigm for parameter importance in any "pretrain-to-downstream-adaptation" scenario, not just VLAs.
Limitations & Future Work¶
- Pruning ratio still has a ceiling, with no "free lunch": Under the most aggressive configuration, OpenVLA's average SR drops to 52.5% (\(-24.0\) percentage points). Although far superior to traditional methods, it is still not ready for production environments. The compression limit supported by the \(\Delta W\) signal at the current granularity (head/channel-level structured pruning) is around 30%. More extreme compression may require finer-grained redundancy criteria or integration with other compression techniques (quantization, token pruning).
- Validation is limited to only two VLM-VLA pairs: Although these two pairs represent the two mainstream design paradigms of current VLAs (the large LLM-derived OpenVLA and the modular lightweight \(\pi_{0.5}\)), the generalizability of these findings across more VLA architectures (e.g., RT-2, CogACT, Gr00t) has not yet been verified. Especially for VLAs not directly initialized from VLMs, whether the \(\Delta W\) diagnostic framework applies requires additional research—perhaps requiring a broader definition of "pre-and-post training parameter differences."
- Deep reasons for module heterogeneity are not fully explained: The paper reveals the "sensitivity reversal" phenomenon of Attention and FFN in DINOv2 but does not provide a mechanistic explanation—why are stable heads in visual Attention functionally critical? The authors merely attribute it to "different computational paths" without theoretical modeling or representative analysis experiments to uncover the underlying causes.
- The zero-pruning strategy for the Projector is too conservative: The paper concludes that the Projector must be strictly protected, but the performance gap at a 0.2 pruning ratio (84.5% vs 78.5%) implies that some redundancy signals also exist within the Projector—only that finer granularities (such as lower pruning ratios or sub-structural levels) might be needed to exploit them. Similarly, the differences across fc1/fc2/fc3 within the Projector (where \(\Delta W\) shows a monotonic increase) suggest different layers have different importances, which is worth exploring further.
- Only structured pruning is investigated, leaving unstructured sparsity unaddressed: Wanda's unstructured sparsity scheme (Full Sparse) performs poorly (7.1% SR), showing that literal-level sparsity is also non-viable in VLAs. However, whether the \(\Delta W\) signal can guide unstructured sparsity selection (e.g., locating individual weights that can be sparsified) is not discussed in the paper.
Related Work & Insights¶
- vs. RLRC (Chen et al.): RLRC also tackles VLA parameter pruning but employs the LLM-Pruner + Taylor importance + RL/SFT recovery paradigm. The core difference of this paper lies in fundamentally rejecting recovery dependence, exposing the critical flaw of methods like RLRC through direct evaluation without recovery—they measure the model's relearning capacity rather than the accuracy of redundancy identification. Experiments show that LLM-Pruner achieves only 1.0% SR without recovery, whereas the proposed method maintains 62.3% under the same parameter count.
- vs. GLUESTICK (Jabbour et al.): GLUESTICK extracts the main directions of weight differences using SVD and adds a lightweight correction term to avoid fine-tuning, which is essentially post-pruning repair. This paper goes a step further—if the \(\Delta W\) signal itself can precisely localize redundancy, even SVD corrections are unnecessary. Both share the realization of the importance of \(\Delta W\), but GLUESTICK uses it for repair, whereas this paper uses it for diagnostics and selection.
- vs. VLM4VLA (Zhang et al.): VLM4VLA systematically compares the impact of different VLM backbones on downstream VLA tasks, finding that the baseline performance of VLMs is not necessarily correlated with VLA task performance, implying complex structural reorganization during adaptation. This paper provides a mechanistic explanation from the parameter dimension—different modules follow different parameter update patterns during adaptation, and it is precisely this structured reorganization that prevents the direct transfer of VLM performance to VLA performance.
- vs. Actions as Language (Hancock et al.): This work focuses on the "catastrophic forgetting" of VLM capabilities during the VLA fine-tuning process, maintaining VLM capabilities by relabeling robot actions as image-text pairs. The \(\Delta W\) analysis in this paper is complementary—if the parameter changes during adaptation are selective (updating only sub-structures related to action decision-making), it can theoretically reduce the destruction of VLM knowledge. An interesting future direction is to use the proposed \(\Delta W\) diagnostic framework to guide "minimal-damage adaptation," updating only on highly important channels with high \(\Delta W\) and protecting VLM priors in low-\(\Delta W\) channels.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Reconceptualizing VLA parameter pruning from "how to recover" (engineering optimization) to "what is true redundancy" (analytical reasoning) is highly original. Using controlled pruning as diagnostic probes to establish a causal mapping between \(\Delta W\) and functional importance provides independent methodological value.
- Experimental Thoroughness: ⭐⭐⭐⭐ The experimental design is rigorous—with step-by-step verification of four hypotheses, cross-validation on two VLA architectures, generalization testing on multiple benchmarks (RoboTwin 2.0), and recovery-free comparisons with multiple pruning baselines. A minor limitation is that it only covers two VLA model pairs, lacking validation on a wider range of architectures and larger scale models.
- Writing Quality: ⭐⭐⭐⭐⭐ The logical chain is exceptionally clear—progressing from a phenomenon (pruning sensitivity) to questioning (recovery masking mistaken deletion) to hypotheses (\(\Delta W\) contains structured signals) to validation (controlled diagnostic experiments) to application (multi-module joint pruning). The four-hypothesis advancement structure ensures the reader constantly knows the current step and what needs to be proven next.
- Value: ⭐⭐⭐⭐⭐ The findings themselves (module heterogeneity, sensitivity reversal) provide profound insights into VLA understanding and design. The methodology (diagnosis before design) can be transferred to all model compression scenarios involving adaptation processes. The practical value (12%--30% compression without recovery) directly advances the feasibility of deploying VLAs on resource-constrained devices.