title: >- [Paper Note] Gripper-aware Vision Language Action Models description: >- [ECCV 2026][Robotics & Embodied AI][VLA] Addressing the gripper-invariance limitation in existing VLAs, this paper introduces MiGA, a multi-gripper dataset of 103K trajectories across 5 gripper types, and GVLA, combining multi-granularity soft prompt tokenization with dual Mixture-of-Adapters to achieve morphology-conditioned manipulation and cross-embodiment generalization. tags: - ECCV 2026 - Robotics & Embodied AI - Vision-Language-Action Models - Gripper-Aware - Embodied AI date: 2026-09-19 content_hash: 7a21a9eadae845df

Gripper-aware Vision Language Action Models¶

Conference: ECCV 2026
Paper: ECCV Official Poster
Code: https://airvlab.github.io/G-VLA/
Area: Robotics & Embodied AI
Keywords: Vision-Language-Action Models, Gripper-Aware Learning, Multi-Gripper Manipulation, Mixture of Adapters, Embodied Generalization

TL;DR¶

Addressing the pervasive "gripper invariance" assumption in vision-language-action (VLA) models, this paper introduces MiGA, a comprehensive dataset of 103,000 trajectories across 5 distinct gripper types, and proposes GVLA, which combines multi-granularity soft prompt tokenization with a dual Mixture-of-Adapters to achieve strategy-level gripper conditioning and robust cross-embodiment adaptation.

Background & Motivation¶

Vision-language-action (VLA) models have emerged as a dominant paradigm for general-purpose robotic manipulation, mapping visual observations and natural language instructions directly into continuous executable robot actions. Despite remarkable progress in open-world reasoning, existing VLA architectures and training protocols implicitly embrace an unrealistic assumption: gripper invariance. Current systems routinely assume that achieving a shared manipulation goal follows a uniform trajectory distribution regardless of the robot's physical end-effector. In reality, robotic grasping and manipulation strategies are inherently embodiment-dependent: the physical geometry, kinematic degrees of freedom (DoF), and contact mechanics of the end-effector dictate what interactions are physically feasible. Grasping a thin flat box resting on a tabletop, for instance, allows a vacuum suction gripper to descend vertically from above and lift directly; a conventional parallel-jaw gripper, however, must execute a complex contact-rich maneuver—sliding the object to the table edge to expose a graspable margin before executing a side grasp.

The primary tension behind this capability deficit is the acute embodiment bias in available robot demonstration datasets. Foundational manipulation datasets such as Open X-Embodiment, DROID, and Bridge V2 are overwhelmingly collected using parallel-jaw grippers, completely ignoring suction cups, three-finger hands, soft compliant grippers, or five-finger multi-DoF hands. When researchers attempt to condition VLAs with end-effector identities, standard methods like naive MLP embeddings or natural language prompts fail: language prompts collapse to homogeneous representations in embedding space, while generic encoders fail to capture mechanical and kinematic morphology. Consequently, existing models cannot learn strategy-level divergence where identical task goals necessitate fundamentally distinct trajectories across different end-effectors.

To break this false premise of gripper invariance, robotic foundation models must explicitly encode embodiment-conditioned affordances and strategy branching. Core idea: build MiGA, a large-scale multi-gripper manipulation dataset comprising 103,000 demonstrations across 5 distinct gripper types, and establish GVLA, a framework pairing a three-level hierarchical soft prompt tokenizer with a dual Mixture-of-Adapters (MoA) to govern embodiment-dependent action execution.

Method¶

Overall Architecture¶

Built upon conditional flow-matching VLA backbones (specifically instantiated with $\pi_{0.5}$ and $\pi_0$), GVLA maps multi-view RGB-D observations $O_t$, natural language instructions, and robot hardware identifiers to continuous action trajectories. The architecture decomposes gripper-aware conditioning into two coordinated mechanisms: at the representation level, a multi-granularity tokenizer encodes hardware configurations into decoupled soft prompt tokens spanning platform, gripper-type, and instance granularities; at the execution level, a dual Mixture-of-Adapters (MoA) dynamically routes action tokens at the final action expert layer via parallel platform and gripper gating networks, applying residual policy adjustments without disrupting shared foundation representations.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}, 'subGraphTitleMargin': {'top': 8, 'bottom': 16}}}%%
flowchart TD
    A["Multi-view Observations + Instructions + Hardware ID"] --> B["Multi-Granularity Soft Prompt Tokenization<br/>Platform / Gripper Type / Instance Decoupled Tokens"]
    B --> C["Pretrained VLA Backbone Encoding<br/>Shared Vision-Language Feature Interactions"]
    C --> D["Dual Mixture-of-Adapters Modulation<br/>Parallel Platform Router & Gripper Router Gates"]
    D --> E["Multi-Task Joint Optimization<br/>Flow Matching Action Loss + Auxiliary Prediction + Load Balancing"]
    E --> F["Executable Morphology-Conditioned Robot Actions"]

Key Designs¶

1. Multi-Granularity Soft Prompt Tokenization: Decoupling Platform and Morphology Prior attempts to incorporate end-effector configurations using natural language prompts collapse in latent space, while unconstrained MLPs or VQ-VAEs produce chaotic embeddings that fail to reflect morphological similarities (Fig. 2). To balance cross-robot knowledge sharing with fine-grained physical discrimination, GVLA factorizes embodiment conditioning into a continuous three-level learnable prompt matrix: $$P^{(h)} = \left[ P^{(r)};\, P^{(g)};\, P^{(u)} \right] \in \mathbb{R}^{(p_r + p_g + p_u) \times d}$$ Here, the platform token $P^{(r)} \in \mathbb{R}^{p_r \times d}$ encodes robot arm kinematics and base geometry (e.g., Franka Panda, UR5, xArm7); the gripper-type token $P^{(g)} \in \mathbb{R}^{p_g \times d}$ captures category-level contact mechanics shared by all end-effectors of that class (e.g., parallel-jaw, suction cup, 3-finger, dexterous 5-finger, compliant soft gripper); and the instance token $P^{(u)} \in \mathbb{R}^{p_u \times d}$ preserves unique physical parameters (e.g., specific fingertip stroke, pad friction, or chassis clearance). The composite token is prepended directly to the visual-language token sequence: $$\tilde{X} = \left[ P^{(h)};\, X \right] \in \mathbb{R}^{(p_r + p_g + p_u + n_{\text{obs}}) \times d}$$ This structured factorization allows common mechanism categories to form well-separated clusters in latent space while facilitating rapid few-shot adaptation to novel gripper instances by merely updating instance-level prompts.

2. Dual Mixture-of-Adapters (MoA): Layer-Probed Action Modulation Input soft prompts alone provide contextual conditioning but cannot prevent mode collapse during deep trajectory generation. Using linear probing across the depth of the VLA action expert (Fig. 6), the authors discovered that sensitivity to gripper types remains minimal throughout early and intermediate layers (which handle spatial scene understanding and object semantics) and spikes sharply in the final action generation layer. GVLA therefore inserts the MoA precisely at this final layer. To eliminate interference between arm-level kinematics and gripper-level interaction strategies, GVLA deploys two parallel routing gates: $$G(P) = \text{Softmax}\left(\text{TopK}\left(\text{MLP}(\text{mean}(P))\right)\right)$$ A platform-aware gate $G^{(p)}$ driven by $\text{mean}(P^{(r)})$ selects top-$k$ experts from a platform adapter pool, while a gripper-aware gate $G^{(g)}$ driven by $\text{mean}([P^{(g)}; P^{(u)}])$ selects experts from a type-stratified gripper adapter pool. Each adapter implements a bottleneck residual transformation $\mathcal{A}(x) = \mathcal{W}^{\text{up}}(\text{GeLU}(\mathcal{W}^{\text{down}}(x)))$, dynamically added to the hidden activation: $$x_{\text{out}} = x + \sum_i G^{(p)}_i \cdot \mathcal{A}^{(p)}_i + \sum_j G^{(g)}_j \cdot \mathcal{A}^{(g)}_j$$ This dual routing architecture preserves global kinematic consistency across different robot arms while providing the flexibility needed to generate distinct contact trajectories.

3. Multi-Task Joint Optimization with Load Balancing To guarantee that the shared visual-language representation actively retains gripper-sensitive spatial features and to prevent the routing gates from collapsing onto a few dominant experts, GVLA optimizes a tripartite loss function: $$\mathcal{L} = \mathcal{L}_{\text{action}} + \lambda_{\text{gripper}}\mathcal{L}_{\text{gripper}} + \lambda_{\text{LB}}\mathcal{L}_{\text{LB}}$$ The primary objective $\mathcal{L}_{\text{action}}$ is the conditional flow-matching loss supervising predicted vector fields against noisy action trajectories $u(A_t^\tau | A_t) = \epsilon - A_t$. The auxiliary classification loss $\mathcal{L}_{\text{gripper}}$ predicts the active gripper category from pooled visual observations, compelling the visual backbone to retain affordance-relevant geometric cues. The load balancing regularization $\mathcal{L}_{\text{LB}}$ penalizes the variance of expert activation frequencies across platform and gripper pools, preventing underutilization of specialized adapters and preserving modular representational capacity.

Loss & Training¶

The flow-matching loss serves as the primary optimization objective. Training proceeds via parameter-efficient fine-tuning on the MiGA dataset. The majority of the foundational VLM backbone remains frozen, focusing gradient updates on the learnable prompt embeddings, linear classification heads, and the low-rank bottleneck parameters of the dual MoA. This scheme prevents catastrophic forgetting of broad open-world semantic knowledge while achieving rapid convergence for multi-gripper strategies.

Key Experimental Results¶

Main Results¶

In Isaac Lab simulation environments covering 5 distinct gripper types, GVLA was evaluated against leading traditional grasp detection pipelines and state-of-the-art VLA baselines across four core task scenarios exhibiting significant strategy divergence: Flat object singulation, Stacked clutter, Constrained spaces, and Semantic grasping. Success rates (SR %) from Table 2 are detailed below.

Method	Flat	Stacked	Constrained	Semantic	Avg. (%)
AnyGrasp	0.00	0.00	46.00	0.00	11.50
GraspMAS	0.00	0.00	40.00	8.00	12.00
GraspVLA	1.50	0.00	30.00	0.00	7.88
OpenVLA-OFT	41.00	52.50	24.00	50.00	41.88
$\pi_0$ Baseline	30.00	21.50	36.00	65.00	38.13
$\pi_{0.5}$ Baseline	52.50	71.00	57.50	52.50	58.38
GVLA ($\pi_0$ backbone)	27.50	45.00	50.00	70.00	48.13
GVLA ($\pi_{0.5}$ backbone)	53.00	76.00	62.50	72.50	66.00

Ablation Study¶

Ablations on the MiGA dataset examine the individual contributions of prompt tokens and dual MoA routing, reporting pre-training action prediction error (PE ↓) and adaptation success rate (Adapt. ↑) when transferring to an unseen gripper (Robotiq 2F-85) after 20K fine-tuning steps (from Table 4), along with tokenizer comparisons for counterfactual divergence (CAPD ↑) and gripper contribution score (GCS ↑) (from Table 3).

Configuration / Tokenizer	PE ↓	CAPD ↑	GCS ↑	Adapt. ↑	Note
GVLA (Full Model)	0.032	1.34	0.249	0.92	Full architecture with superior adaptation
w/o Gripper Type Token ($w/o\ P^{(g)}$)	0.037	—	—	0.86	Category priors lost; both PE and adaptation drop
w/o Platform Token ($w/o\ P^{(p)}$)	0.035	—	—	0.54	Disentanglement broken; sharp cross-platform drop
w/o Instance Token ($w/o\ P^{(u)}$)	0.032	—	—	0.90	Minor loss in fine-grained instance precision
w/o Dual MoA ($w/o\ \text{MoA}$)	0.037	—	—	0.52	Lacks execution modulation; adaptation drops 40%
w/o Gripper MoA ($w/o\ \text{MoA}^{(g)}$)	0.033	—	—	0.56	Loss of morphology-specific trajectory routing
w/o Platform MoA ($w/o\ \text{MoA}^{(p)}$)	0.034	—	—	0.54	Kinematic mismatch causes severe execution failure
Tokenizer Baseline: MLP	0.053	0.82	0.014	—	Feature confusion without morphological clustering
Tokenizer Baseline: VQ-VAE	0.051	0.70	0.002	—	Discrete codebook fails to separate gripper mechanisms
Tokenizer Baseline: Language Prompt	0.053	0.64	0.225	—	Textual embeddings collapse in action generation

Key Findings¶

Dual MoA routing is indispensable for cross-gripper transfer: Removing the dual MoA mechanism causes adaptation success to plunge from 0.92 to 0.52, proving that high-level prompt conditioning alone cannot drive diverse low-level continuous action spaces without parameterized execution-layer routing.
Platform and gripper disentanglement is essential: Omitting the platform token $P^{(p)}$ causes only a slight change in in-distribution PE (0.032 vs. 0.035) but cuts unseen gripper adaptation to 0.54, demonstrating that base robot kinematics must be decoupled from end-effector contact dynamics.
Real-robot few-shot generalization: On a physical UR5 arm with a Robotiq 2F-85 gripper evaluated across unseen tasks using only 10 demonstrations and 20k fine-tuning steps, GVLA achieved success rates of 0.50 (Flat), 0.90 (Stacked), 0.90 (Constrained), and 0.85 (Semantic), significantly outperforming the vanilla $\pi_{0.5}$ baseline (0.40, 0.70, 0.70, and 0.75, respectively).

Highlights & Insights¶

First-principles motivation addressing gripper invariance: Uncovers and formalizes the reality that identical task goals require divergent physical strategies depending on end-effector morphology, resolving a blind spot in current VLA research.
Probe-guided architectural intervention: Rather than scattering adapters throughout every Transformer block, the authors leverage empirical linear probe analysis to identify that gripper sensitivity peaks at the final action expert layer, designing a compute-efficient and highly targeted MoA module.
Hierarchical morphology tokenization: Decoupling embodiment into platform, category, and instance embeddings provides a scalable blueprint for foundation models to accommodate diverse robotic hardware without retraining backbone weights.

Limitations & Future Work¶

Lack of explicit local geometric and contact field modeling: GVLA relies on latent soft prompts for strategy-level modulation but does not incorporate explicit 3D mesh representations or signed distance fields (SDF), leading to millimeter-level contact misalignment when adapting within the same gripper family.
Residual entanglement under heavy domain shifts: While the dual MoA promotes expert specialization, vision and embodiment representations remain partially entangled. Under severe visual distribution shifts, policies can over-rely on visual tokens rather than hardware constraints. Incorporating tactile feedback and explicit geometric contact constraints presents a promising path forward.

vs AnyGrasp / GraspMAS / GraspVLA: Open-loop grasp pose detectors and standard grasping foundation models rely heavily on top-down grasp priors, completely failing on flat singulation and occluded clutter (success rates near 0%); GVLA delivers closed-loop, multi-step trajectory planning capable of complex pre-grasp manipulation.
vs $\pi_0$ / $\pi_{0.5}$ / OpenVLA: Mainstream VLAs treat hardware variations as unstructured text prompts or uniform actions, causing policy interference in multi-embodiment setups; GVLA demonstrates that structured prompt tokenization paired with modular MoA routing yields superior task performance and cross-gripper transfer.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ Formulates the critical gripper-invariance limitation in VLAs, supported by the large-scale MiGA benchmark and hierarchical GVLA architecture.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorously validated with 103,000 demonstrations across 5 gripper classes, real-robot UR5 trials, probing analyses, and comprehensive ablations.
Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally clear narrative, insightful visualizations (feature heatmaps and layer probes), and well-grounded physical motivation.
Value: ⭐⭐⭐⭐⭐ Highly influential for scaling general-purpose robot foundation models to diverse industrial and service embodiments.