CulinaryCut: A Physics-aware Vision-Language-Action Benchmark for Food Cutting via Material Point Method¶
Conference: ECCV2026
Paper: ECCV official page · Paper PDF
Area: Robotics / Embodied AI
Keywords: vision-language-action models, food cutting, material point method, contact forces, sim-to-real transfer
This note retains the conference manifest title. The cached PDF is titled CulinaryCut: A Physics-Grounded Cutting Benchmark for Vision-Language-Action Models.
TL;DR¶
CulinaryCut couples MLS-MPM with ManiSkill to generate food-cutting demonstrations with contact-force records, exposing both ratio-grounding and physical-severing failures while improving RDT's orange-cutting success in IntSim from 23% to 59% without changing its architecture or supplying force inputs.
Background & Motivation¶
Moving an object to a designated location primarily tests perception, language grounding, and trajectory execution. Cutting a banana or orange also requires the knife to induce an interaction capable of overcoming material resistance. Reaching the correct location with the correct orientation does not establish that the material has separated. Cutting also changes the object's shape and connectivity, so later cuts cannot always reuse the geometry observed before the first cut. Food cutting therefore probes both spatial understanding and contact dynamics.
Large manipulation datasets such as DROID and Open X-Embodiment provide language and robot actions but were not designed around material fracture. Cutting research such as DiSECt, TopoCut, and SliceIt! does not jointly cover the language instructions, multi-view observations, and continuous robot-action interface targeted here. The missing piece is not another demonstration of a moving knife: images, actions, deformation, separation, and force profiles must describe the same physically consistent interaction. Only then can an evaluator distinguish cutting in the wrong place from reaching the right place without actually cutting through.
The authors put physics into data generation instead of building a new force-conditioned policy. Existing VLAs retain their visual, language, and conventional observation interfaces and learn demonstration actions executed under material dynamics. Core idea: use physically consistent demonstrations and layered evaluation to separate the geometry gap in ratio and direction grounding from the physics gap in material severing, then test whether changing training data alone improves cutting.
Method¶
Overall Architecture¶
CulinaryCut is a dataset and benchmark, not a new VLA network. Ratio, direction, and sequential-division tasks first specify where to cut; IntSim then couples robot execution in ManiSkill with material-state updates in MLS-MPM to produce aligned actions, observations, and force records. Augmented demonstrations fine-tune RDT, Octo, and OpenVLA, which are evaluated for geometric generalization, actual separation in simulation, and limited real-robot transfer.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
Input["Food scene and instruction"] --> Task["Geometric Task Definition"]
Task --> Physics["IntSim Physics Coupling"]
Physics --> Data["Consistent Trajectories and<br/>Two-Axis Augmentation"]
Data --> Policy["Fine-tune existing VLAs<br/>No force inputs"]
Policy --> Eval["Layered Evaluation and<br/>Velocity Guardrail"]
Eval --> Output["Geometric success / Physical severing<br/>Limited real-robot validation"]
Key Designs¶
1. Geometric Task Definition: ground language ratios in the current object extent
A single cut requires the policy to identify the target's size from images and map expressions such as "half" or "one-quarter from the right" to a physical location. Left and right are defined relative to the camera viewpoint. The same ratio combined with a different direction can specify a different location, so memorizing a fixed motion for a keyword is insufficient. The continuous-ratio task spans 0.1 to 0.9, probing whether models learn a continuous object-centric coordinate rather than a few center-cut templates.
A second task asks the robot to divide an object into a specified number of equal pieces. This requires more than repeating an action: later cutting positions must be recomputed as cuts change the object's topology. The paper describes piece count and uniformity as task objectives, but Table 3 reports whether each step's cut position falls within the target tolerance. Those step-wise values should not be treated as final equal-division success rates. The distinction makes failures after a partially successful first cut directly observable.
2. IntSim Physics Coupling: make severing depend on material response, not just the knife path
ManiSkill handles robot execution and visual observations, while the moving least squares material point method, MLS-MPM, handles large deformation and topology changes. Material is represented by particles carrying position, velocity, deformation gradient, and a scalar damage value, with state evolution mediated by particle-grid transfers. Corotated elasticity, J2 plasticity, and a scalar damage model represent elastic response, irreversible deformation, and material failure. A knife passing through a region therefore does not automatically mean that the region has separated.
Food parameters are drawn from food-engineering literature and adjusted for numerical stability. For example, Young's modulus is \(7.0\times10^3\) Pa for banana, \(3.0\times10^6\) Pa for apple, and \(5.84\times10^5\) Pa for orange. Density, Young's modulus, and Poisson's ratio govern mass distribution, stiffness, and deformation response. These quantities parameterize demonstration interactions rather than serving as policy inputs, and should not be mistaken for constants measured separately for every real specimen.
The knife and board are represented as signed distance fields. When contact constraints change material velocities, the simulator accumulates reaction impulses from momentum exchange between tool and material, then computes contact force over the output interval. Force profiles, actions, deformation, and severing consequently arise from the same physical rollout. The main text uses these signals for data construction, analysis, and guardrail-related design, not as policy observations. Full constitutive equations, damage updates, and guardrail implementation are deferred to supplementary material absent from this cache; their specific thresholds cannot be reconstructed from the main text alone.
3. Consistent Trajectories and Two-Axis Augmentation: expand vision and language without inflating independent interaction counts
The pipeline figure starts with teleoperated seed demonstrations for each cutting style, while the main text describes trajectory generation through an axis-aligned bounding box, or AABB, motion planner. The planner estimates object extent, selects a cutting point from the instruction's ratio and direction, and records first contact. Material simulation updates forces, deformation, and topology during contact. Randomization covers object position, scale, rotation, cutting height, and velocity. The imitation target is therefore an action sequence constrained by simulated interaction, rather than a nominal straight line independent of the material.
The base dataset contains 21,000 trajectories over 15 foods: banana, cucumber, apple, peach, melon, orange, strawberry, pear, kiwi, tomato, lemon, grape, shine muscat, plum, and cherry. Each trajectory is rendered in 15 indoor backgrounds drawn from ReplicaCAD, AI2-THOR, and ProcTHOR and paired with at least five language variants. Thus, \(21{,}000\times15\times5=1{,}575{,}000\) denotes approximately 1.57 million augmented instances, not 1.57 million independent physical demonstrations.
The language engine fills templates with object, position, ratio, and direction slots, draws synonymous expressions such as "50%," "half," and "two quarters," and uses LLM paraphrases to broaden wording. Initial object state is not supplied in text, so the policy still has to infer the current situation from images. Each instance contains language, multi-view observations, continuous actions, and a contact-force profile. The presence of force annotations does not imply that a force-regression loss is used during policy training.
4. Layered Evaluation and Velocity Guardrail: distinguish alignment, severing, and transfer
Geometric evaluation covers unseen random seeds for position and scale, multi-object distractions, held-out instructions, and leave-one-object-category-out transfer. The main text defines geometric success through four simultaneous conditions: the knife tip reaches the table surface with positional error below 0.05 m; the blade intersects the designated cutting region; contact angle is \(90^\circ\pm10^\circ\); and the cutting point lies within one-tenth of object size from its target. Physical evaluation in IntSim additionally requires actual material severing, so its success rate must not be conflated with geometric success.
The data-level comparison fixes the RDT architecture and policy inputs. One policy learns from geometrically valid ManiSkill trajectories; the other learns from IntSim trajectories executed and filtered under MLS-MPM-coupled dynamics. Physically grounded demonstrations implicitly express velocity micro-adjustments and sustained contact needed to overcome resistance through their action sequences, rather than providing force, contact, or damage-state observations. All baselines use the same execution-time velocity guardrail, which clips commands outside the Franka arm's safe range without adding policy observations or replanning. This comparison supports the usefulness of physically consistent data, but does not prove that the policy develops explicit material reasoning.
Loss & Training¶
RDT uses diffusion-based action-trajectory modeling, Octo is a generalist robot policy, and OpenVLA predicts actions through a vision-language backbone. All three are fine-tuned on the CulinaryCut training split. The paper introduces neither a shared new loss nor a force-conditioned branch for this comparison. The cached main text does not specify each model's learning rate, batch size, or number of fine-tuning steps; filling these gaps with conventional settings would be unsupported.
Each object and task configuration uses 20 randomized trials with seeds excluded from training. MLS-MPM uses a \(120^3\) grid, \(\Delta t=2\times10^{-5}\) s, and six substeps. Real-force validation uses a Franka Emika Panda performing a constant-speed downward banana cut, with a wrist force/torque sensor recording vertical reaction force at 60 Hz. This is distinct from the orange-cutting policy-transfer experiment and does not constitute multi-material force calibration.
Key Experimental Results¶
Main Results¶
The following success rates are stated explicitly in Section 4.3. The single-object column is the reference performance used in the text, and unseen-object transfer uses a leave-one-category-out setting. These are not per-food success rates for every category.
| Model | Single-object reference | Multi-object scene | Unseen-object transfer |
|---|---|---|---|
| RDT | 69% | 31% | 43% |
| Octo | 42% | 17% | 34% |
| OpenVLA | 51% | 28% | 26% |
RDT loses 38 percentage points in multi-object scenes, showing that explicit target naming does not eliminate visual distraction. Its unseen-object result falls by 26 percentage points. Leaving out a category changes geometry, appearance, and material together, so this decrease cannot be attributed exclusively to stiffness generalization.
Ablation Study¶
Original Table 4 fixes RDT and orange cutting while changing the source of training demonstrations. The real-robot column retains raw success counts to keep the small sample size visible.
| Training data | ManiSkill geometric evaluation | IntSim physical-severing evaluation | Real-robot successes |
|---|---|---|---|
| ManiSkill | 62% | 23% | 1/10 |
| IntSim | 68% | 59% | 3/10 |
Geometric success improves by only six percentage points, whereas physical severing improves by 36 points. The ManiSkill-trained policy has a 39-point gap between the two evaluation environments; the IntSim-trained policy has a nine-point gap. Geometric acceptance therefore cannot substitute for checking material separation. This is a training-data-source ablation, not an ablation that removes force inputs: neither policy receives them.
The next table excerpts the four-piece division analysis from original Table 3. Values are RDT's step-wise cut-position success rates, not complete-task success rates or established conditional probabilities.
| Object | Target | Cut 1 | Cut 2 | Cut 3 |
|---|---|---|---|---|
| Banana | Four equal pieces | 10% | 5% | 0% |
| Cucumber | Four equal pieces | 10% | 0% | 0% |
Key Findings¶
- Direction and ratio changes are not merely synonymous rephrasings. RDT falls from 55% to 20% when transferring from a right-side 0.25 training configuration to its left-side counterpart, and from 60% to 25% when transferring from ratio 0.5 to 0.25. Dense ratio supervision is part of the benchmark design; the main text does not report an isolated ablation gain for it.
- Force validation requires baseline correction. Approximately 25 N of real pre-contact force comes from tool load and offsets. After subtraction, the real peak is approximately 12.5 N versus approximately 12.0 N in IntSim, a relative error within 5%. Normalized curves primarily compare temporal shape and do not establish accurate calibration across all foods.
- Near-zero success on later cuts indicates difficulty relocating targets after topology changes. However, the reported step-wise table checks cutting positions rather than separately quantifying final piece uniformity or complete sequential-task success.
Highlights & Insights¶
- Physics can enter learning through demonstration actions. A policy can imitate movements constrained by material dynamics without directly observing force. This provides a data-level route to improving existing VLAs without immediately changing their network interfaces.
- The evaluation environment changes what success means. The same policy's 62% geometric success and 23% physical severing expose a proxy metric's blind spot. Other contact-rich tasks could similarly report both trajectory correctness and changes in the manipulated object's physical state.
- Independent trajectories and augmented instances should be counted separately. Visual and language augmentation expands observation coverage without a proportional increase in material-interaction diversity. Keeping these counts distinct clarifies what a larger benchmark actually adds.
Limitations & Future Work¶
- The authors acknowledge coverage of only 15 foods, excluding categories such as bread, meat, and non-food deformable objects. The simulator also omits viscoelastic relaxation and moisture-dependent changes, limiting its representation of real material responses.
- Real-force profiles are validated only on banana. Discrepancies remain during force release, plausibly involving adhesion, fiber tearing, and robot-controller compliance. Multi-material, multi-speed validation should jointly examine forces and separation rather than only normalized peak shapes.
- The orange real-robot comparison contains just 10 trials per condition, and 3/10 is far from reliable deployment. The main text does not provide statistical uncertainty analysis sufficient for a strong transfer claim; larger samples and repeated experiments are needed.
- This note's assessment: the paper states 20 trials per configuration but does not fully explain the aggregation denominators behind every summary percentage. The cache also omits supplementary implementation details, so the main text alone does not specify complete training procedures or material-fracture criteria for reproduction.
- A visual domain gap remains. Combining the simulation with visual generative models such as Cosmos is proposed future work, not a demonstrated gain. Augmented datasets should also state whether splits are made at the base-trajectory level so that possible leakage among augmented instances can be audited.
Related Work & Insights¶
- Compared with ForceVLA: ForceVLA conditions policies on force information; CulinaryCut keeps policy interfaces unchanged and places physics in data construction. These approaches could complement each other, but no direct comparison establishes superiority.
- Compared with DiSECt, SliceIt!, and TopoCut: These works address differentiable cutting, dual-simulator slicing, or multi-step cutting. CulinaryCut focuses on organizing material simulation, language, multi-view observations, and continuous actions into a benchmark usable by standard VLAs, rather than claiming the first physically simulated cutting system.
- Compared with DROID and Open X-Embodiment: General manipulation datasets offer broader task coverage, whereas this narrower benchmark adds contact-force and topology detail. A transferable research idea is to include task-specific physical end states in generalist-policy evaluation, rather than increasing image and language volume alone.
Rating¶
- Novelty: 4/5. The contribution is a physically consistent cutting benchmark for VLAs and a two-gap diagnosis, rather than a new material solver or policy architecture.
- Experimental Thoroughness: 3/5. Three policies, several geometric shifts, and a data-source ablation provide useful coverage, but real-force validation and deployment are limited in scale.
- Writing Quality: 4/5. The distinction between geometric success and physical severing is clear; some metric aggregation and implementation details still depend on supplementary material.
- Value: 4/5. The benchmark supplies concrete evaluation questions and data-improvement directions for contact-rich deformable manipulation, while reliable real-world cutting remains substantially unresolved.