Inferring the Unspoken: Aligning Embodied Agents with Implicit Preferences¶
Conference: EMNLP 2026
Paper: ACL Anthology
Project: Project Page
Area: Robotics & Embodied AI
Keywords: Embodied AI, Pragmatic Reasoning, Implicit Preference Alignment, Two-Stage Planning, PBP Benchmark
TL;DR¶
Addressing the failure of embodied agents to satisfy idiosyncratic user preferences under underspecified natural language commands, this paper introduces the 290-preference PBP benchmark and the InTU framework, demonstrating that verbalizing latent preferences prior to planning cuts alignment error roughly in half and localizes the bottleneck to preference acquisition rather than downstream execution.
Background & Motivation¶
Natural language serves as a flexible interface for human-robot collaboration, yet human instructions in embodied household environments are almost never exhaustive. Under Gricean pragmatics and Zipf's principle of least effort, human speakers systematically omit whatever contextual information they presume the listener can supply. An instruction such as "prepare an apple" or "tidy up the kitchen" is inherently underspecified: it fixes neither whether the fruit must be washed, peeled, or sliced, nor where the items should be placed. Crucially, such choices vary sharply across individuals and are rarely articulated verbally; instead, they are enacted in routine behavior. While multimodal foundation models have advanced embodied commonsense reasoning and task decomposition, they settle these residual degrees of freedom with generic behavioral priors, completely missing user-specific idiosyncrasies.
The core tension stems from the prevailing paradigm of implicit visual imitation, which attempts to map behavioral demonstration videos directly to physical action sequences end-to-end. This formulation conflates two fundamentally distinct challenges: inferring what latent constraint governs the user's intent, and grounding a feasible kinematic plan that respects it. Human behavioral demonstrations are high-dimensional, sparse, and dominated by task-irrelevant perceptual variance such as camera viewpoints, illumination, background clutter, and object instances. Meanwhile, the governing preference itself is sparse, symbolic, and semantic (e.g., "wash fruits before cutting" or "store perishables in the refrigerator"). Requiring an agent to perform end-to-end mapping from raw pixels under few-shot demonstrations forces it to overfit to superficial environmental features, causing the policy to collapse when transferred across unfamiliar rooms.
The authors approach this dilemma through the lens of Theory of Mind and pragmatic reasoning: language provides an ideal intermediate semantic abstraction that bridges noisy continuous perception and discrete symbolic decision-making. Verbalizing behavioral regularities into explicit natural language rules allows the agent to preserve human intent while discarding perceptual clutter. The core idea is to factorize personalized embodied planning into a two-stage framework (InTU) that first verbalizes latent user preferences from few-shot demonstrations and then plans conditioned on that explicit rule, establishing on the new PBP benchmark that the primary bottleneck in personalization lies in preference acquisition rather than downstream planning.
Method¶
Overall Architecture¶
The Inferring the Unspoken (InTU) framework formalizes personalized embodied planning as a two-stage Bayesian pragmatic inference problem. In the first stage, the agent examines a few-shot set of unlabeled behavioral demonstrations alongside the current environment context, inferring and verbalizing a compact natural-language constraint describing the user's latent preference. In the second stage, this linguistic rule is injected into an embodied planner as an explicit semantic condition, guiding the generation of an executable sequence of parameterized action primitives. The overall system is underpinned by the Preference-based Planning (PBP) hierarchical benchmark.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input: Few-shot Unlabeled Demonstrations + Underspecified Instruction + Scene"] --> B["Hierarchical Pragmatic Preference Benchmark<br/>290 preferences across Atomic / Strategic / Temporal tiers"]
B --> C["Context-Driven Implicit Preference Verbalization<br/>Identify behavioral regularities and emit explicit natural language constraint"]
C --> D["Preference-Conditioned Action Sequence Planning<br/>Generate grounded action primitives respecting the inferred semantic rule"]
D --> E["Output: Aligned Physical Action Execution Sequence"]
Key Designs¶
1. Hierarchical Pragmatic Preference Benchmark: Establishing a three-tiered semantic evaluation field Existing embodied planning benchmarks (such as ALFRED, Habitat, and LoTa-Bench) evaluate task success against explicit task specifications. PBP instead evaluates the recovery of unspoken, user-specific constraints across 290 semantic preferences structured into a three-tiered hierarchy: - Atomic Semantics (Action level, 75 types): Captures continuous execution parameters omitted from verbal instructions (e.g., tap water temperature, slicing thickness, pouring "half a cup" vs. "full cup"). The core challenge lies in mapping continuous environmental state variations into discrete semantic attributes. - Strategic Semantics (Option level, 135 types): Dictates category-level placement and interaction policies (e.g., storing fruits on tables vs. in refrigerators, organizing tools into upper vs. lower shelves). This level demands category-level abstraction capable of generalizing across diverse object instances and rooms. - Temporal Semantics (Sequence level, 80 types): Governs long-horizon ordering and causal dependencies among subtasks (e.g., "clean surfaces before placing items", "wash fruits before cutting"). The preference cannot be extracted from any single visual frame, residing entirely in the relative temporal structure of actions.
Built within NVIDIA's OmniGibson simulator across 50 photorealistic interactive indoor scenes and over 3,000 object assets, PBP comprises 5,000 evaluation groups (15,000 demonstration episodes in total). Each group provides \(K=3\) unlabeled demonstration episodes together with an underspecified query instruction, ensuring that multiple regularities coexist and forcing the agent to extract the specific constraint relevant to the prompt.
2. Context-Driven Implicit Preference Verbalization: Compressing multimodal demonstrations into compact linguistic rules In end-to-end architectures, high-dimensional visual demonstrations cause models to correlate spurious visual features (such as wall color or object texture) with task success. InTU introduces an explicit latent variable \(\hat{S}_{\text{pref}}\) to formulate preference acquisition as pragmatic inference:
where \(\mathcal{C}_{\text{demo}} = \{X_i\}_{i=1}^K\) denotes the set of unlabeled demonstration episodes (egocentric video frames paired with top-down bird's-eye maps in the visual setting, or symbolic action logs in the symbolic setting), \(I\) is the underspecified natural language instruction, and \(E_{\text{curr}}\) is the current environment state. The multimodal foundation model isolates the relevant behavioral regularity from the demonstrations and commits to an explicit verbalized statement (e.g., "Preference: wash fruits before cutting"). This linguistic compression discards perceptual noise, provides human-interpretable transparency, and yields rules that transfer seamlessly across visually disparate environments.
3. Preference-Conditioned Action Sequence Planning: Decoupling perceptual noise for robust cross-scene generalization Once the implicit preference has been verbalized into natural language, downstream action planning reduces to standard conditional plan generation without requiring the planner to concurrently deduce latent user intent:
The planner consumes the environment context \(E_{\text{curr}}\), instruction \(I\), and the inferred semantic preference \(\hat{S}_{\text{pref}}\) to synthesize an executable plan from PBP's 17 parameterized action primitives (including Move_to, Pick, Place, Cut, Cook, Wash, Clean, and Toggle_on/off, executed via A* pathfinding and Inverse Kinematics / OMPL motion planners). By decoupling preference inference from execution, this setup enables an oracle evaluation setting (Second-stage gt) that directly exposes the true residual error of action planning.
A Worked Example¶
Consider an evaluation episode under the preference Put fruit on the bed:
1. Behavioral Context \(\mathcal{C}_{\text{demo}}\):
- Demo 1 (Scene Beechwood): The user moves to an orange, picks it up, and places it on the bed; picks up an apple and places it on the bed; picks up a DVD and places it on the table.
- Demo 2 (Scene Rs): The user picks up a banana and places it on the bed; places a sandwich on the countertop; places another orange on the bed.
- Demo 3 (Scene Merom): The user places a pen on the desk; places a strawberry on the bed; places a textbook into a bookshelf.
2. Current Query: The agent is placed in scene Beechwood containing a grape, a pencil, and a banana, receiving the underspecified instruction \(I\): "Organize the items".
3. InTU Stage 1 (Preference Inference): Jointly inspecting the demonstrations, the model identifies the common categorical rule that all fruits are routed to the bed while non-food stationery and media are routed to tables or shelves, verbalizing \(\hat{S}_{\text{pref}}\): "Put fruit on the bed".
4. InTU Stage 2 (Conditioned Planning): Conditioned on \(\hat{S}_{\text{pref}}\), the planner categorizes grape and banana as fruits and pencil as stationery, generating the aligned sequence: [Move_to(grape), Pick(grape), Place(grape, bed), Move_to(banana), Pick(banana), Place(banana, bed), Move_to(pencil), Pick(pencil), Place(pencil, table)]. By contrast, an end-to-end baseline (e.g., Llama 3) fails to infer the category-level constraint, erroneously placing the pencil on the bed alongside the fruits.
Key Experimental Results¶
Main Results¶
Alignment between generated action sequences and human ground-truth executions is evaluated via Levenshtein distance (\(\downarrow\), lower indicates closer alignment with the user). Evaluations span both video-based inputs (ViViT, LLaVA-NeXT, EILEV, Gemini-3.1-Pro) and symbolic action traces (Llama3-8B, DeepSeek-v4-pro, Claude-Opus-4.7, GPT-5.5).
| Input Modality | Model | Option Level \(\downarrow\) | Sequence Level \(\downarrow\) | Overall \(\downarrow\) |
|---|---|---|---|---|
| Video Input | ViViT (Video-only baseline) | 15.49 \(\pm\) 1.29 | 34.04 \(\pm\) 11.84 | 24.76 |
| LLaVA-NeXT (End-to-end) | 15.94 \(\pm\) 3.41 | 34.76 \(\pm\) 11.25 | 25.35 | |
| LLaVA-NeXT (Two-stage InTU) | 12.46 \(\pm\) 3.23 | 30.02 \(\pm\) 13.54 | 21.24 | |
| LLaVA-NeXT (Second-stage gt) | 3.28 \(\pm\) 5.29 | 18.92 \(\pm\) 14.18 | 11.10 | |
| Gemini-3.1-Pro (End-to-end) | 8.72 \(\pm\) 2.26 | 24.51 \(\pm\) 4.15 | 16.62 | |
| Gemini-3.1-Pro (Two-stage InTU) | 4.28 \(\pm\) 1.76 | 12.57 \(\pm\) 2.92 | 8.43 | |
| Gemini-3.1-Pro (Second-stage gt) | 0.16 \(\pm\) 1.82 | 10.24 \(\pm\) 2.58 | 5.20 | |
| Symbolic Input | Llama3-8B (End-to-end) | 14.74 \(\pm\) 3.21 | 31.79 \(\pm\) 7.32 | 23.26 |
| Llama3-8B (Two-stage InTU) | 9.67 \(\pm\) 5.16 | 25.46 \(\pm\) 5.93 | 17.56 | |
| Llama3-8B (Second-stage gt) | 8.22 \(\pm\) 5.58 | 19.02 \(\pm\) 7.10 | 13.62 | |
| DS-v4-pro (End-to-end) | 6.42 \(\pm\) 2.33 | 22.84 \(\pm\) 4.01 | 14.63 | |
| DS-v4-pro (Two-stage InTU) | 2.31 \(\pm\) 1.67 | 12.10 \(\pm\) 2.85 | 7.21 | |
| DS-v4-pro (Second-stage gt) | 0.16 \(\pm\) 1.46 | 10.19 \(\pm\) 2.47 | 5.18 | |
| Claude-Opus-4.7 (End-to-end) | 6.72 \(\pm\) 2.48 | 21.47 \(\pm\) 3.12 | 14.10 | |
| Claude-Opus-4.7 (Two-stage InTU) | 2.56 \(\pm\) 2.04 | 13.84 \(\pm\) 2.81 | 8.20 | |
| Claude-Opus-4.7 (Second-stage gt) | 0.16 \(\pm\) 1.99 | 9.17 \(\pm\) 2.90 | 4.67 | |
| GPT-5.5 (End-to-end) | 6.07 \(\pm\) 2.00 | 21.92 \(\pm\) 3.45 | 14.00 | |
| GPT-5.5 (Two-stage InTU) | 2.41 \(\pm\) 1.72 | 12.48 \(\pm\) 2.45 | 7.45 | |
| GPT-5.5 (Second-stage gt) | 0.14 \(\pm\) 1.21 | 9.77 \(\pm\) 2.69 | 4.96 |
Ablation Study¶
The first ablation tracks preference inference accuracy (Accuracy \(\uparrow\)) prior to planning, contrasting the Few-shot setting (\(K=3\)) against an Ablative zero-shot setting where demonstrations are withheld:
| Setting | Model | Option Level Acc \(\uparrow\) | Sequence Level Acc \(\uparrow\) | Overall Acc \(\uparrow\) |
|---|---|---|---|---|
| Few-shot (K=3 Demos) | Gemini-3.1-Pro (Video) | 63.28% | 52.10% | 57.69% |
| LLaVA-NeXT (Video) | 36.87% | 24.85% | 30.86% | |
| EILEV (Video) | 38.33% | 32.69% | 35.51% | |
| ViViT (Video) | 9.38% | 4.24% | 6.81% | |
| Llama3-8B (Symbolic) | 72.98% | 67.18% | 70.08% | |
| DeepSeek-v4-pro (Symbolic) | 91.28% | 76.48% | 83.88% | |
| Claude-Opus-4.7 (Symbolic) | 89.91% | 76.92% | 83.42% | |
| GPT-5.5 (Symbolic) | 92.48% | 78.01% | 85.25% | |
| Ablative (Zero-shot) | Gemini-3.1-Pro (Video) | 31.92% | 14.28% | 23.10% |
| LLaVA-NeXT (Video) | 15.47% | 8.13% | 11.80% | |
| Llama3-8B (Symbolic) | 39.50% | 6.25% | 22.88% | |
| DeepSeek-v4-pro (Symbolic) | 83.55% | 18.58% | 51.07% | |
| GPT-5.5 (Symbolic) | 82.01% | 18.71% | 50.36% |
The second ablation analyzes explicit preference reasoning (Table A8), comparing vanilla End-to-End, internal Preference-aware reasoning without outputting text (P-aware), and explicit Two-stage InTU:
| Model | Reasoning Mode | Option Level \(\downarrow\) | Sequence Level \(\downarrow\) | Overall \(\downarrow\) |
|---|---|---|---|---|
| Gemini-3.1-Pro | Vanilla End-to-end | 8.72 \(\pm\) 2.26 | 24.51 \(\pm\) 4.15 | 16.62 |
| P-aware End-to-end | 4.56 \(\pm\) 1.96 | 15.73 \(\pm\) 3.55 | 10.14 | |
| Two-stage InTU (Verbalized) | 4.28 \(\pm\) 1.76 | 12.10 \(\pm\) 2.85 | 8.19 | |
| Llama3-8B | Vanilla End-to-end | 14.74 \(\pm\) 3.21 | 31.79 \(\pm\) 7.32 | 23.26 |
| P-aware End-to-end | 9.24 \(\pm\) 4.11 | 28.13 \(\pm\) 5.34 | 18.68 | |
| Two-stage InTU (Verbalized) | 9.67 \(\pm\) 5.16 | 25.46 \(\pm\) 5.93 | 17.56 |
Key Findings¶
- Acquisition, not planning, is the primary failure mode: Providing ground-truth preferences (Second-stage gt) drives option-level planning error close to zero for frontier models (Gemini 0.16, GPT-5.5 0.14). However, when required to infer those same preferences from behavior, edit distance surges by more than an order of magnitude. This isolates the failure to preference acquisition rather than physical action decoding.
- Verbalization yields orthogonal benefits over internal reasoning: Explicitly articulating the preference as text (Two-stage InTU) outperforms internal latent consideration (P-aware), yielding an additional 3.6-point error reduction on complex sequence-level planning for Gemini-3.1-Pro (15.73 vs. 12.10).
- Perceptual abstraction remains a significant bottleneck: Top VLMs operating on video keyframes (Gemini-3.1-Pro, 57.69% accuracy) lag heavily behind LLMs operating on clean symbolic traces (GPT-5.5, 85.25%). In cross-scene evaluation (Direct vs. Gen), visual models suffer substantial degradation (Gemini falls from 68.21% to 52.10% on sequence tasks), whereas symbolic models retain rule invariance (GPT-5.5 only shifts from 86.90% to 78.01%).
Highlights & Insights¶
- Pragmatic grounding as a foundational embodied principle: Recasting personalized embodied planning as pragmatic Theory of Mind inference addresses the reality that natural human communication is inherently underspecified.
- Language as a robust noise-filtering bottleneck: Compressing high-dimensional multimodal interaction traces into symbolic linguistic statements strips away perceptual noise and spatial layout variations, enabling zero-shot rule transfer to novel environments.
- Rigorous three-tiered benchmark taxonomy: PBP establishes a comprehensive standard spanning action parameters, categorical placements, and temporal causal dependencies, providing actionable diagnostics for embodied personalization.
Limitations & Future Work¶
- Simulation-to-reality gap: PBP is curated within OmniGibson simulation environments; real-world environments introduce complex physical deformations, sensor noise, and unscripted human dynamics.
- Discrete predefined preference ontology: The 290 discrete preference types, while diverse, cannot fully capture continuous drifts, conflicting habits, or contextual nuances of real human behavior.
- Assumption of pre-filtered demonstration context: The benchmark assumes \(K=3\) relevant demonstrations are supplied directly; deployed embodied systems will need to actively retrieve and filter relevant interaction histories from long-term episodic memory.
Related Work & Insights¶
- vs. Standard Embodied Instruction Benchmarks (ALFRED, Habitat): Standard benchmarks assume fully specified instructions and measure execution success; PBP specifically investigates resolving ambiguous instructions via historical behavioral context.
- vs. Heuristic Tidying Systems (TidyBot, Kapelyukh et al.): Prior personal assistants focus on object rearrangement within static rooms; InTU evaluates a structured hierarchy across action, option, and temporal causal constraints across diverse environments.
- vs. End-to-End Visual Imitation (VLP, EILEV): Direct video imitation overfits to scene textures; InTU demonstrates that explicit linguistic mediation produces vastly superior cross-scene generalization.
Rating¶
- Novelty: โญโญโญโญโญ Formulates implicit embodied preference alignment through pragmatic inference, identifying the preference acquisition bottleneck with clarity.
- Experimental Thoroughness: โญโญโญโญโญ Rigorous evaluation across 8 multimodal/symbolic models, isolating perception, reasoning, scaling, and scene transfer.
- Writing Quality: โญโญโญโญโญ Elegant conceptual framing, clear mathematical problem formulation, and compelling analytical depth.
- Value: โญโญโญโญโญ Provides a foundational benchmark and architectural blueprint for personalized, human-aligned robotic agents.