---

title: >- [Paper Note] WALL-EVE: World Alignment with Rule Learning in Visual Environments description: >- [ECCV2026][Robotics & Embodied AI][World Model] Addressing the heavy compute overhead and physical hallucinations of generative video world models, WALL-EVE proposes a training-free neurosymbolic alignment framework. By parsing multimodal observations into structured JSON states, inducing and refining executable Python rules from transition conflicts via maximum coverage pruning, it calibrates an LLM world model to drive Model-Predictive Control (MPC), outperforming prior SOTA on ALFWorld and Minecraft with only 17% of baseline token usage. tags: - ECCV2026 - Robotics & Embodied AI - World Model - Neurosymbolic AI - Model-Predictive Control date: 2026-09-19 content_hash: 9369d8fad0c49d5a

WALL-EVE: World Alignment with Rule Learning in Visual Environments

Conference: ECCV 2026
Paper: ECCV Official Link
Area: Robotics & Embodied AI / Reinforcement Learning
Keywords: World Model, Rule Learning, Model-Predictive Control (MPC), Embodied AI, Neurosymbolic System

TL;DR

Addressing the compute-heavy training and physical rule hallucinations of generative video world models, WALL-EVE introduces a training-free neurosymbolic world alignment paradigm that learns compact, executable Python rules from prediction-vs-reality conflicts to calibrate an LLM world model, achieving superior embodied planning under Model-Predictive Control with only 17% of baseline token cost on ALFWorld and Minecraft.

Background & Motivation

Constructing accurate and computationally tractable world models for visual embodied environments is foundational for enabling autonomous agents to perform prospective planning and long-horizon decision-making. Agents rely on world models to simulate, anticipate, and evaluate the consequences of hypothetical action sequences over future horizons without costly real-world trial and error. However, contemporary approaches to visual world models predominantly depend on end-to-end video or image generation architectures (e.g., Sora, Genie, GAIA-1). These generative models incur prohibitive training and inference overheads, and more critically, prioritize low-level pixel textures and visual plausibility over semantic and physical dynamics. Consequently, they suffer from severe hallucinations and violations of foundational physical laws, such as erratic object morphing, loss of object permanence, or arbitrary attribute mutations during multi-step rollouts.

The core tension lies in the fact that embodied decision-making fundamentally hinges on discrete, causal, and logically grounded state transition dynamics rather than raw pixel rendering; yet relying purely on pretrained Large Language Models (LLMs) to predict visual dynamics directly leads to subtle domain misalignment, as generic LLMs lack precise knowledge of environment-specific discrete mechanics (such as container capacities, crafting recipes, and hand-occupancy constraints). Existing alignment efforts attempt either massive reinforcement learning fine-tuning or manual prompt engineering: the former is bottlenecked by catastrophic forgetting and poor sample efficiency, while the latter lacks scalability and fails in open-world settings.

This paper's angle of attack is that generic LLMs already possess rich macroscopic commonsense priors, and environment-specific dynamics gaps can be framed as a compact set of missing residual rules. Instead of training heavyweight generative models from scratch, the environment's multimodal inputs can be verbalized into structured text states, allowing an LLM to inductively extract explicit symbolic rules from observed prediction conflicts. Core idea: establish a training-free neurosymbolic world alignment framework that iteratively induces, refines, and prunes executable code-based rules from trajectory conflicts to calibrate residual errors of an LLM world model, empowering a gradient-free Model-Predictive Control (MPC) planner with high fidelity and minimal token consumption.

Method

Overall Architecture

WALL-EVE operates through an integrated loop consisting of multimodal parsing, neurosymbolic world model forecasting, Model-Predictive Control (MPC) decision-making, and conflict-driven closed-loop rule learning. First, a Multimodal Parser converts raw visual observations and task instructions into unified JSON-formatted state dictionaries. During planning, the agent model leverages LLM commonsense priors as a look-ahead optimizer to propose candidate action sequences over horizon \(H\), which are evaluated by the neurosymbolic world model (LLM + executable rule set) to compute discounted cumulative rewards and select the optimal first action. Once the action is executed in the environment, the resulting actual transition is compared against the predicted transition. Any mismatch triggers an iterative rule learning pipeline—inducing new natural language rules, refining them across historical trajectories, compiling them into executable Python functions, and solving a maximum coverage problem to prune redundant rules—continually updating the compact rule set that guards world model transitions.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Env["Visual Environment (Egocentric Video + Task Text)"] --> Parser["Multimodal Parser<br/>Detector + VQA + Text structuring into JSON state"]
    Parser --> Agent["LLM Look-Ahead Action Optimizer<br/>Proposes candidate H-step action sequences"]
    Agent --> WM["Neurosymbolic World Model<br/>LLM state proposal + executable rule conflict check"]
    WM --> MPC["Model-Predictive Control (MPC) Planning<br/>Evaluates cumulative reward & selects optimal action"]
    MPC --> Act["Environment Execution & Trajectory Collection<br/>Records actual (s, a, s') vs predicted transitions"]
    Act --> RL["Closed-Loop Rule Learning & Pruning<br/>Conflict induction -> Refinement -> Code gen -> Max coverage"]
    RL -->|Updates minimal rule set R_Code| WM

Key Designs

1. Multimodal Parser: Denoising Visual Inputs into Unified Symbolic State
Raw pixel inputs are replete with occlusions, perspective changes, and visual noise that impede reliable symbolic reasoning. WALL-EVE introduces a multimodal parsing function \(f_{\text{parser}}\) that converts raw observation \(s_h = (s_h^t, s_h^v)\) into a structured JSON state representation \(s_h = f_{\text{parser}}(s_h^t, s_h^v)\). For textual task descriptions \(s_h^t\), it extracts normalized goal definitions; for egocentric visual frames \(s_h^v\), it coordinates an object detector (e.g., Mask R-CNN in ALFWorld or VLMs in open-world settings) to identify bounding boxes and object categories, combined with Visual Question Answering (VQA) modules to inspect container states (open/closed, clean/dirty). The resulting JSON state explicitly tracks held items, receptacle contents, and spatial attributes, bypassing expensive pixel-level generation and framing environment simulation as discrete state transformations.

2. Decoupled Two-Stage Transition Modeling: Decomposing Success and State Shift
Directly predicting the full subsequent state \(\hat{s}_{h+1}\) from \((s_h, a_h)\) in open-ended environments is prone to hallucinations and easily corrupted by environmental stochasticity (e.g., weather or dynamic entities). WALL-EVE decomposes dynamics modeling into a two-step chain-of-thought formulation: first predicting binary transition validity \(\hat{suc}_h \in \{0, 1\}\), and subsequently determining state attribute modifications conditioned on that outcome: $\(\hat{suc}_h = f_{\text{wm}}(s_h, a_h), \quad \hat{s}_{h+1} = f_{\text{wm}}(s_h, a_h, \hat{suc}_h)\)$ If an action succeeds (e.g., "craft a wooden pickaxe"), inventory counts increment deterministically while prerequisite materials are deducted; if it fails, the state remains unchanged. This separation reduces cognitive load on the LLM and provides an explicit binary interface for symbolic rule interception.

3. Closed-Loop Code Rule Learning and Maximum Coverage Pruning
Because pretrained LLMs already possess broad commonsense knowledge, they only require corrections on residual edge-case errors. WALL-EVE isolates the set of mispredicted transitions \(\mathcal{D}_{\text{inc}}\) and triggers a five-step neurosymbolic rule learning pipeline: - Conflict Induction: An induction LLM \(f_{\text{gen}}\) compares predicted vs. real transitions in trajectory \(\vartheta^{\text{real}}\) and hypothesizes new candidate natural language rules \(R_{\text{NL}}^{\text{new}}\) explaining the discrepancies; - Trajectory-Wide Refinement: A refinement LLM \(f_{\text{refine}}\) checks all candidate rules against historical trajectories to resolve contradictions and prevent overfitting; - Code Translation: A code generator \(f_{\text{Code}}\) translates verified textual rules into executable Python functions \(r_{\text{Code}}(s, a)\) returning (feedback, success, suggestion); - Maximum Coverage Pruning: To prevent rule bloat and contradictory logic, candidate rules are verified on ground-truth transitions. Any rule that misfires or raises runtime errors is removed. WALL-EVE then formulates rule pruning as a maximum coverage problem, greedily selecting an optimal subset \(\mathcal{R}^* \subseteq \mathcal{R}_{\text{Code}}\) bounded by cardinality \(k\) that maximizes coverage over mispredicted transitions \(\mathcal{D}_{\text{inc}}\): $\(\mathcal{R}^* = \arg\max_{\mathcal{R} \subseteq \mathcal{R}_{\text{Code}}, |\mathcal{R}| \le k} \left| \bigcup_{r_{\text{Code}} \in \mathcal{R}} \mathcal{D}_r \right|\)$ This guarantees that the retained rule set remains compact, executable, and targeted strictly at systematic model errors.

4. Rule-Guarded Post-Prediction Alignment in Model-Predictive Control
At inference time, executable rules act as a post-prediction guardrail rather than an indiscriminate replacement for LLM simulation. When a candidate action is evaluated during MPC rollout, the rule set is queried; only if an active rule produces a success flag \(suc_h^{\text{Code}}\) contradicting the world model's initial prediction \(\hat{suc}_h\) is the prediction corrected: $\(\hat{s}_{h+1} = \begin{cases} f_{\text{wm}}(s_h, a_h, suc_h^{\text{Code}}), & \text{if } suc_h^{\text{Code}} \neq \hat{suc}_h \\ f_{\text{wm}}(s_h, a_h, \hat{suc}_h), & \text{otherwise} \end{cases}\)$ Embedded within Model-Predictive Control, the agent uses its pretrained policy prior \(\omega_{\text{LLM}}\) to generate high-probability action proposals over horizon \(H\), rolling them out through the rule-aligned world model to calculate cumulative discounted rewards \(\sum_{i=0}^{H-1} \varpi^i \mathcal{F}(\hat{s}_{h+i+1})\). This eliminates random trajectory sampling in classical MPC while preventing unaligned hallucinations from corrupting multi-step planning.

A Worked Example

Consider an agent in ALFWorld tasked with placing two cups on the same desk: 1. Initial Unaligned Failure: The agent picks up cup 1. When planning the subsequent action take cup 2 from countertop 2, the unaligned baseline LLM world model incorrectly predicts Success, unaware of the environment's implicit single-hand capacity constraint. In the real environment, the execution fails. 2. Rule Induction & Code Generation: Detecting the conflict between predicted Success and actual Failure, the induction module extracts the rule: "If the agent's hand is already holding an item, picking up another item will fail." This is translated into Python:

def rule_2_take(state, action):
    if action["name"] == "take" and state.get("in_hand") is not None:
        return "hand is already holding an item", False, "put current item down first"
    return "success", True, None

3. Pruning & Integration: The rule correctly covers the failure transition and passes execution verification, being integrated into \(\mathcal{R}_{\text{Code}}\) via maximum coverage. 4. Adaptive MPC Replanning: In subsequent steps, when evaluating candidate sequences, rule_2_take immediately intercepts the action and outputs Failure with near-zero reward. The MPC optimizer adapts the look-ahead plan to go to desk 1 \(\to\) put cup 1 on desk 1 \(\to\) take cup 2 from table 1, clearing hand occupancy before picking up the second cup and succeeding seamlessly.

Key Experimental Results

Main Results

Experiments were conducted on ALFWorld (134 test tasks across 6 household categories, evaluated in both visual and text settings) and Minecraft (MineDojo TechTree covering 6 difficulty tiers from Wood to Redstone).

Table 1: Performance Comparison on ALFWorld Benchmark (Success Rate %, Table 1 in paper)

Paradigm	Method	Visual Avg.	Pick	Clean	Heat	Cool	Look	Pick2	Text Avg.
Trained	MiniGPT-4	16	4	0	19	17	67	6	-
Trained	InstructBLIP	22	50	26	23	6	17	0	-
Task-Trained	EMMA (specialized)	82	71	94	85	83	88	67	-
Training-free	ReAct	50	66	31	61	70	32	39	54
Training-free	AdaPlanner	62	83	75	54	63	46	35	91
Training-free	Reflexion	68	79	71	51	72	61	74	86
Training-free	RAFA (prior SOTA)	72	86	72	65	71	58	77	95
Training-free	WALL-EVE (Ours)	75	88	77	61	68	61	94	95

Note: EMMA is specifically trained on ALFWorld tasks, whereas WALL-EVE is entirely training-free. On the complex two-object interaction task (Pick2), WALL-EVE achieves 94% success rate due to precise hand-capacity rule alignment.

Table 2: Comparison on Minecraft TechTree Tasks (Success Rate % and Replanning Rounds, Table 3 in paper)

Method	Framework Components	Avg. SR (%) \(\uparrow\)	Avg. Rounds \(\downarrow\)	Wooden	Stone	Iron	Golden	Diamond	Redstone
GPT-4o (Zero-shot)	Rea	9	46.70	36 (35.50)	19 (44.72)	0 (-)	0 (-)	0 (-)	0 (-)
DEPS	Rea, Ref	37	35.36	83 (10.67)	41 (33.26)	33 (35.27)	22 (45.29)	24 (42.46)	17 (45.22)
Plan4MC	Rea, SkLi	48	28.71	91 (5.77)	75 (13.01)	43 (34.03)	32 (36.59)	26 (39.38)	21 (43.47)
Jarvis-1 / Voyager / GITM	Rea, Ref, Me/SkLi	54	25.49	96 (3.42)	92 (6.01)	57 (23.93)	29 (37.17)	30 (39.80)	22 (42.63)
Optimus-1	Rea, Ref, Me, KnGr	61	22.45	95 (3.29)	94 (4.68)	64 (21.69)	41 (30.49)	31 (39.88)	38 (34.67)
WALL-EVE (Ours)	Rea, Ref, Ru	69	15.77	98 (1.64)	91 (4.58)	63 (19.38)	69 (15.61)	46 (27.08)	48 (26.33)

Values in parentheses indicate replanning rounds (lower reflects fewer execution failures and higher sample efficiency). WALL-EVE demonstrates substantial margins on advanced tiers (Gold, Diamond, Redstone).

Table 3: Token Usage and Financial API Cost (Tables 2 & 4 in paper)

Benchmark	Metric	ReAct	AdaPlanner	Reflexion	RAFA / Optimus-1	WALL-EVE (Ours)
ALFWorld	Avg. Token Cost per task \(\downarrow\)	53,210.94	74,560.55	68,727.18	392,268.28 (RAFA)	68,288.96
	API Cost (USD) \(\downarrow\)	0.37	0.52	0.48	2.73 (RAFA)	0.47
Minecraft	Avg. Token Cost per task \(\downarrow\)	-	-	-	70,566.67 (Optimus-1)	60,348.71
	API Cost (USD) \(\downarrow\)	-	-	-	0.48 (Optimus-1)	0.41

Ablation Study

Table 4: Ablation on Agent and World Model Rule Configurations (Minecraft, Table 5 in paper)

Agent Configuration	World Model Configuration	Success Rate (%) \(\uparrow\)	Replanning Rounds \(\downarrow\)	Insights
LLM	None (Model-free)	37	35.36	Pure reactive behavior lacks prospective trajectory rollout
LLM	LLM (Unaligned)	38	33.53	Unaligned world model hallucinates, offering minimal gain (+1%)
LLM + rules	None (Model-free)	61	23.13	Hardcoded prompt rules guide agent but lack multi-step lookahead
LLM	LLM + rules (WALL-EVE)	69	15.77	Optimal decoupling: agent maintains high entropy, WM guards rules
LLM + rules	LLM + rules	67	16.59	Rule-restricted agent suffers "entropy collapse", hurting exploration

Table 5: Ablation of Rule Pruning Limits on Minecraft (Table 7 in paper)

Rule Set Configuration	Success Rate (%) \(\uparrow\)	Replanning Rounds \(\downarrow\)	Takeaway
Without Pruning (All rules kept)	12	46.79	Conflicting and noisy rules paralyze agent planning
Max Coverage Pruned (Limit \(k=2\))	52	24.11	Correcting the primary failure modes restores basic performance
Max Coverage Pruned (Limit \(k=4\))	64	18.71	Captures the majority of transition discrepancies
Max Coverage Pruned (No limit / Greedy)	69	15.77	Yields the cleanest, highest-impact rule subset

Table 6: World Model Prediction Fidelity on 500 Held-Out Transitions (Table 8 in paper)

Metric	Environment	LLM Alone	LLM + Rules (Ours)	Absolute Gain
Success/Failure Acc. (%) \(\uparrow\)	ALFWorld	73	94	+21%
	Minecraft	65	85	+20%
Next-state Attr. Acc. (%) \(\uparrow\)	ALFWorld	70	93	+23%
	Minecraft	59	77	+18%
5-step Rollout Attr. Acc. (%) \(\uparrow\)	Minecraft	31	68	+37%

Key Findings

• World Model Alignment is the Key Bottleneck for MPC: As shown in Table 4, introducing an unaligned LLM world model into MPC yields virtually no benefit over model-free LLM planning (37% vs 38%). Once aligned with residual rules, success surges to 69%, verifying that inaccurate world models misdirect look-ahead search.
• Asymmetric "Entropy-Preserving" Architecture: Restricting both the agent and the world model with rules causes performance to drop from 69% to 67% (Table 4). Hard-constraining the agent model causes "entropy collapse", dampening exploration and diversity during candidate action generation. Maintaining an unconstrained agent optimizer coupled with a strictly rule-aligned world model validator delivers the best synergy.
• Suppression of Cascading Rollout Errors: In multi-step autoregressive rollouts (Table 6), baseline LLM attribute accuracy plummets to 31% after 5 steps due to cascading hallucinations. The rule-aligned world model maintains 68% accuracy, enabling reliable deep-tree search in MPC.
• Dramatic Token Efficiency: In ALFWorld, WALL-EVE outperforms RAFA while requiring only 17% of RAFA's token consumption (68k vs 392k tokens, Table 3). Executable Python checks run in sub-milliseconds with zero API cost, eliminating the heavy overhead of multi-turn verbal self-reflection.

Highlights & Insights

• Residual Neurosymbolic Alignment: Rather than training massive video foundation models from scratch, WALL-EVE leverages the commonsense foundation of general LLMs and frames domain alignment as learning a lightweight set of residual Python rule guards.
• Maximum Coverage Formulation for Rule Curating: Modeling rule pruning as a maximum coverage problem provides a principled safeguard against rule bloat, filtering out runtime bugs, tautological rules, and conflicting predicates while maximizing empirical error correction.
• Zero-Token Millisecond Guardrails: Replacing expensive prompt-based LLM self-reflection loops with local executable code checks dramatically slashes inference latency and financial cost, offering an exceptionally viable design for practical deployment.

Limitations & Future Work

• Discrete Assumption in Dynamics: Current rule induction operates on discrete states and boolean predicates (e.g., inventory counts, daylight thresholds, container status). Modeling continuous dynamics (fluid mechanics, rigid body velocity, continuous motor torques) remains beyond the scope of discrete Python assertions.
• Perception Module Dependency: The system relies heavily on the accuracy of the upstream multimodal parser (e.g., Mask R-CNN / VQA). While rule induction is robust to sporadic visual noise (Table 6), persistent systematic perception failures directly corrupt the JSON state and can mislead rule generation.
• Future Directions: Extending the framework to differentiable neurosymbolic representations with tunable continuous parameters, and integrating active visual probing where the agent proactively asks VQA questions to resolve state ambiguities.

Related Work & Insights

• vs. Generative Video World Models (Genie, GAIA-1, Sora): Generative models suffer from immense computational overhead and long-horizon visual drift. WALL-EVE operates entirely in parsed semantic space, guaranteeing physical consistency with deterministic code execution and negligible inference latency.
• vs. Full-Code World Models (WorldCoder): WorldCoder attempts to synthesize the entire environment simulator in code from scratch, which is error-prone in open-ended domains. WALL-EVE relies on the LLM for 90% of commonsense state evolution, generating code only for sparse residual conflicts.
• vs. Memory / Skill Library Agents (Reflexion, Voyager, Jarvis-1): Traditional agents accumulate reflections into memory prompts or action skill libraries. WALL-EVE demonstrates that deploying learned constraints to guard world model simulation yields significantly higher planning performance (+10% gain) than feeding them directly into the agent.

Rating

• Novelty: ⭐⭐⭐⭐⭐ (Formulates world model alignment as residual neurosymbolic rule learning with maximum coverage pruning in a training-free MPC framework)
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ (Rigorous evaluation across two distinct embodied benchmarks, testing multi-step rollout fidelity, rule pruning boundaries, and agent entropy dynamics)
• Writing Quality: ⭐⭐⭐⭐⭐ (Clear mathematical formulation, thorough experimental tables, and coherent architectural motivation)
• Value: ⭐⭐⭐⭐⭐ (Provides a highly practical, token-efficient paradigm for deploying LLM world models in visual environments without GPU-heavy video generation)

Related Papers

• [ECCV 2026] Divide and Conquer: Decoupled Representation Alignment for Multimodal World Models
• [ICML 2026] Generative Visual Code Mobile World Models
• [NeurIPS 2025] RLVR-World: Training World Models with Reinforcement Learning
• [ICLR 2026] Designing Rules to Pick a Rule: Aggregation by Consistency
• [ECCV 2026] V-Co: A Closer Look at Visual Representation Alignment via Co-Denoising