Re2A: Situated Conversational Recommendation via Rubric-based Preference Reasoning and Alignment¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/DongdingLin/Re2A
Area: Recommender Systems
Keywords: situated conversational recommendation, rubric-based reasoning, preference tracking, dual alignment, reinforcement learning
TL;DR¶
Re2A formulates situated conversational recommendation as a structured reason-then-align process, using automated rubric-guided policy optimization to infer explicit preference states and heterogeneous preference-conditioned DPO to resolve the dual alignment dilemma between user needs and physical scene constraints.
Background & Motivation¶
Conversational recommendation systems (CRS) have long aimed to deliver personalized recommendations through interactive natural language dialogues. However, the vast majority of existing conversational recommendation frameworks operate exclusively in language-only settings, completely detached from the physical environments in which users interact. In real-world shopping or retail scenarios—such as browsing fashion apparel or arranging home furniture—interactions are inherently situated: conversational assistants and human users share a co-observed physical visual space. This imperative has driven the study of Situated Conversational Recommendation (SCR), transforming recommendation agents from isolated textual responders into physically grounded, context-aware assistants that must jointly reason over multi-turn dialogue histories, visual scenes, and fine-grained in-scene item attributes.
Despite this promising paradigm shift, situated conversational recommendation introduces two fundamental, intertwined challenges that severely degrade standard multimodal models. First, inferring situated user preferences across multimodal contexts is notoriously difficult. Unlike text-only dialogues where constraints are directly stated, situated user preferences are deeply entangled with the physical scene, frequently involving spatial references, visual grounding, negation constraints (e.g., "anything but that light wood shelf"), and dynamic intent shifts. Black-box multimodal models suffer from a severe "reasoning deficit", easily succumbing to visually salient distractors or hallucinated justifications that drift from actual physical cues. Second, response generation in SCR faces a persistent "alignment dilemma." Recommendations must simultaneously satisfy latent user desires and remain strictly faithful to physical environment constraints. Conventional models either hallucinate non-existent items to appease user preferences, or recommend scene-valid items that completely violate the user's explicit budget or stylistic requirements.
To resolve these tensions, neither end-to-end black-box generation nor unconstrained free-form Chain-of-Thought (CoT) suffices, as both lack verifiable process-level supervision and collapse multi-dimensional constraints into monolithic objectives. This work re-envisions situated conversational recommendation as a transparent, two-stage reason-then-align paradigm. Core idea: decouple situated preference reasoning from response generation by leveraging automated context-aware rubrics to guide policy optimization into explicit, inspectable preference states, and subsequently optimizing response generation via preference-conditioned DPO with heterogeneous negative samples to achieve dual alignment with both user intent and physical scene reality.
Method¶
Overall Architecture¶
Re2A structures situated conversational recommendation into a cascaded, modular pipeline that bridges raw multimodal observations with faithful natural language recommendations. Given the situated context \(X_t\) comprising the co-observed visual scene \(S_t\), candidate items \(I_t\), and multi-turn dialogue history \(C_t\), Re2A avoids direct end-to-end black-box response synthesis. Instead, the preference reasoning model \(\pi_\theta\) first produces an intermediate reasoning trace along with a structured preference state \(P\). A deterministic preference-to-item parser maps this explicit state onto physical scene candidates to resolve the target item anchor \(\hat{o}\). Finally, the response generation model \(\phi_\theta\) generates the recommendation utterance conditioned jointly on the situated context, the inferred preference state, and the grounded item anchor.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Situated Context Input<br/>Visual Scene S + Candidates I + Dialogue History C"] --> B["Dynamic Rubric-Guided Explicit Preference Reasoning<br/>Dynamic gating selects active rubric subset R_act<br/>GRPO optimizes reasoning thoughts and structured preference state P"]
B --> C["Cascaded Constrained Item Retrieval and Parsing<br/>Deterministic parser projects P constraints onto visible candidate items<br/>Filters distractors and outputs top-ranked physical anchor entity ô"]
C --> D["Heterogeneous Preference-Conditioned Aligned Generation<br/>Conditions on context tuple with anchored entity ô<br/>Trains on mixed situation-conflicting and preference-conflicting negatives via DPO"]
D --> E["Final Output<br/>Physically grounded target entity ô + Dually aligned recommendation response Y"]
Key Designs¶
1. Dynamic Rubric-Guided Explicit Preference Reasoning: Mitigating Black-Box Reasoning Deficit via Multi-Faceted Predicates and Group Policy Optimization
In situated conversational contexts, user preferences are deeply latent and implicit, whereas conventional training only provides end-to-end supervision on the final recommended item ID. This sparse signal causes models to learn superficial visual correlations while failing to master the underlying spatial and logical causal reasoning. To provide verifiable, process-oriented guidance without incurring prohibitive human annotation costs, Re2A introduces an automated rubric induction mechanism driven by a frozen teacher MLLM. The resulting global rubric set \(R\) covers four complementary dimensions: Situatedness \(R_{\text{sit}}\) (verifying spatial fidelity and visual presence), Item Relevance \(R_{\text{rel}}\) (verifying compliance with hard constraints and abstract need mappings), Multi-turn Consistency \(R_{\text{con}}\) (tracking user rejections and intent shifts), and Auxiliary criteria \(R_{\text{aux}}\) (verifying causal explanation and safety). Each rubric is formulated as a standardized binary predicate \(r(\cdot) \in \{0, 1\}\) equipped with explicit observable positive and negative evidence indicators.
To prevent noisy and irrelevant criteria from obscuring the reasoning focus during dynamic multi-turn interactions, a frozen lightweight MLLM acts as a dynamic rubric selector \(\pi_{\text{sel}}\). It analyzes the user's immediate communicative intent (e.g., distinguishing pure spatial queries from multi-attribute critiques) and selectively activates an active subset \(R_{\text{act}} \subset R\). The preference reasoning model \(\pi_\theta\) is trained via Group Relative Policy Optimization (GRPO) to generate a thinking trace \(T_g\) followed by a structured textual preference state \(P_g = (\text{Intent}, \text{Constraints}, \text{Visual Target})\). For each group of \(G\) sampled candidates, a frozen judge evaluates active rubrics to produce a scalar reward: $\(R_g = \frac{1}{|R_{\text{act}}|} \sum_{r \in R_{\text{act}}} f_{\text{judge}}(O_g, r, X_t)\)$ The policy is updated using the group-normalized relative advantage \(A_g = \frac{R_g - \mu_R}{\sigma_R + \epsilon}\), entirely bypassing the memory overhead of a critic network while instilling strict visual grounding into intermediate reasoning.
2. Cascaded Constrained Item Retrieval and Parsing: Operationalizing Structured States into Deterministic Physical Hard Constraints
Conventional conversational recommendation methods rely heavily on dense semantic similarity matching over neural item embeddings. However, neural retrieval consistently breaks down when dealing with spatial negations (e.g., "not the one on the left desk"), multi-attribute intersections, or crowded visual scenes with numerous visually plausible distractors. Re2A decouples item identification from response generation by establishing a deterministic preference-to-item parser.
Rather than introducing an auxiliary learned ranking model that might suffer from distribution shifts, the parser directly consumes the structured preference state \(P\) produced by \(\pi_\theta\). It systematically maps explicit fields (such as target category, excluded color/material, relative spatial offsets, and price bounds) onto the metadata and bounding boxes of all detected items visible in scene \(S_t\). By operationalizing preference reasoning into strict verification rules, this step filters out visually salient but constraint-violating distractors, ranks the candidate set, and delivers the top-ranked physical item \(\hat{o}\) as a verified grounding anchor. This transparent filtering reduces retrieval inference latency from 538.2 ms (in ReGeS) to 267.5 ms while eliminating false item attributions upstream of response generation.
3. Heterogeneous Preference-Conditioned Aligned Generation: Eliminating Hallucinations and Weak Matches via Multi-Conflict Negative DPO
Securing the correct physical item does not automatically guarantee a faithful and helpful conversation. Standard Supervised Fine-Tuning (SFT) optimized via maximum likelihood often leads to sycophantic hallucinations—inventing non-existent attributes to flatter the user—or dry, context-insensitive responses that ignore conversational nuances. Re2A resolves this alignment dilemma by formulating response generation as a preference-conditioned contrastive learning problem with targeted heterogeneous negative samples: - Situation-conflicting negatives (\(Y_{l1}\)): Synthesized by replacing the target item or spatial landmarks in the ground-truth response \(Y_w\) with objects that do not exist anywhere in the observed visual scene \(S\), explicitly penalizing visual and spatial hallucinations; - Preference-conflicting negatives (\(Y_{l2}\)): Synthesized by substituting the target item with a distractor that is physically present in the scene but directly violates explicit constraints in the preference state \(P\) (e.g., recommending a visible light wood shelf when the user specifically requested a dark finish), explicitly penalizing weak-match preference failures.
The response generation model \(\phi_\theta\) is trained on the combined dataset \(D_{\text{mix}}\) via Direct Preference Optimization (DPO), explicitly conditioned on the full grounded context \(\tilde{X} = (S, C, P, \hat{o})\): $\(\mathcal{L}_{\text{DPO}}(\theta) = -\mathbb{E}_{(\tilde{X}, Y_w, Y_l) \sim D_{\text{mix}}} \left[ \log \sigma \left( \beta_{\text{DPO}} \log \frac{\phi_\theta(Y_w \mid \tilde{X})}{\phi_{\text{ref}}(Y_w \mid \tilde{X})} - \beta_{\text{DPO}} \log \frac{\phi_\theta(Y_l \mid \tilde{X})}{\phi_{\text{ref}}(Y_l \mid \tilde{X})} \right) \right]\)$ where \(\phi_{\text{ref}}\) is initialized from teacher-forced SFT. By leveraging dual negative penalties, the model learns to firmly ground its descriptions in the physical anchor \(\hat{o}\) while naturally satisfying user preferences.
A Worked Example¶
Consider a challenging multi-turn interaction in a furniture shopping scene from the SCREEN dataset. The user states: "I also need extra storage space to fit the ambiance of my new decor style... I don’t want anything in that light wood finish. Is there a darker one near the desk?" The visible room includes a desk area on the left featuring a light-wood desk, an adjacent light-wood cabinet, and a black metal shelving unit.
- Dynamic Rubric Gating: The selector \(\pi_{\text{sel}}\) detects negation feedback ("don't want light wood"), spatial constraint ("near the desk"), and category need ("storage"), activating SIT-01 (visual presence), SIT-02 (spatial accuracy), REL-01 (constraint adherence), and CON-02 (state update).
- Preference Reasoning: Model \(\pi_\theta\) formulates an explicit thought trace noting the user's rejection of light wood and focus on the desk area, outputting the structured state:
Intent: Get Storage Unit,Constraints: ¬ Light Wood ∧ Near Desk,Visual Target: Black Metal Shelving. - Item Resolution: The deterministic parser compares visible candidates against \(P\). The visually salient light-wood cabinet is filtered out due to the
¬ Light Woodconstraint. The black metal shelving unit satisfies all criteria and is selected as anchor \(\hat{o}\). - Aligned Generation: Model \(\phi_\theta\) receives \((S, C, P, \hat{o})\) and generates: "I found a black metal shelving unit next to the desk. It matches the darker ambiance and avoids the light wood finish you rejected." The response is completely factual, contextually grounded, and respects all negative preferences.
Loss & Training¶
Re2A is trained in two distinct stages: 1. Preference Reasoning Optimization: Trained via GRPO with AdamW (\(lr = 2 \times 10^{-5}\), cosine annealing scheduler with 3% warmup). GRPO parameters are set to \(\beta_{\text{grpo}} = 0.04\), group size \(G = 8\), and sampling temperature \(\tau = 0.9\). Visual encoders and projection layers are frozen; Low-Rank Adaptation (LoRA) is applied exclusively to the Query and Value attention projection matrices (\(r = 64, \alpha = 128\), dropout 0.05). Training runs for 3 epochs with a global batch size of 16 across 8 NVIDIA A100 (80GB) GPUs in bfloat16. 2. Response Generation Alignment: First initialized with teacher-forced SFT on ground-truth targets \(o^*\) and gold responses \(Y_w\), followed by preference-conditioned DPO on \(D_{\text{mix}}\) with \(\beta_{\text{dpo}} = 0.1\). At inference time, \(\pi_\theta\) utilizes greedy decoding (temperature = 0) for deterministic state generation, whereas \(\phi_\theta\) uses nucleus sampling (top-\(p = 0.75\), top-\(k = 40\), temperature = 0.7) to ensure conversational fluidity.
Key Experimental Results¶
Main Results¶
Re2A is comprehensively evaluated on two benchmark datasets: SIMMC 2.1 (demanding 3D visual grounding and spatial coreference) and SCREEN (demanding implicit preference reasoning and complex multi-turn logic). Baselines span inference-only techniques, standard SFT, and adapted conversational recommendation methods (CRAG, ReGeS).
| Dataset | Backbone | Method | Hit@1 (%) | Recall@5 (%) | MRR@5 (%) | NDCG@5 (%) |
|---|---|---|---|---|---|---|
| SIMMC 2.1 | LLaVA-NeXT | Vanilla Prompting | 11.63 | 12.08 | 11.80 | 11.87 |
| SIMMC 2.1 | LLaVA-NeXT | ICL | 14.02 | 15.71 | 14.65 | 14.91 |
| SIMMC 2.1 | LLaVA-NeXT | CoT | 13.59 | 14.84 | 14.05 | 14.25 |
| SIMMC 2.1 | LLaVA-NeXT | SFT | 23.14 | 30.67 | 25.93 | 27.11 |
| SIMMC 2.1 | LLaVA-NeXT | CRAG | 27.91 | 38.04 | 31.86 | 33.43 |
| SIMMC 2.1 | LLaVA-NeXT | ReGeS | 32.18 | 56.89 | 39.09 | 43.34 |
| SIMMC 2.1 | LLaVA-NeXT | Re2A (Ours) | 40.35 | 60.37 | 48.83 | 51.74 |
| SIMMC 2.1 | Qwen3-VL | Vanilla Prompting | 15.37 | 17.91 | 16.52 | 16.88 |
| SIMMC 2.1 | Qwen3-VL | ICL | 18.25 | 22.06 | 20.16 | 20.65 |
| SIMMC 2.1 | Qwen3-VL | CoT | 17.55 | 19.53 | 18.42 | 18.70 |
| SIMMC 2.1 | Qwen3-VL | SFT | 31.24 | 39.14 | 33.20 | 34.61 |
| SIMMC 2.1 | Qwen3-VL | CRAG | 35.64 | 45.31 | 38.62 | 40.28 |
| SIMMC 2.1 | Qwen3-VL | ReGeS | 39.45 | 58.74 | 43.78 | 47.30 |
| SIMMC 2.1 | Qwen3-VL | Re2A (Ours) | 48.12 | 65.08 | 54.22 | 56.89 |
| SCREEN | Qwen3-VL | Vanilla Prompting | 19.62 | 21.03 | 20.14 | 20.36 |
| SCREEN | Qwen3-VL | ICL | 22.17 | 25.91 | 23.55 | 24.14 |
| SCREEN | Qwen3-VL | CoT | 21.41 | 25.76 | 23.02 | 23.70 |
| SCREEN | Qwen3-VL | SFT | 34.58 | 40.71 | 36.58 | 37.58 |
| SCREEN | Qwen3-VL | CRAG | 37.09 | 48.35 | 40.18 | 42.13 |
| SCREEN | Qwen3-VL | ReGeS | 41.72 | 64.19 | 50.03 | 53.56 |
| SCREEN | Qwen3-VL | Re2A (Ours) | 49.63 | 70.47 | 58.44 | 61.52 |
Response generation performance evaluated under cascaded execution with user simulation and GPT-5.2 evaluation (Qwen3-VL backbone):
| Dataset | Method | Judge-Avg (1-10) ↑ | Simulation SR (%) ↑ | VHR (%) ↓ | WMR (%) ↓ | Human-Avg (0-2) ↑ |
|---|---|---|---|---|---|---|
| SIMMC 2.1 | Vanilla | 5.29 | 15.1 | 45.7 | 32.8 | 0.94 |
| SIMMC 2.1 | ICL | 6.02 | 17.7 | 39.6 | 29.2 | 1.11 |
| SIMMC 2.1 | CoT | 6.32 | 16.4 | 38.4 | 25.9 | 1.20 |
| SIMMC 2.1 | SFT | 7.82 | 29.6 | 25.4 | 22.6 | 1.47 |
| SIMMC 2.1 | CRAG | 8.17 | 34.3 | 19.7 | 18.4 | 1.57 |
| SIMMC 2.1 | ReGeS | 8.49 | 38.6 | 15.2 | 15.7 | 1.65 |
| SIMMC 2.1 | Re2A | 9.31 | 46.8 | 5.2 | 7.4 | 1.87 |
| SCREEN | Vanilla | 5.49 | 16.9 | 42.1 | 34.2 | 0.97 |
| SCREEN | SFT | 7.94 | 31.7 | 24.1 | 23.6 | 1.51 |
| SCREEN | ReGeS | 8.56 | 40.9 | 14.6 | 16.2 | 1.70 |
| SCREEN | Re2A | 9.19 | 48.6 | 5.1 | 7.9 | 1.86 |
Ablation Study¶
Component and facet-level ablations on SIMMC 2.1 with the Qwen3-VL backbone:
| Module / Stage | Configuration | Hit@1 (%) ↑ | VHR (%) ↓ | WMR (%) ↓ | Degradation Note |
|---|---|---|---|---|---|
| Full Architecture | Re2A (Full Model) | 48.12 | 5.2 | 7.4 | Complete rubric-guided reasoning and heterogeneous DPO |
| Reasoning Ablation | Fine-tuned CoT (w/o Rubric) | 40.50 | 10.2 | 13.4 | Free-form reasoning drifts, dropping Hit@1 by 7.62% |
| Reasoning Ablation | End-to-End SFT (w/o Reasoning) | 31.24 | 25.4 | 22.6 | Severe reasoning deficit; Hit@1 collapses by 16.88% |
| Alignment Ablation | SFT Only (No DPO) | 48.12 | 25.4 | 22.6 | Retains recommendation state, but VHR surges to 25.4% |
| Alignment Ablation | Homogeneous DPO (w/o Hetero Negs) | 48.12 | 13.4 | 11.2 | Generic negatives fail to penalize subtle spatial hallucinations |
| Rubric Facet Ablation | w/o Situatedness (\(R_{\text{sit}}\)) | 44.15 | 8.9 | 9.5 | Loss of physical visual grounding raises VHR by 3.7% |
| Rubric Facet Ablation | w/o Item Relevance (\(R_{\text{rel}}\)) | 42.85 | 6.1 | 16.5 | Inability to track attribute constraints surges WMR by 9.1% |
| Rubric Facet Ablation | w/o Consistency (\(R_{\text{con}}\)) | 45.32 | 5.8 | 11.2 | Impairs multi-turn intent tracking and rejection memory |
Key Findings¶
- Reasoning Deficit and the Failure of Unconstrained CoT: Prompted CoT consistently underperforms ICL (e.g., 17.55% vs. 18.25% on SIMMC 2.1), frequently inventing non-existent visual details during intermediate text generation. Bypassing reasoning altogether (End-to-End SFT) drops Hit@1 by 16.88 percentage points, proving that explicit, structured cognitive states are essential for grounding multimodal recommendations.
- Heterogeneous Negatives Break the Alignment Dilemma: Relying solely on standard SFT or generic homogeneous DPO leaves VHR high (25.4% and 13.4%). Constructing explicit situation-conflicting (\(Y_{l1}\)) and preference-conflicting (\(Y_{l2}\)) pairs reduces VHR to 5.2% and WMR to 7.4%, proving that targeted contrastive supervision is vital for multi-constraint alignment.
- Exceptional Sample Efficiency: Re2A trained on merely 25% of the training data attains 43.50% Hit@1, surpassing the fully-trained (100% data) SFT baseline (31.24%) by 12.26 percentage points. The structured rubric prior delivers dense learning signals that drastically accelerate policy convergence.
- Latency and Distillation Trade-Off: Explicit preference reasoning increases total per-turn latency to 2342.7 ms (vs. 1220.9 ms for ReGeS), yet it actually reduces retrieval latency from 538.2 ms to 267.5 ms through cleaner state filtering. Furthermore, distilling the 8B reasoner into a 4B student model reduces total per-turn latency by 504.3 ms to 1838.4 ms while preserving 46.71% Hit@1, establishing an effective route for practical deployment.
Highlights & Insights¶
- Transforming Unsupervised Reasoning into Verifiable Predicates: Unlike mathematical tasks where ground truth answers provide automatic reward signals, situated preference reasoning lacks verifiable intermediate labels. Re2A converts qualitative reasoning into a multi-dimensional binary verification game via induced rubrics, enabling stable reinforcement learning via GRPO without human trace annotation.
- Decoupling Physical Grounding from Preference Alignment: The identification of the dual alignment dilemma represents a key conceptual insight. By explicitly constructing situation-conflicting and preference-conflicting counterfactual responses, Re2A forces the generative policy to balance factual precision with customer satisfaction.
- Transparent White-Box Interface: Intermediate preference states (\(P\)) make the agent's internal thought process fully inspectable by users and developers, while acting as a deterministic filter that prevents visual saliency biases from corrupting the item retrieval stage.
Limitations & Future Work¶
- Cascaded Error Propagation: The sequential reason-then-align design is vulnerable to upstream error cascading: an erroneous preference state directly misguides candidate item resolution and response synthesis. Integrating an autonomous verifier to trigger clarification questions upon detecting state ambiguity is an important next step.
- Reliance on Closed-Source Teacher Priors: Inducing the 18 global rubrics and synthesizing negative training pairs relies on offline access to a frontier teacher model (GPT-5.2), potentially transferring the teacher's latent biases or blind spots into the induced rubrics.
- Static Scene Boundary: Benchmarks currently focus on static 3D and 2D scene snapshots. Extending the framework to dynamically evolving environments—where users interact with objects or navigate across changing rooms—presents a critical frontier for embodied assistants.
Related Work & Insights¶
- vs Conventional CRS (CRAG, ReGeS): Traditional conversational recommender systems operate in text-only semantic vector spaces. They lack awareness of physical surroundings, often failing when handling spatial references or visual negations. Re2A extends conversational recommendation into fully embodied, situated multimodal settings.
- vs Multimodal CoT Reasoning (MMIR): Standard multimodal CoT in open-domain tasks tends to suffer from semantic drift and ungrounded hallucinations. Re2A demonstrates that dynamic rubric gating combined with relative advantage policy optimization successfully constrains reasoning trajectories to verifiable visual evidence.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ [Pioneers the formalization of the dual alignment dilemma in situated recommendation, introducing rubric-guided GRPO and heterogeneous DPO.]
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive evaluations across two major benchmarks, two multimodal backbones, multi-facet ablations, human evaluations, latency profiling, and cross-domain transfer.]
- Writing Quality: ⭐⭐⭐⭐⭐ [Impeccable structural clarity, rigorous mathematical formulations, and seamless narrative progression from motivation to empirical validation.]
- Value: ⭐⭐⭐⭐⭐ [Offers an actionable, highly robust architectural blueprint for embodied shopping assistants, mixed reality (MR) agents, and verifiable multimodal dialogue systems.]