Enhancing Goal-oriented Proactive Dialogue Systems via Consistency Reflection and Correction¶
Conference: ACL 2025
arXiv: 2506.13366
Code: None
Area: Dialogue Systems
Keywords: Goal-oriented Dialogue, Proactive Dialogue Systems, Consistency Reflection, Consistency Correction, Model-Agnostic Framework
TL;DR¶
A model-agnostic two-stage CRC framework (Consistency Reflection & Correction) is proposed. By first prompting the model to reflect on inconsistencies between the generated response and the dialogue context and then correcting the response accordingly, it significantly improves the consistency of generated responses with the dialogue context in goal-oriented proactive dialogue systems.
Background & Motivation¶
Background: Goal-oriented proactive dialogue systems aim to guide user conversations seamlessly toward specific targets, typically requiring the planning of a path (dialogue path) from the current topic to the target topic, and then step-by-step generating responses along this path. Such systems are widely used in domains like recommendation, customer service, and psychological counseling. Existing approach mainly focus on optimizing dialogue path planning strategies—such as selecting intermediate sub-goals and making smooth transitions between topics.
Limitations of Prior Work: Prior work has heavily focused on optimizing path planning but overlooked a key issue: potential inconsistencies between generated responses and the dialogue context. These inconsistencies manifest at multiple levels—contradictions with user profiles (e.g., incorrect address/honorific), contradictions with dialogue history (e.g., repeating answered questions), contradictions with domain knowledge (e.g., providing incorrect information), or contradictions with temporary sub-goals (e.g., straying from the topic direction). These inconsistencies seriously undermine user experience and dialogue effectiveness.
Key Challenge: Dialogue path planning and response generation are coupled but asynchronous processes. Path planning focuses on "whether the macro-direction is correct," but lacks mechanisms to ensure "whether the micro-expressions are consistent." When generating specific responses, models may only focus on linguistic fluency while neglecting consistency constraints with multidimensional context.
Goal: To design a universal framework capable of detecting and correcting various types of inconsistencies in generated responses without relying on specific model architectures—making it applicable to both encoder-decoder (BART, T5) and decoder-only (GPT-2, LLaMA3, etc.) models.
Key Insight: Inspired by the human cognitive process in dialogue of "thinking before speaking, reflecting after speaking, and correcting mistakes," the authors divide consistency refinement into two explicit stages: first reflection (identifying issues) and then correction (resolving issues).
Core Idea: To prompt the model to first conduct multidimensional consistency reflection on the generated response (identifying where it is inconsistent with user profiles, dialogue history, domain knowledge, or sub-goals), and then regenerate a more consistent response based on the reflection results.
Method¶
Overall Architecture¶
The CRC framework is a two-stage post-processing workflow that can be inserted after any dialogue generation model: (1) Consistency Reflection Stage: Given the initially generated response and the complete dialogue context (user profile, dialogue history, domain knowledge, and current sub-goal), the model is prompted to analyze the discrepancies between the response and the context, outputting specific inconsistencies and suggested correction directions. (2) Consistency Correction Stage: Taking the reflection results as additional input, the model is guided to generate a new response that is more consistent with the dialogue context. The input consists of the original dialogue context + initial response + reflection analysis, and the output is the corrected response.
Key Designs¶
-
多维一致性反思（Consistency Reflection）:
- Function: Systematically identify specific issues in the initial response that are inconsistent with the dialogue context.
- Mechanism: Build structured reflection prompts requiring the model to check consistency across four dimensions—(a) User profile consistency: whether the response aligns with known user preferences and background information; (b) Dialogue history consistency: whether there are repetitions, contradictions, or forgotten discussions; (c) Domain knowledge consistency: whether the provided information aligns with facts in the knowledge base; (d) Sub-goal consistency: whether the response facilitates progression towards the current sub-goal. The model needs to provide specific inconsistency descriptions and modification suggestions for each dimension.
- Design Motivation: Decomposing the general "response quality" problem into four checkable dimensions reduces the difficulty of reflection and makes correction more targeted. This structured reflection is more effective than simple "please improve this response" prompts.
-
一致性纠正（Consistency Correction）:
- Function: Generate responses that are more consistent with the dialogue context based on the reflection results.
- Mechanism: Concatenate the inconsistency analysis and modification suggestions from the reflection stage into the original input to form enhanced generation conditions. When generating the corrected response, the model refers to both the original context and the reflection results, ensuring inconsistencies are fixed while maintaining dialogue fluency. The correction is not simple post-editing; instead, it is a complete regeneration based on the original context, with reflection results acting as an extra "attention guide."
- Design Motivation: Once the reflection stage makes the problem "explicit," the correction stage can focus more precisely on the parts needing repair, avoiding blind rewriting.
-
模型无关架构设计:
- Function: Ensure the framework is adaptable to language models of different scales and architectures.
- Mechanism: CRC achieves reflection and correction solely through prompt construction, without modifying internal model structures or training objectives. For encoder-decoder models (BART, T5), the context + reflection results serve as the encoder input; for decoder-only models (GPT-2, DialoGPT, Phi3, Mistral, LLaMA3), everything is concatenated into a single input sequence. The framework can be combined plug-and-play with various backbones.
- Design Motivation: Goal-oriented dialogue systems may deploy various models in practice; model-agnosticism ensures the wide applicability of the framework.
Loss & Training¶
Training incorporates standard cross-entropy objectives for sequence-to-sequence supervised learning on generated responses. Serving as a training framework, CRC conducts forward and backward passes separately in the reflection and correction stages. For fine-tuning, the standard training sets of each task dataset are utilized.
Key Experimental Results¶
Main Results¶
Comparison of consistency metrics on 3 goal-oriented dialogue datasets:
| Model | Method | User Profile Consistency | History Consistency | Knowledge Consistency | Sub-goal Consistency | Overall Quality |
|---|---|---|---|---|---|---|
| BART | Baseline | Baseline | Baseline | Baseline | Baseline | Baseline |
| BART | +CRC | +Significant ↑ | +Significant ↑ | +Significant ↑ | +Significant ↑ | +Significant ↑ |
| T5 | Baseline | Baseline | Baseline | Baseline | Baseline | Baseline |
| T5 | +CRC | +Significant ↑ | +Significant ↑ | +Significant ↑ | +Significant ↑ | +Significant ↑ |
| LLaMA3 | Baseline | Baseline | Baseline | Baseline | Baseline | Baseline |
| LLaMA3 | +CRC | +Significant ↑ | +Significant ↑ | +Significant ↑ | +Significant ↑ | +Significant ↑ |
Cross-model architecture comparison (improvements observed on all 7 models across 3 datasets):
| Model Type | Model | Baseline → CRC Avg. Gain |
|---|---|---|
| Encoder-Decoder | BART | Significant Gain |
| Encoder-Decoder | T5 | Significant Gain |
| Decoder-Only | GPT-2 | Significant Gain |
| Decoder-Only | DialoGPT | Significant Gain |
| Decoder-Only | Phi3 | Significant Gain |
| Decoder-Only | Mistral | Significant Gain |
| Decoder-Only | LLaMA3 | Significant Gain |
Ablation Study¶
| Configuration | Dialogue Consistency | Description |
|---|---|---|
| Full CRC (Reflection + Correction) | Optimal | Two-stage synergy is most effective |
| Correction Only (w/o Reflection) | Moderate Gain | Lacks reflection guidance, blind correction |
| Reflection Only (w/o Correction) | Slight Gain | Only identifies issues without fixing them |
| No CRC (Baseline) | Baseline | Control group |
| Remove User Profile Reflection | Significant Drop | User profile consistency is most easily ignored |
| Remove Sub-goal Reflection | Significant Drop | Sub-goal consistency affects dialogue progression |
Key Findings¶
- Both stages are indispensable: Correction alone (without reflection guidance) yields significantly worse improvements than reflection followed by correction, indicating that "making problems explicit" is crucial for correction quality.
- Consistently effective across architectures: From the small-scale GPT-2 to the large-scale LLaMA3, and from encoder-decoder to decoder-only, CRC consistently brings significant improvements in consistency, validating its model-agnostic nature.
- Consistency in all four dimensions is important: Ablating any reflection dimension leads to a drop in overall quality, with user profile and sub-goal dimensions having the largest impact.
- Large models benefit more: Models with larger parameter scales experience larger performance gains from CRC, likely due to their superior capability for reflection and self-correction.
Highlights & Insights¶
- Systematizing consistency issues into four checkable dimensions: Unlike prior implicit evaluations of dialogue quality, CRC decomposes consistency into four orthogonal dimensions: user profile, history, knowledge, and sub-goals, with each dimension checked and repaired independently. This structured mindset can be transferred to any generation task requiring multidimensional quality control.
- Reflection-correction two-stage paradigm: Explicitly identifying problems before targets-oriented repairing is more effective than one-step prompts like "generate a better response." This approach aligns with self-refining paradigms but is more precise by focusing specifically on consistency.
- Model-agnostic, plug-and-play design: Achieved solely via prompt engineering without modifying model architectures or training pipelines, offering extremely low deployment costs.
Limitations & Future Work¶
- Double-stage inference increases latency: Reflection and correction require two complete model forward passes, doubling the inference time, which could be a bottleneck in real-time dialogue scenarios.
- Reflection quality depends on model capabilities: Small models exhibit limited reflection abilities and may fail to accurately identify all inconsistency issues, leading to inadequate subsequent corrections.
- Evaluation mainly relies on automatic metrics: Consistency evaluation is subjective, and whether the automatic metrics used in the paper fully reflect human-perceived consistency remains to be verified.
- Iterative reflection unexplored: The current framework only executes a single round of reflection-correction. While multi-round iteration might further improve quality, it could also introduce new issues.
Related Work & Insights¶
- vs. Self-Refine: Self-Refine is a general self-improvement framework, whereas CRC is designed specifically for consistency issues in goal-oriented dialogues, employing a structured four-dimensional reflection mechanism that is more targeted.
- vs. Topic-Grounded Dialogue: Traditional topic-grounded dialogue approaches focus on path planning, whereas CRC focuses on the consistency of response generation after the path is determined, making the two complementary.
- vs. Chain-of-Thought Prompting: The reflection stage of CRC is similar to the "think before acting" concept of CoT, but has clearer dimensions—focusing on multi-angle scrutiny of generated content rather than step-by-step reasoning during generation.
Rating¶
- Novelty: ⭐⭐⭐ The idea of two-stage reflection-correction is not entirely new (similar to Self-Refine), but systematizing four-dimensional consistency checks in goal-oriented dialogue is a meaningful contribution.
- Experimental Thoroughness: ⭐⭐⭐⭐ Covered extensively with 7 model architectures and 3 datasets along with a complete ablation study; the experimental design is highly comprehensive and rational.
- Writing Quality: ⭐⭐⭐⭐ Clearly defined problems, intuitive and easy-to-understand framework description, and tight logical link between motivation and method.
- Value: ⭐⭐⭐⭐ The model-agnostic plug-and-play feature offers high practical value, and the four-dimensional consistency framework is transferable to other dialogue tasks.