Learning Diagnostic Clarifying Questions for Conversational Question Answering via Data-Efficient Dynamic Distillation¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: Information Retrieval & RAG
Keywords: conversational QA, clarifying question generation, dynamic distillation, ambiguity resolution, hindsight relabeling
TL;DR¶
To tackle the lack of progressive disambiguation protocols and the prevalence of non-diagnostic questions in open-domain conversational QA, D4CQ introduces procedural scaffolding with contrastive intent-convergence alignment, diagnosticity-based dynamic distillation, and hindsight relabeling with compatibility gating, achieving superior ambiguity resolution and answer accuracy under limited interaction budgets.
Background & Motivation¶
Open-domain conversational question answering (OD-CQA) aims to resolve missing information and underspecified queries via interactive dialogue, enabling accurate retrieval and answering in vast knowledge spaces. In real-world scenarios, initial user queries are often heavily underspecified or polysemous, harboring multiple plausible latent intents that each lead to completely different valid answers. Existing disambiguation paradigms generally fall into two categories: prompt-based ambiguity-aware methods that decide whether to clarify or answer at inference time but suffer from high context sensitivity and poor generalization across ambiguity types, and trajectory distillation methods that directly imitate teacher-generated clarify-or-answer rollouts without explicitly modeling progressive intent convergence.
Two major limitations severely hinder these approaches. First, supervision is defined purely on token-level imitation over clarification candidates, without teaching the model how to systematically reason over unresolved ambiguity axes before posing a question. Consequently, the model may learn superficial candidate preferences but fails to progressively narrow the intent hypothesis space across dialogue turns. Second, standard trajectory filtering treats all rollouts that happen to reach a correct downstream answer as equally positive supervision, ignoring the diagnostic quality of the clarifying questions. A model can be reinforced for asking generic, verbose, or confirmatory questions whose answers fit almost any candidate intent, encouraging uninformative questions during test time. Furthermore, strict trajectory filtering discards numerous valid rollouts that happen to land on alternative plausible intents, leading to severe data inefficiency.
The critical challenge is to empower the model to not only recognize ambiguity but also formulate diagnostic questions that provide maximal information gain. The core idea is to establish a structured disambiguation protocol with representation-level intent-convergence alignment, dynamically reweight distillation based on intent-discriminative response likelihood, and recycle failed rollouts via hindsight relabeling with compatibility gating.
Method¶
Overall Architecture¶
D4CQ focuses on training an effective interrogator model for conversational question answering. The overall pipeline operates across multi-turn clarification interactions and an offline dynamic distillation stage: given an ambiguous user query and dialogue history, the interrogator reasons over ambiguity axes to choose between generating a clarifying question or returning a final direct answer; a simulated user (a fixed responsor LM) returns a concise answer conditioned on the ground-truth intent, continuing the dialogue until a terminal answer is reached or the turn budget \(K_{\max}\) is exhausted.
During offline distillation, D4CQ unifies four mutually reinforcing components: Procedural Scaffolding guides the model through a five-stage cognitive protocol to identify unresolved ambiguity axes; Contrastive Intent-Convergence Alignment optimizes intermediate rewritten queries in representation space to pull them closer to the true latent intent; Diagnosticity-based Dynamic Distillation calculates the likelihood margin of user responses under true versus competing intents to scale sample-level distillation loss; and Hindsight Relabeling with Compatibility Gating safely recovers otherwise discarded rollouts that match alternative legitimate intents.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Ambiguous Query q & Dialogue History"] --> B["Procedural Scaffolding<br/>Rewrite Query + Identify Ambiguity Axes"]
B --> C{"Action Decision<br/>Clarifying Question or Direct Answer"}
C -->|Direct Answer| D["Terminal Answer & Accuracy Evaluation"]
C -->|Clarification| E["Generate Question ck & Obtain Response rk"]
E --> F["Diagnosticity Scoring & Dynamic Distillation<br/>Compute Margin-based Weight w(ฯ)"]
F --> G["Contrastive Intent-Convergence Alignment<br/>Representation-level InfoNCE Loss"]
D -->|Fails Gold Intent| H{"Hindsight Relabeling & Compatibility Gating<br/>Match Alternative Intent + Likelihood Gap"}
H -->|Passes Gate| I["Recycled into Supervision Set Dr"]
H -->|Fails Gate| J["Discard Invalid Trajectory"]
G --> K["Multi-Task End-to-End Optimization"]
I --> K
Key Designs¶
1. Procedural Scaffolding with Contrastive Intent-Convergence Alignment: structured multi-stage disambiguation Naรฏve supervised fine-tuning lacks an explicit multi-turn protocol and easily loses track of initial constraints due to positional sensitivity in long conversational contexts. D4CQ establishes a five-stage chain-of-thought scaffolding protocol: Stage I recaps verified constraints and explicitly rewrites the query into an updated self-contained form; Stage II recalls relevant parametric background knowledge; Stage III conducts a singleton check to distinguish single-target questions from inherently multi-target ones; Stage IV discovers ambiguity axes by term substitution (e.g., teams, institutions, temporal anchors); and Stage V identifies hidden axes via qualifier addition and deletion. To explicitly enforce semantic convergence in model representations, the hidden state at the last token of the rewritten query \(u = z_\theta(\tau, \bar{q})\) is paired with candidate intent representations \(\{v_j\}\) under an InfoNCE contrastive objective:
This loss directly pulls the interrogator's internal representation toward the ground-truth intent \(x_i\) while repelling within-query alternative intents \(x_j\), ensuring steady progressive disambiguation across dialogue turns.
2. Diagnosticity-based Dynamic Distillation: penalizing uninformative confirmations Not all successful teacher trajectories contain high-quality interactions; many clarifying questions are confirmatory platitudes whose responses remain consistent with multiple candidate intents, failing to reduce intent uncertainty. D4CQ formulates an information-theoretic diagnosticity metric: for the \(k\)-th clarifying question \(\bar{c}_k\) and response \(r_k\), the fixed responsor model computes length-normalized log-likelihoods \(\ell_j^{(k)}\) across all candidate intents \(x_j\). The turn-level diagnosticity is defined via a log-mean-exp margin:
A large margin indicates that the response provides highly specific evidence elicited by \(\bar{c}_k\) for the true intent rather than generic information. Turn-level scores are averaged across the trajectory and converted into a non-negative sample weight \(w(\tau) = \text{softplus}(\beta \bar{d}(\tau))\), scaling the token-level cross-entropy loss \(\mathcal{L}_{\text{DD}}(\tau)\) so that highly diagnostic interactions dominate the distillation gradients.
3. Hindsight Relabeling with Compatibility Gating: safely recovering discarded trajectories Strict terminal correctness filtering discards a substantial fraction of teacher rollouts that are internally coherent and locally sound but happen to land on an alternative plausible intent of the ambiguous query. D4CQ introduces hindsight relabeling: if a failed trajectory \(\tau\) matches an alternative intent \(x_{j^*}\) with high terminal correctness \(\psi(\hat{a}, A(x_{j^*})) \ge \eta\), it becomes a candidate for recycling. To avoid introducing semantic contradictions where prior user responses conflict with the new intent, a cumulative likelihood compatibility gate is enforced:
Only trajectories whose observed turns remain sufficiently probable under \(x_{j^*}\) are accepted into \(\mathcal{D}_r\). Furthermore, supervision on relabeled trajectories is applied strictly to reasoning steps and clarifying question tokens while detaching the final answer span, allowing the student to inherit effective questioning strategies without overfitting to potentially noisy terminal answers.
Loss & Training¶
The overall training objective combines originally retained trajectories \(\mathcal{D}_o\) and accepted hindsight-relabeled trajectories \(\mathcal{D}_r\):
Teacher rollouts are generated by Llama3-70B-Instruct prompted with few-shot procedural scaffolding exemplars. Student models are fine-tuned using 4-bit QLoRA via Unsloth on 4 A100-80GB GPUs with AdamW, learning rate \(2 \times 10^{-4}\), batch size 64, and 4 training epochs. The default hyperparameters are \(\eta = 0.8\), \(\lambda = 1.0\), \(\beta = 1.5\), and \(\epsilon = 1.0\). Shared prefix KV-caches are precomputed and reused across intent continuations during contrastive computation, substantially saving compute overhead.
Key Experimental Results¶
Main Results¶
On the AmbigQA benchmark, D4CQ is evaluated against Direct Answering, prompt-based Ambig-Aware, alignment-based APAGEN, and preference-optimized CDADPO. Evaluated metrics include unambiguous subset F1 (\(F1_u\)), ambiguous subset F1 (\(F1_a\)), full set F1 (\(F1_{all}\)), and the average number of clarification turns (\(\#\)).
| Backbone & Method | Turns # | \(F1_u\) (%) | \(F1_a\) (%) | \(F1_{all}\) (%) | Gain vs. CDADPO |
|---|---|---|---|---|---|
| Llama3-8B | |||||
| Direct Answer | 0.00 | 25.0 | 16.8 | 21.1 | โ |
| Ambig-Aware | 0.12 | 27.4 | 20.9 | 23.0 | โ |
| APAGEN | 0.37 | 31.6 | 24.1 | 27.9 | โ |
| CDADPO | 1.00 | 42.9 | 31.7 | 36.5 | Baseline |
| D4CQ (Ours) | 1.08 | 47.2 | 34.3 | 40.1 | \(F1_a\) +8.2% / \(F1_{all}\) +9.9% |
| Gemma-7B | |||||
| Direct Answer | 0.00 | 31.2 | 19.2 | 24.8 | โ |
| Ambig-Aware | 0.17 | 34.8 | 23.6 | 28.7 | โ |
| APAGEN | 0.39 | 29.8 | 23.9 | 25.3 | โ |
| CDADPO | 1.00 | 40.7 | 28.6 | 33.9 | Baseline |
| D4CQ (Ours) | 1.06 | 42.6 | 31.1 | 36.0 | \(F1_a\) +8.7% / \(F1_{all}\) +6.2% |
Ablation Study¶
A component ablation on Llama3-8B over the ambiguous subset of AmbigQA isolates the contributions of the three core ideas, using the inference-only Procedural-Scaffolding baseline (27.8%) as the reference:
| Configuration | \(F1_a\) (%) | Relative Gain \(\Delta F1_a\) | Note |
|---|---|---|---|
| Procedural-Scaffolding (Inference only) | 27.8 | โ | No parameter updates |
| w/o Module I3 (PROCICA + DYNDIST) | 32.5 | +16.9% | Discards failed rollouts, reducing sample efficiency |
| w/o Module I2 (PROCICA + HINDREL) | 33.3 | +19.8% | Omits diagnosticity weighting, keeping generic queries |
| w/o Module I1 (DYNDIST + HINDREL) | 33.9 | +21.9% | Removes representation intent alignment |
| D4CQ (Full Model) | 34.3 | +23.4% | Synergistic combination achieving best performance |
In zero-shot out-of-domain evaluation on SituatedQA using Mistral-7B, D4CQ achieves an ambiguity detection score \(F1_{detect}\) of 44.75% and direct answer score \(F1_{unp}\) of 41.01% on SituatedQA-Geo (surpassing APAFIXED's 41.84% and 38.43%), and improves \(F1_{detect}\) to 48.12% on SituatedQA-Temp, proving strong out-of-distribution transferability.
Key Findings¶
- Hindsight relabeling contributes the largest individual gain: In pairwise ablation setups, retaining hindsight relabeling (HINDREL) yields the highest scores, highlighting that strict trajectory filtering severely starved models of challenging and diverse ambiguity patterns.
- Performance boosts stem from quality rather than frequency: D4CQ uses approximately 1.08 clarification turns on average, nearly identical to CDADPO (1.00), yet achieves significant gains across ambiguous and unambiguous subsets alike, confirming improved clarification selectivity rather than indiscriminate questioning.
- Intent representations monotonically converge to gold targets: Representation tracking reveals that D4CQ reaches a 60.8% monotonic convergence rate (MonoRate) and a Hit@1 of 0.612, demonstrating that procedural rewriting and contrastive loss steadily steer the dialogue toward the latent intent.
Highlights & Insights¶
- Likelihood margin as a principled diagnostic proxy: Formulating diagnosticity via log-mean-exp margins across responsor likelihoods effectively captures intent discrimination without relying on expensive reward modeling or human annotations.
- Hindsight experience replay adapted for NLP distillation: Extending hindsight experience replay to conversational distillationโpaired with strict likelihood compatibility gating and terminal answer exclusionโoffers a highly reliable data efficiency blueprint.
- Zero-shot out-of-domain decision calibration: Without exposure to temporal or geographical context markers during training, the model generalizes robustly to SituatedQA, demonstrating that procedural scaffolding and contrastive intent alignment teach generalized clarify-or-answer boundary decision policies.
Limitations & Future Work¶
- Short-form answer assumption: The formulation is primarily developed for open-domain QA with short entity answers and compact clarification budgets (\(K_{\max} \le 2\)), leaving long-form answer generation and multi-step complex workflows unexplored.
- Absence of extra-linguistic contexts: Ambiguity is assumed to stem solely from underspecified text queries, without incorporating explicit user profiles, multimodal cues, or persistent environmental states.
- Simulated user gap: The responsor is implemented using a fixed LLM rather than actual human users; human conversational nuances such as evasive replies or typo-laden clarification responses remain to be evaluated in live settings.
Related Work & Insights¶
- vs. APAGEN / APAFIXED: APA models train an explicit clarify-or-answer decision policy via supervised alignment, but lack structured multi-step reasoning protocols and representation-level intent convergence signals. D4CQ decomposes multi-axis ambiguity via procedural scaffolding and drives intent convergence with InfoNCE loss.
- vs. CDADPO: CDADPO applies DPO on rollout trajectories based on final answer accuracy, implicitly treating all successful questions as equally good regardless of actual intent diagnosticity. D4CQ dynamically scales distillation loss based on turn-level diagnosticity and recycles valid alternatives through hindsight relabeling.
Rating¶
- Novelty: โญโญโญโญโ Grounding question diagnosticity in responsor likelihood margins and applying gated hindsight relabeling is novel and effective.
- Experimental Thoroughness: โญโญโญโญโญ Comprehensive evaluations across multiple backbones, ablation variants, hyperparameter sweeps, intent representation tracking, and out-of-domain benchmarks.
- Writing Quality: โญโญโญโญโญ The paper is structured cleanly with precise mathematical formulations and insightful discussions on conversational ambiguity.
- Value: โญโญโญโญโ Highly relevant for conversational search, customer support agents, and interactive clarification systems.