Skip to content

Learning Diagnostic Clarifying Questions for Conversational Question Answering via Data-Efficient Dynamic Distillation

Conference: EMNLP 2026
Paper: ACL Anthology
Area: Dialogue Systems
Keywords: Conversational QA, Clarifying Questions, Dynamic Distillation, Intent Convergence, Disambiguation

TL;DR

Addressing the dual challenges of non-convergent clarification and strict rollout filtering in open-domain conversational QA, D4CQ introduces procedural scaffolding, contrastive intent alignment, response-diagnosticity loss reweighting, and compatibility-gated hindsight relabeling, substantially outperforming existing clarify-or-answer baselines under a comparable interaction budget.

Background & Motivation

In open-domain conversational question answering (OD-CQA), initial user queries are often underspecified and inherently ambiguous, where a single surface form may correspond to multiple valid interpretations depending on time, location, or implicit context. Rather than committing to a premature one-shot guess, a practical conversational agent must interactively elicit missing information by asking clarifying questions when necessary and answering directly once the user's intent is disambiguated. Prior methods primarily explore two directions: prompt-based ambiguity-aware methods that decide whether to clarify or answer at inference time via prompting, which often exhibit brittle generalization across diverse ambiguity types; and trajectory-distillation methods that fine-tune student interrogators on teacher-generated conversational traces.

However, existing trajectory-distillation pipelines suffer from three fundamental limitations. First, supervision is defined solely over candidate clarifying questions at the token level without providing an explicit protocol for multi-turn disambiguation or a representational signal driving convergence toward the latent intent. Second, distillation ignores the diagnostic quality of clarifying questions: a question in an accepted teacher trajectory might be verbose, trivial, or confirmatory, yielding a response that fits many competing intents rather than sharply isolating the gold intent. Distilling such low-diagnostic questions encourages unnecessary turns and hurts final precision. Third, strict trajectory filtering discards every rollout that fails to reach the original gold answer, discarding structurally sound interactions that simply resolved to an alternative valid interpretation and exacerbating data inefficiency.

This paper tackles these bottlenecks by turning multi-turn disambiguation into a structured reasoning process and weighting distillation by informational diagnosticity. The core idea is to combine a procedural refinement scaffold with contrastive intent-convergence alignment, reweight distillation loss using the response likelihood margin across competing intents, and safely recycle discarded rollouts via compatibility-gated hindsight relabeling.

Method

Overall Architecture

The D4CQ (Data-Efficient Diagnosticity-based Dynamic Distillation for Clarifying Question) framework optimizes an interrogator language model \(p_\theta\) interacting with a frozen responsor \(p_\phi\) that simulates user replies. Given an ambiguous question \(q\) and a maximum turn budget \(K_{\max}\), the interrogator generates clarifying questions \(c_k\) for \(K(\tau)\) turns and eventually produces a final answer \(\hat{a}\).

During distillation data construction and training, D4CQ integrates four interconnected components: (1) Procedural Scaffolding: prompting a large teacher model with structured chain-of-thought steps (recap/refinement, background recall, singleton check, and replacement/addition/deletion ambiguity discovery) to generate systematic multi-turn traces; (2) Contrastive Intent-Convergence Alignment: pulling the latent representation of the refined query toward the ground-truth intent and repelling within-query alternative intents; (3) Diagnosticity-based Dynamic Distillation: scoring clarifying questions by the likelihood margin of the elicited responses under competing intents to dynamically reweight sequence cross-entropy loss; (4) Hindsight Relabeling with Compatibility Gating: recycling failed rollouts whose terminal prediction matches an alternative plausible intent, verified through a multi-turn response compatibility gate.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["User Initial Query q"] --> B["Procedural Scaffolding: Structured Disambiguation<br/>Recap & refinement / Background recall / Singleton check / Axis discovery"]
    B --> C["Contrastive Intent-Convergence Alignment: Latent Pulling<br/>InfoNCE aligns refined query with gold intent vs alternative intents"]
    C --> D["Diagnosticity-based Dynamic Distillation: Informative Reweighting<br/>Score question-response pairs via log-mean-exp margin over responsor"]
    D --> E["Hindsight Relabeling with Compatibility Gating: Recycling Rollouts<br/>Map failed terminal answers to alternative intents via likelihood gate"]
    E --> F["Produce High-Precision Clarifying Questions or Direct Answers"]

Key Designs

1. Procedural Scaffolding: Structured Disambiguation

Standard autoregressive language models struggle with positional sensitivity over extended dialogues, frequently losing track of early constraints or failing to systematically test ambiguity axes. Procedural scaffolding addresses this by enforcing a five-stage reasoning protocol within the teacher's <thinking> tags: Stage I recaps confirmed constraints from prior turns and produces an updated refined query \(\bar{q}\), recentering the context hypothesis; Stage II recalls parametric background knowledge (entities, timelines, relationships); Stage III performs a singleton check to distinguish inherently multi-answer queries from single-target intents; Stages IV and V systematically discover ambiguity axes via controlled perturbationsโ€”term replacement (e.g., entity, league, level) and modifier addition/deletion (e.g., timeframe, location, gender). Under a conservative policy, any identified unresolved axis triggers a targeted clarifying question, whereas an unambiguous state immediately triggers direct answer generation.

2. Contrastive Intent-Convergence Alignment: Latent Pulling

Token-level imitation learning alone fails to provide an explicit geometric signal driving the dialogue state toward the latent intent. To bridge this gap, D4CQ introduces a representation-level contrastive objective on the final refined query. For a retained trajectory \(\tau\) with ground-truth intent \(x_i\), dialogue history \(h_{K(\tau)+1}\), teacher-forced reasoning prefix \(\pi(\tau)\), and refined query \(\bar{q}\), let \(u = z_\theta(\tau, \bar{q})\) denote the conditional embedding from the final hidden state. Given the candidate intent set \(\{x_j\}_{j=1}^{m_q}\) associated with \(q\) and their embeddings \(v_j = z_\theta(\tau, x_j)\), an InfoNCE-style intent-convergence loss is minimized:

\[L_{\mathrm{IC}}(\tau) = -\log \frac{\exp(\mathrm{sim}(u, v_i)/\gamma)}{\sum_{j=1}^{m_q} \exp(\mathrm{sim}(u, v_j)/\gamma)}\]

where \(\mathrm{sim}(\cdot,\cdot)\) represents cosine similarity and \(\gamma\) is a temperature hyperparameter. This objective pulls the refined question representation directly toward the true latent intent while pushing away competing interpretations for the same query.

3. Diagnosticity-based Dynamic Distillation: Informative Reweighting

Not all successful rollouts are equally informative; questions that receive generic or confirmatory replies fail to distinguish among candidate hypotheses. D4CQ evaluates the diagnosticity of each turn by measuring how selectively the observed response \(r_k\) supports the true intent \(x_i\) over competing intents \(x_j\) under the frozen responsor \(p_\phi\). The length-normalized log-likelihood is computed as \(\ell_j^{(k)} = \frac{1}{|r_k|} \log p_\phi(r_k \mid q, \bar{c}_k, x_j, A(x_j))\). Turn-level diagnosticity is quantified using the log-mean-exp margin:

\[d^{(k)}(\tau) = \ell_i^{(k)} - \log \left( \frac{1}{m_q - 1} \sum_{j \neq i} \exp \ell_j^{(k)} \right)\]

Aggregating across turns yields the trajectory score \(\bar{d}(\tau) = \frac{1}{K(\tau)} \sum_{k=1}^{K(\tau)} d^{(k)}(\tau)\), converted into a nonnegative scaling factor \(w(\tau) = \mathrm{softplus}(\beta \bar{d}(\tau))\). The token-level cross-entropy distillation loss is reweighted as \(L_{\mathrm{DD}}(\tau) = w(\tau) \sum_{t \in T(\tau)} \mathrm{CE}_t\), focusing student optimization on highly diagnostic clarifications while downweighting uninformative interactions.

4. Hindsight Relabeling with Compatibility Gating: Recycling Rollouts

Conventional trajectory distillation enforces a rigid answer correctness threshold (\(\psi(\hat{a}, A(x_i)) \ge \eta\)), discarding sensible rollouts that converge onto another valid intent of the ambiguous query. D4CQ recycles such failed rollouts by identifying whether the terminal prediction \(\hat{a}\) aligns with an alternative intent \(x_j\), selecting \(j^* = \arg\max_{j \in I(\tau)} \psi(\hat{a}, A(x_j))\) where \(I(\tau) = \{j \neq i : \psi(\hat{a}, A(x_j)) \ge \eta\}\). To ensure that historical response turns remain plausible under the relabeled target, a likelihood-based compatibility gap is checked:

\[\Delta_{i, j^*}(\tau) = \sum_{k=1}^{K(\tau)} \left( \ell_i^{(k)} - \ell_{j^*}^{(k)} \right)\]

A candidate is accepted only if \(\Delta_{i, j^*}(\tau) \le \epsilon\). Furthermore, accepted hindsight trajectories supervise only the interrogator's reasoning and clarifying question tokens, omitting the terminal answer span to avoid forcing incompatible answer tokens.

Loss & Training

The joint optimization objective balances dynamically weighted token distillation and representation-level intent convergence:

\[L = \frac{1}{|D_o \cup D_r|} \sum_{\tau \in D_o \cup D_r} \left( L_{\mathrm{DD}}(\tau) + \lambda L_{\mathrm{IC}}(\tau) \right)\]

where \(D_o\) and \(D_r\) represent the originally retained and hindsight-relabeled trajectory sets, respectively. The model is trained using LoRA with 4-bit quantization via Unsloth, optimized with AdamW across 4 A100-80GB GPUs over 4 epochs (learning rate \(2 \times 10^{-4}\), batch size 64). Key hyperparameters are configured as: retention threshold \(\eta = 0.8\), intent convergence weight \(\lambda = 1.0\), diagnosticity reweight factor \(\beta = 1.5\), and compatibility threshold \(\epsilon = 1.0\). Shared prefix KV caching over \(h_{K(\tau)+1} \oplus \pi(\tau)\) is employed to avoid redundant forward computations across candidate intents in Eq. (4).

Key Experimental Results

Main Results

Evaluations on the AmbigQA benchmark measure alias-max F1 on unambiguous (\(F1_u\)), ambiguous (\(F1_a\)), and full test subsets (\(F1_{all}\)), where "#" indicates the average number of clarifying turns. The proposed D4CQ is benchmarked against direct answering, prompt-based ambiguity-aware baselines, and preference-optimized CDADPO.

Student Model Method Clarification Turns (#) Unambiguous \(F1_u\) Ambiguous \(F1_a\) Full Subset \(F1_{all}\)
LLaMA-3-8B DIRECT ANSWER 0 25.0 16.8 21.1
LLaMA-3-8B AMBIG-AWARE 0.12 27.4 20.9 23.0
LLaMA-3-8B APAGEN 0.37 31.6 24.1 27.9
LLaMA-3-8B CDADPO 1.00 42.9 31.7 36.5
LLaMA-3-8B D4CQ (Ours) 1.08 47.2 34.3 40.1
Gemma-7B DIRECT ANSWER 0 31.2 19.2 24.8
Gemma-7B AMBIG-AWARE 0.17 34.8 23.6 28.7
Gemma-7B APAGEN 0.39 29.8 23.9 25.3
Gemma-7B CDADPO 1.00 40.7 28.6 33.9
Gemma-7B D4CQ (Ours) 1.06 42.6 31.1 36.0

Zero-shot cross-distribution evaluation on SituatedQA-Geo (geographic ambiguity) and SituatedQA-Temp (temporal ambiguity) with Mistral-7B, measuring unambiguous direct-answer F1 (\(F1_{unp}\)) and ambiguity detection F1 (\(F1_{detect}\)):

Benchmark Metric DIRECT ANSWER AMBIG-AWARE APAFIXED D4CQ (Ours)
SituatedQA-Geo \(F1_{unp}\) 11.29 3.66 38.43 41.01
SituatedQA-Geo \(F1_{detect}\) 0.00 26.01 41.84 44.75
SituatedQA-Temp \(F1_{unp}\) 15.34 8.43 45.01 47.64
SituatedQA-Temp \(F1_{detect}\) 0.00 22.48 43.95 48.12

Ablation Study

Component ablation on the ambiguous subset of AmbigQA with LLaMA-3-8B isolates procedural scaffolding and intent alignment (I1: PROCICA), dynamic distillation (I2: DYNDIST), and hindsight relabeling (I3: HINDREL):

Pipeline Category Configuration Ambiguous \(F1_a\) Relative Gain \(\Delta F1_a (\%)\) Note
Inference only AMBIG-AWARE 20.9 โ€“ Standard prompt baseline
Inference only PROCEDURAL-SCAFFOLDING 27.8 โ€“ Zero-shot structured scaffold without training
Training PROCICA (I1) + DYNDIST (I2) 32.5 +16.9% w/o hindsight relabeling
Training PROCICA (I1) + HINDREL (I3) 33.3 +19.8% w/o dynamic diagnosticity reweighting
Training DYNDIST (I2) + HINDREL (I3) 33.9 +21.9% w/o intent-convergence loss
Full Model D4CQ (Proposed) 34.3 +23.4% Full unified framework

Intent-convergence progression on trajectories containing \(\ge 2\) clarifying turns:

Model Configuration Final Margin \(\Delta_{IC}^{(K(\tau))}\) \(\mathrm{Hit@1}_{int}\) Monotonic Rate (MR) Per-turn Gain \(\Delta \mathrm{sim}\)
D4CQ w/o Intent Alignment (-ICA) 0.034 0.481 0.419 0.042
D4CQ w/o Dynamic Distillation (-DYNDIST) 0.065 0.569 0.505 0.080
D4CQ (Full Model) 0.079 0.612 0.608 0.094

Key Findings

  • Higher Quality without Over-Questioning: On LLaMA-3-8B, D4CQ achieves a substantial gain over CDADPO on ambiguous questions (+8.2% from 31.7 to 34.3) and full test questions (+9.9% from 36.5 to 40.1) while maintaining an average clarification turn count of 1.08 (virtually identical to CDADPO's 1.00), demonstrating that improvements stem from better question quality rather than excessive clarification.
  • Critical Sample Recovery via Hindsight Relabeling: Pairwise ablations show that omitting hindsight relabeling causes the largest drop (from 34.3 to 32.5), validating that salvaging failed rollouts that reach valid alternative answers provides indispensable coverage for challenging ambiguity patterns.
  • Monotonic Intent Convergence: D4CQ achieves a monotonic intent similarity increase rate of 60.8% across turns and a 0.612 \(\mathrm{Hit@1}_{int}\). Removing the contrastive alignment objective drops \(\mathrm{Hit@1}_{int}\) to 0.481, proving that representation alignment actively drives the query toward the gold intent.
  • Robust Transfer under Distribution Shift: Zero-shot transfer to SituatedQA confirms that D4CQ excels not only in recognizing when to clarify (\(F1_{detect}\) improved by +2.91 on Geo and +4.17 on Temp over APAFIXED), but also in accurately answering when inputs are already clear (\(F1_{unp}\) improved to 41.01 and 47.64).

Highlights & Insights

  • Likelihood-Margin Diagnosticity Metric: Evaluating response log-likelihood margins across competing candidate intents offers an unsupervised, reward-model-free method to assess the information gain of clarifying questions.
  • Gated Hindsight Policy Recycling: Adapting hindsight experience replay to conversational distillation with an explicit multi-turn compatibility gap safely salvages high-quality interaction trajectories without introducing intent-response divergence.
  • Bridging Sequence Generation and Embedding Geometry: Combining step-wise reasoning prompts with an InfoNCE representation loss at the final query rewrite step successfully couples autoregressive generation with continuous semantic convergence.

Limitations & Future Work

  • Evaluation Restricted to Short-Form QA: The formulation focuses on factual, short-answer open-domain questions; its effectiveness in long-form generation or multi-step agent tool use requires further investigation.
  • Simulated Responsor vs. Human Behavior: Experiments rely on a frozen instruction-tuned LLM as the user surrogate; real human users may provide ambiguous, incomplete, or uncooperative replies.
  • Predefined Candidate Intent Sets: Calculating diagnosticity margins and contrastive losses assumes a known, discrete set of candidate interpretations, which may not be readily available in open-ended deployment scenarios.
  • vs CDADPO: CDADPO optimizes clarify-or-answer decisions via downstream outcome preferences without assessing the informational diagnosticity of individual questions, whereas D4CQ dynamically prioritizes questions that discriminate between intents.
  • vs APAGEN / APAFIXED: APA focuses on aligning models to output clarification tokens rather than answering, whereas D4CQ addresses both question selection and the multi-turn reasoning structure needed to converge toward latent intents.
  • vs Standard Trajectory Distillation (e.g., STaR): Traditional rejection-sampling distillation discards all non-gold rollouts; D4CQ's compatibility-gated hindsight relabeling recycles alternative valid trajectories, dramatically improving data efficiency.

Rating

  • Novelty: โญโญโญโญโ˜† Principled integration of diagnosticity scoring and hindsight relabeling for conversational clarification.
  • Experimental Thoroughness: โญโญโญโญโญ Comprehensive evaluations across multiple student backbones, out-of-domain transfer, and detailed intent dynamics.
  • Writing Quality: โญโญโญโญโญ Rigorous mathematical formulation and structured problem analysis.
  • Value: โญโญโญโญโ˜† Highly relevant for practical, low-latency conversational agents requiring calibrated clarification policies.