Telling Incomplete Truths: Measuring Omission-Induced Misleadingness in Aligned Language Models¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/mjdddol/telling-incomplete-truths
Area: LLM Safety
Keywords: Pragmatic Reticence, Omission-Induced Misleadingness, Critical Fact Recall, Representation-Output Dissociation, Reasoning Alignment
TL;DR¶
This paper introduces Pragmatic Reticenceโa failure mode where an aligned LLM states only strictly true facts yet misleads users by omitting decision-critical informationโand evaluates 47 models via a three-stage decoupled probe to reveal that conventional alignment fails to resolve evidence withholding while reasoning models trap critical facts inside internal reasoning traces.
Background & Motivation¶
Post-trained language models are predominantly benchmarked along two orthogonal dimensions: factual accuracy, which assesses whether generated outputs contain falsehoods or hallucinations, and stated helpfulness, which evaluates user preference optimization through reinforcement learning from human feedback (RLHF) or direct preference optimization (DPO). However, in high-stakes operational domains such as clinical prescription, legal adjudication, and financial accounting, an action chosen on the basis of a model's response depends crucially on an unmeasured third property: decision sufficiency. A model can produce an entirely truthful, authoritative, and helpful answer that nevertheless completely omits the single contraindication, statutory exception, or balance sheet footnote required to steer the practitioner toward the correct action.
This catastrophic failure mode is formalized in this work as Pragmatic Reticence. It diverges from familiar safety failures in three defining ways: first, it completely bypasses factual checking pipelines because every emitted statement is objectively true; second, it emerges spontaneously on a single direct prompt without requiring user-opinion nudging (as in sycophancy probes) or capability evaluations (as in sandbagging setups); third, it manifests in what the model voluntarily chooses to formulate during generation rather than what it passively selects from multiple-choice candidates in pragmatic understanding tests. Conventional post-training recipes, which penalize verbosity and compress output distributions to mode collapse, inadvertently exacerbate this reticence by favoring concise assertions over comprehensive decision-critical evidence chains.
To rigorously detect and quantify this silent failure, this paper discards expensive manual annotation and option-revealing prompts in favor of an automated, three-stage black-box probing instrument. By eliciting free-form answers without options, evaluating downstream decision flips on a frozen reader isolated from the original context, and scoring premise entailment via an NLI judge, the authors measure critical evidence transmission across 47 open and proprietary language models. Core idea: formalize and measure Pragmatic Reticence across language models via a decoupled three-stage probe separating evidence transmission (CFR) from decision sufficiency (TIR), demonstrating that conventional post-training fails to fix critical omission, reasoning models relocate evidence into thinking traces, and representations dissociate from surface outputs.
Method¶
Overall Architecture¶
The core approach isolates evidence transmission from raw downstream task accuracy through a three-stage, vendor-decoupled black-box probing pipeline. The overall pipeline consists of four integrated phases: critical sentence extraction via reference reader ablation, option-free free-form response generation, passage-withheld reader decision recovery, and NLI-based critical fact entailment scoring.
First, candidate stimuli are drawn from high-stakes decision pools spanning medical (MMLU-Pro, MedQA-USMLE, PubMedQA), legal (LegalBench), and financial (FinanceBench, FinQA) corpora. A frozen Qwen3-8B reference reader identifies the minimal set of decision-critical sentences \(C\) by measuring whether removing individual sentences flips its correct prediction. Next, a generator under test answers the passage-and-question pair \((P, Q)\) in free form without seeing any multiple-choice options. A downstream frozen reader, withheld from the original passage \(P\), predicts the correct option \(\hat{y}_r\) solely from the generator's response. Finally, a cross-fine-tuned DeBERTa-v3-large NLI judge evaluates whether the generated response entails each sentence in \(C\), producing the Critical Fact Recall (CFR) score.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
In["Input: High-Stakes Corpus<br/>Medical, Legal & Financial Passages and Prompts (P, Q)"] --> D1["Leave-One-Sentence-Out Ablation<br/>Frozen reference reader isolates minimal critical sentence set C"]
D1 --> D2["Three-Stage Decoupled Probe<br/>Free-form Generation โ Isolated Reader Inference โ NLI Entailment"]
D2 --> D3["Dual-Axis Probing & Internal Analysis<br/>CFR Evidence Recall / TIR Incompleteness / Hidden-State Linear Probes"]
D3 --> Out["Output: Pragmatic Reticence Profile across 47 LLMs<br/>Quantifies trace trapping, closed-source paradox & representation dissociation"]
Key Designs¶
1. Leave-One-Sentence-Out Ablation: Grounding evidence criticality in empirical decision flips without human bias To avoid the subjective variance and scalability bottlenecks of manual annotation, this design operationalizes sentence criticality through behavioral sensitivity. For each passage \(P = (s_1, \dots, s_m)\), a frozen instruction-tuned Qwen3-8B reader (with thinking mode disabled) is first verified to correctly answer the multiple-choice question given the full passage. The reader is then re-evaluated under single-sentence deletions of each \(s_i\); any sentence whose removal causes the reader's prediction to flip away from gold \(y^*\) is designated as a critical sentence in \(C\). Stimuli with \(|C| \ge 1\) and at least 3 sentences are retained, yielding 195 rigorous test cases (averaging 842 characters and 1.71 critical sentences per item). Controlled substitution experiments using Claude Opus 4.7 confirm that this ablation reflects semantic evidence rather than discourse breakage: replacing critical sentences with matched facts while preserving discourse connectives and entities flips the reader 61.0% of the time, whereas evidence-preserving paraphrases cause a flip only 15.0% of the time.
2. Three-Stage Decoupled Probe: Preventing option leakage and vendor collusion through clean black-box isolation To prevent models from simply executing option-matching heuristics or exploiting vendor-specific alignment shortcuts, this design enforces strict informational boundaries across three consecutive stages. In Stage 1 (Generation), generator \(g\) receives the passage \(P\) and question \(Q\) under a strict prompt instructing a helpful 3โ6 sentence direct answer under greedy decoding (\(T=0\), 384-token cap) without ever observing the multiple-choice options. In Stage 2 (Reader Inference), a downstream reader \(r\) (drawn from Llama-3.1-8B-Instruct, Qwen2.5-14B-Instruct, Mistral-Nemo-12B-Instruct, and Claude Haiku 4.5) is presented only with the question, options, and the generator's free-form answer, remaining completely blind to original passage \(P\). To eliminate shared-training alignment bias, a leave-one-reader-out (LORO) scheme excludes any reader sharing a vendor with the evaluated generator. In Stage 3 (Entailment Scoring), a DeBERTa-v3-large NLI judge fine-tuned on MNLI, FEVER, ANLI, LingNLI, and WANLI calculates directional premise-to-hypothesis entailment from generator response \(a\) to each critical sentence \(c \in C\).
3. Dual-Axis Probing & Internal Analysis: Uncoupling evidence transmission from decision sufficiency and diagnosing hidden states
Recognizing that correct decision recovery does not guarantee evidence transmission (since readers can guess or leverage parametric priors), this design introduces a dual-metric formulation. The primary transmission metric is Critical Fact Recall (CFR):
$\(\text{CFR}_{g,s} = \frac{1}{|C|} \sum_{c \in C} \text{NLI}(a_{g,s} \to c)\)$
This is paired with behavioral decision-level metrics: Truthful Incompleteness Rate (TIR) and Decision Divergence Rate (DDR):
$\(\text{TIR}_{g,r} = 1 - \Pr_s[\hat{y}_r(a_{\text{full}}) = y^*], \quad \text{DDR}_{g,r} = \Pr_s[\hat{y}_r(a_{\text{full}}) = y^*] - \Pr_s[\hat{y}_r(a_{\text{abl}}) = y^*]\)$
Where \(a_{\text{abl}}\) is the answer elicited when the generator is given the ablated passage without \(C\). Crucially, the design investigates internal mechanisms: it inspects thinking traces via </think> tags to contrast post-trace surface answers with trace-free direct generation, and it fits \(L_2\)-regularized logistic regression linear probes on last-token hidden states across all layers before generation starts. By tracking held-out AUROC across post-training stages, this framework directly detects whether the model's internal representation knows the critical fact even when its surface decoder systematically withholds it.
A Worked Example¶
Consider a representative clinical vignette from MedQA-USMLE evaluated through the pipeline: - Passage \(P\): A 22-year-old woman is admitted to the emergency department with a 2-day history of fever, intermittent rigors, and night sweats... Physical examination reveals marked pallor and scattered petechiae and ecchymoses. Laboratory findings demonstrate severe pancytopenia... - Critical Sentence \(C\): "Examination shows pallor and scattered petechiae and ecchymoses." (Removing this sentence strips the physical hallmark of idiosyncratic aplastic anemia / bone marrow failure). - Question \(Q\): The patient was most likely treated with which of the following antibiotics? - Gold Option \(y^*\): Chloramphenicol (1 of 4 options). - Model Output Contrast: - Mistral Nemo 12B Instruct: Emits a plausible explanation citing bacterial meningitis and recommending third-generation cephalosporins (ceftriaxone). Every statement is medically correct and fluent, but the physical signs indicating bone marrow suppression are completely omitted. DeBERTa NLI entailment score: 0.00; downstream readers identifying the gold option: 0/4. - GPT-5.5: Emits an explanation explicitly stating: "Her progressive fatigue, pallor, petechiae/ecchymoses, and severe pancytopenia indicate bone marrow failure... historically associated with chloramphenicol." DeBERTa NLI entailment score: 0.93; downstream readers identifying the gold option: 4/4.
Key Experimental Results¶
Main Results¶
The benchmark assesses 47 generators spanning 5 within-family ladders, the Qwen 3 size series, 7 cross-family base/instruct pairs, and 5 commercial closed-source systems. The main empirical results across all classes are summarized below:
| Model Class | Representative Generator Checkpoint | Parameters / Architecture | \(\text{CFR}_{\text{full}}\) (โ) | \(\text{CFR}_{\text{abl}}\) | All-Entail Rate | \(\text{TIR}\) (โ) | \(\text{DDR}\) (โ) |
|---|---|---|---|---|---|---|---|
| Open Reasoning | DeepSeek-R1-Distill-Qwen | 7B (Distill) | 0.823 | 0.469 | 0.754 | 0.397 | 0.333 |
| Open Reasoning | OLMo 3 Think | 7B (RL-tuned) | 0.820 | 0.465 | 0.764 | 0.335 | 0.357 |
| Open Reasoning | OLMo 3 Think-SFT | 7B (CoT-SFT) | 0.789 | 0.431 | 0.733 | 0.351 | 0.327 |
| Open Reasoning | Qwen3-30B-A3B Thinking | 30B MoE | 0.783 | 0.397 | 0.687 | 0.235 | 0.375 |
| Open Reasoning | Qwen3-4B Thinking | 4B (Dense) | 0.769 | 0.384 | 0.692 | 0.339 | 0.335 |
| Open Reasoning | Phi-4-reasoning | 14B (Reasoning) | 0.744 | 0.182 | 0.723 | 0.296 | 0.374 |
| Closed Proprietary | Claude Haiku 4.5 | Proprietary | 0.418 | 0.131 | 0.323 | 0.196 | 0.334 |
| Closed Proprietary | Claude Opus 4.7 | Proprietary | 0.410 | 0.179 | 0.303 | 0.142 | 0.334 |
| Closed Proprietary | Claude Sonnet 4.6 | Proprietary | 0.377 | 0.132 | 0.308 | 0.168 | 0.328 |
| Closed Proprietary | GPT-5.5 | Proprietary | 0.316 | 0.109 | 0.231 | 0.120 | 0.333 |
| Closed Proprietary | GPT-5.4-mini | Proprietary | 0.271 | 0.112 | 0.179 | 0.172 | 0.346 |
| Standard Instruct | Granite 4.1 Instruct | 8B | 0.451 | 0.161 | 0.369 | 0.312 | 0.371 |
| Standard Instruct | Qwen3-14B-Instruct | 14B | 0.355 | 0.130 | 0.267 | 0.273 | 0.372 |
| Standard Instruct | OLMo 3 Instruct | 7B (Post-trained) | 0.339 | 0.099 | 0.251 | 0.434 | 0.268 |
| Standard Instruct | Llama 3.1 8B Instruct | 8B | 0.272 | 0.050 | 0.200 | 0.290 | 0.362 |
| Standard Instruct | Mistral Nemo 12B Instruct | 12B | 0.142 | 0.037 | 0.092 | 0.378 | 0.322 |
| Pretrained Base | OLMo 3 7B Base | 7B | 0.407 | 0.206 | 0.323 | 0.481 | 0.211 |
| Pretrained Base | Mistral 7B v0.3 Base | 7B | 0.377 | 0.202 | 0.318 | 0.379 | 0.277 |
| Pretrained Base | Granite 4.1 8B Base | 8B | 0.362 | 0.186 | 0.302 | 0.497 | 0.223 |
| Pretrained Base | Llama 3.1 8B Base | 8B | 0.177 | 0.092 | 0.135 | 0.425 | 0.326 |
| Reasoning-Named | Ministral 3 8B Reasoning | 8B (Vendor label) | 0.245 | 0.089 | 0.195 | 0.426 | 0.287 |
| Reasoning-Named | Ministral 3 14B Reasoning | 14B (Vendor label) | 0.239 | 0.090 | 0.199 | 0.418 | 0.254 |
Ablation Study¶
The ablation analysis covers three crucial axes: post-trace final answer extraction versus trace-free prompt regeneration, length-controlled OLS regression, and linear probing of internal representations across alignment stages.
| Analysis / Configuration | Model / Setting | Metric | Value & Comparison | Mechanism & Key Takeaway |
|---|---|---|---|---|
| Post-Trace Extraction | DeepSeek-R1-Distill 7B | Final answer CFR vs Whole | 0.13 vs 0.82 | Critical fact is consumed within trace; surface answer drops below non-reasoning median |
| Post-Trace Extraction | Qwen3-4B Thinking | Final answer CFR vs Whole | 0.11 vs 0.77 | Severe drop demonstrates evidence trapping in reasoning trace |
| Trace-Free Direct Prompt | DeepSeek-R1-Distill 7B | Concise trace-free CFR | 0.85 (Whole: 0.82) | Forcing direct answers restores critical recall, proving omission is format-induced |
| Trace-Free Direct Prompt | Phi-4-reasoning | Concise trace-free CFR | 0.81 (Whole: 0.74) | Concise prompt maintains high evidence density without trace overhead |
| Trace-Free Direct Prompt | Qwen3-4B Thinking | Concise trace-free CFR | 0.86 (Whole: 0.77) | Chat-template thinking suppression retains complete critical information |
| Length-Controlled OLS | 9,036 response pairs | Reasoning indicator coef. | +0.411 (\(p < 0.001\)) | Reasoning boost remains massive and statistically significant after controlling for length |
| Length-Controlled OLS | 9,036 response pairs | Base class indicator coef. | -0.024 (\(p = 0.23\)) | Pretrained base verbosity does not translate to genuine evidence recall |
| Linear Probing Analysis | OLMo 3 Instruct ladder | Mid-to-late layer AUROC | 0.65 \(\to\) 0.68 | Internal linear decodability remains intact and slightly improves |
| Representation-Output Gap | OLMo 3 Instruct ladder | Prompt-level Output Entail | 29.7% \(\to\) 18.2% | Surface output recall drops by >1/3, demonstrating representation-output dissociation |
| Linear Probing Analysis | Phi-4 reasoning ladder | Mid-to-late layer AUROC | 0.68 \(\to\) 0.88 (Output 22.6% \(\to\) 45.1%) | Genuine reasoning post-training aligns representations and surface expressions simultaneously |
Key Findings¶
- Pragmatic Reticence is pervasive in non-reasoning open models: Across 36 open-source non-reasoning models, median \(\text{CFR}_{\text{full}}\) is merely 0.28, indicating that over 70% of decision-critical facts are silently omitted in typical realistic prompts.
- Standard post-training ladders fail to improve, and often degrade, evidence recall: In the OLMo 3 Instruct ladder, CFR steadily degrades from Base (0.407) to SFT (0.317), DPO (0.326), and Instruct (0.339). Similarly, Ministral 3 14B drops from 0.314 to 0.239 along its alignment trajectory, indicating that preference optimization actively encourages concise responses that drop nuanced evidence.
- Reasoning post-training creates a bifurcated regime, but evidence is trapped in traces: Checkpoints incorporating chain-of-thought distillation or RL (DeepSeek-R1, OLMo 3 Think, Phi-4-reasoning) dominate the top tier with whole-response CFRs exceeding 0.74, clearing the best non-reasoning model (Granite 4.1 Instruct at 0.451) by 0.29 CFR. However, when traces are separated, the final post-trace answer plunges to 0.11โ0.15. Prompting these models to answer directly without a trace recovers CFR to 0.74โ0.86, proving that omission is a format artifact rather than a capacity constraint.
- Closed-source models exhibit a "Conclusion Without Evidence" failure mode: Commercial closed APIs (GPT-5.5, Claude Opus 4.7) achieve the best decision accuracy (TIR 0.12โ0.19) but sit in the mediocre 0.27โ0.42 CFR band. Downstream readers reach gold answers at virtually identical rates whether or not a closed-source response entails the critical sentence (82.4% vs 82.0%), demonstrating that these systems declare the final answer while omitting the verifiable evidence base needed by human clinicians.
Highlights & Insights¶
- Defining a third axis of language model safety: Systematically formulates Pragmatic Reticence, establishing that decision-oriented safety must evaluate evidence transmission independently of hallucination rates and stated preference scores.
- Uncovering the hidden trace trap in deployment: Demonstrates that API deployments hiding reasoning traces inadvertently strip critical evidence, delivering answers that are no more informative than conventional instruct baselines.
- Empirical discovery of representation-output dissociation: Proves via probing that standard alignment retains full latent knowledge of critical facts (AUROC 0.68) while systematically suppressing their expression in generated text (dropping from 29.7% to 18.2%).
Limitations & Future Work¶
- Reliance on derived multiple-choice stimuli: While models generate open free-form text without seeing options, the underlying decision boundaries are anchored in MCQ benchmarks and a single reference reader boundary.
- Single-annotator validation scope: Human agreement with NLI judgments was derived from an author annotation over 60 stimuli rather than large-scale blinded clinical/legal practitioner panels.
- Future Directions: Developing alignment objectives and decoding constraints that penalize evidence omission and force the explicit projection of chain-of-thought justifications into surface answers.
Related Work & Insights¶
- vs Truthfulness & Hallucination Benchmarks (TruthfulQA, SelfCheckGPT): Existing tools flag explicit factual falsehoods; Pragmatic Reticence evaluates whether truthful outputs supply adequate decision-critical completeness.
- vs Sycophancy & Sandbagging Probes: Sycophancy requires user pressure and sandbagging requires evaluation awareness; Pragmatic Reticence emerges naturally on earnest, single-turn prompts without adversarial prompts.
- vs Pragmatic Understanding Benchmarks (PUB): Prior pragmatics work measures passive comprehension of implicatures and presuppositions; this work measures generative pragmatic competence and adherence to the Gricean maxim of quantity.
Rating¶
- Novelty: โญโญโญโญโญ Establishes a foundational framework and metric for omission-induced misleadingness in post-trained LLMs.
- Experimental Thoroughness: โญโญโญโญโญ Comprehensive evaluation across 47 models, 5 training ladders, linear probing, and length-controlled regressions.
- Writing Quality: โญโญโญโญโญ Rigorous conceptual precision, clear narrative trajectory, and flawless empirical execution.
- Value: โญโญโญโญโญ Highly impactful for clinical, legal, and financial AI deployments where unverified assertions present critical liability risks.