The Trilemma of Truth in Large Language Models¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/carlomarxdk/trilemma-of-truth
Area: Hallucination Detection
Keywords: veracity probing, sparse-aware multiple-instance learning, conformal prediction, three-valued logic, linear representation probing
TL;DR¶
Addressing three flawed assumptions in conventional binary veracity probing, this paper proposes sAwMIL, a multiclass probing framework combining sparse multiple-instance learning with conformal prediction that reliably isolates true, false, and neither signals with causal steering guarantees across 16 LLMs.
Background & Motivation¶
As large language models (LLMs) achieve widespread deployment, the public increasingly attributes human-like epistemic faculties to them, assuming that these systems "know" objective facts. In reality, LLMs merely encode probabilistic knowledge retained during pre-training. Because models generate text with an authoritative and fluent tone, users struggle to recognize hallucinations and misinformation. Existing approaches to evaluate whether an LLM's internal probabilistic knowledge reflects the truth fall broadly into prompt-based black-box evaluations and representation-based internal probing. However, prompting is notoriously sensitive to surface phrasing, syntactic perturbations, and label skew, making internal activation probing the primary tool for mechanistic understanding.
Despite rapid adoption, current representation-based veracity probing rests on three deeply flawed assumptions. First, the single-axis assumption presumes that truth and falsehood are encoded bidirectionally along a single axis (\(P(\phi \mid \mathcal{K}_M) = 1 - P(\neg \phi \mid \mathcal{K}_M)\)), forcing all statements into a binary partition and blinding probes to propositions that lie completely outside the model's learned knowledge. Second, the ground-truth alignment assumption assumes external dataset labels serve as faithful proxies for the LLM's actual internal knowledge, confusing objective world truth with parameter retention and treating uncalibrated hyperplane distances as true degrees of belief. Third, the last-token assumption presumes that the full veracity signal faithfully accumulates at the final token (e.g., the terminal period), ignoring the reality that propositions actualize at specific informative tokens and leading probes to capture spurious correlations on non-actualized tokens.
To escape this forced binary trap, veracity probing must adopt three-valued logic capable of distinguishing between internal support for a claim, internal support for its negation, and an absence of internal knowledge. The core idea is to frame veracity probing as a three-valued problem over True, False, and Neither, introducing the sAwMIL framework that couples sparse-aware multiple-instance learning across all statement tokens with conformal prediction to deliver statistically guaranteed prediction sets.
Method¶
The proposed sAwMIL (Sparse-Aware Multiple-Instance Learning) generalizes support vector machines to the sparse multiple-instance regime by treating the token representations of a statement as a bag of instances, followed by split conformal prediction to deliver uncertainty calibration with guaranteed marginal coverage.
Overall Architecture¶
In the sAwMIL pipeline, an input statement is tokenized, and hidden representations across tokens at a selected decoder layer are collected into a bag. Three separate one-vs-all sAwMIL classifiers are trained for is-true, is-false, and is-neither. Each one-vs-all classifier executes three sequential stages: hybrid bag-instance support vector classification, quantile-based pseudo-label assignment over actualized tokens, and instance-level support vector optimization over all relabeled tokens. The raw outputs are converted into multiclass probabilities via softmax regression and subsequently calibrated by conformal prediction to generate prediction sets.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input Statement & Token Activation Bag"] --> B["Stage 1: Hybrid Bag-Instance SVM"]
B --> C["Stage 2: Quantile & Actualization Relabeling"]
C --> D["Stage 3: Token-Level Single-Instance SVM"]
D --> E["One-vs-All Softmax Probability Fusion"]
E --> F["Conformal Prediction Nonconformity Calibration"]
F -->|Guaranteed Coverage Set| G["Output: True / False / Neither Prediction Set"]
Key Designs¶
1. Hybrid Bag-Instance SVM: Filtering Irrelevant Tokens Under Asymmetric Bag Constraints
Standard single-instance learning requires individual token labels that are unavailable in sentences where only a few tokens reflect factual actualization. sAwMIL's first stage deploys sparse multiple-instance learning (sMIL). For negative bags, every instance is a true negative, yielding strict instance-level margin constraints \(\theta^\top \phi(x_j) + \beta \le -1 + \xi_{x_j}\). For positive bags, instances are predominantly uninformative with very few true positives; hence, sAwMIL imposes a relaxed constraint on the mean representation of each positive bag:
This convex quadratic program penalizes violations while minimizing \(\|\theta\|^2\), yielding initial parameters \((\hat{\theta}, \hat{\beta})\) that separate informative positive tokens from pervasive background instances without requiring pre-assigned token labels.
2. Quantile-Based Actualization Relabeling: Isolating Spurious Correlations with Structural Priors
Although the first stage establishes a separating hyperplane, individual token pseudo-labels remain unresolved. The second stage exploits the statement's syntactic structure, decomposing it into a pre-actualized span \(x_p\) (e.g., "The city of Riga is in") which inherently carries no veracity value, and an actualized span \(x_a\) (e.g., "Latvia.") where veracity actualizes. sAwMIL computes the \((1 - \eta)\)-quantile threshold \(q = Q_{1-\eta}(S^+)\) over all positive bag scores \(S^+\), combining it with the intra-bag actualization mask \(m \in \{0, 1\}^L\):
This operation purges non-actualized filler tokens and anchors the veracity signal strictly to actualized semantic units, preventing the probe from learning artifactual cues.
3. Token-Level SVM and Three-Way Decoupling: Dismantling the Bipolar Symmetry Assumption
With the relabeled dataset \(\tilde{\mathcal{D}}\) containing clean instance-pseudolabel pairs, the third stage discards the bag grouping and optimizes a standard single-instance SVM to obtain the final direction \(\theta\) and intercept \(\beta\). Crucially, sAwMIL trains three separate one-vs-all classifiers (is-true, is-false, and is-neither), extracting distinct vectors \(\theta_{\text{true}}, \theta_{\text{false}}, \theta_{\text{neither}}\) before combining them via softmax regression \(p_k = \frac{\exp(z_k)}{\sum_j \exp(z_j)}\). Analysis reveals that the cosine similarity between \(\theta_{\text{true}}\) and \(\theta_{\text{false}}\) ranges between \(-0.4\) and \(-0.7\) rather than \(-1.0\), with a matrix rank of 2 and an effective rank of \(1.93 \pm 0.004\). This demonstrates that truth and falsehood in LLMs span a multidimensional subspace rather than lying on a single bidirectional axis.
4. Multiclass Conformal Prediction: Supplying Statistically Calibrated Prediction Sets
Raw SVM distances and softmax scores fail to represent calibrated epistemic probabilities. sAwMIL integrates split conformal prediction using a held-out calibration set \(\mathcal{D}_{\text{cal}}\). For any candidate label \(y\), it evaluates a multiclass nonconformity score based on the probability margin:
At test time with significance level \(\alpha = 0.1\), sAwMIL computes conformal p-values to construct prediction set \(C(x)\) with \(\ge 90\%\) marginal coverage. Singleton sets signal confident classifications, multi-element sets quantify epistemic ambiguity, and empty sets signify out-of-distribution inputs where the probe abstains.
Key Experimental Results¶
Main Results¶
Evaluation spans 16 open-source LLMs (3B to 14B parameters across Gemma, Llama-3, Mistral, and Qwen-2.5 families) on three newly curated benchmarks: City Locations, Medical Indications, and Word Definitions (each featuring synthetically generated entities for neither statements). Performance is reported using multiclass Matthews Correlation Coefficient (MCC).
| Evaluation Setting | Probing Method | City Locations (MCC) | Medical Indications (MCC) | Word Definitions (MCC) | Avg Generalization MCC |
|---|---|---|---|---|---|
| Full Bag (Sentence) | sAwMIL (Ours) | 0.99 ยฑ 0.01 | 0.87 ยฑ 0.02 | 0.89 ยฑ 0.02 | 0.85 ยฑ 0.01 |
| Full Bag (Sentence) | Multiclass SVM | 0.52 ยฑ 0.04 | 0.44 ยฑ 0.05 | 0.58 ยฑ 0.04 | 0.34 ยฑ 0.04 |
| Full Bag (Sentence) | TTPD+CP | 0.28 ยฑ 0.06 | 0.14 ยฑ 0.05 | 0.14 ยฑ 0.05 | 0.11 ยฑ 0.03 |
| Full Bag (Sentence) | MD+CP | 0.06 ยฑ 0.03 | 0.08 ยฑ 0.03 | 0.04 ยฑ 0.02 | 0.03 ยฑ 0.01 |
| Full Bag (Sentence) | sPCA+CP | 0.25 ยฑ 0.06 | 0.23 ยฑ 0.05 | 0.10 ยฑ 0.03 | 0.12 ยฑ 0.03 |
| Last Token Only | sAwMIL (Ours) | 0.99 ยฑ 0.01 | 0.87 ยฑ 0.02 | 0.89 ยฑ 0.02 | 0.85 ยฑ 0.01 |
| Last Token Only | Multiclass SVM | 0.97 ยฑ 0.01 | 0.81 ยฑ 0.03 | 0.89 ยฑ 0.02 | 0.71 ยฑ 0.02 |
| Last Token Only | TTPD+CP | 0.67 ยฑ 0.04 | 0.63 ยฑ 0.03 | 0.41 ยฑ 0.02 | 0.49 ยฑ 0.03 |
| Last Token Only | MD+CP | 0.68 ยฑ 0.02 | 0.55 ยฑ 0.02 | 0.40 ยฑ 0.01 | 0.29 ยฑ 0.02 |
| Zero-shot Prompt | Direct Prompting | 0.48 ยฑ 0.06 | 0.29 ยฑ 0.04 | 0.14 ยฑ 0.03 | - |
Causal Steering Ablation¶
To verify whether learned directions causally affect generation rather than picking up surface correlations, Contrastive Token Intervention (CTI) is conducted across 48 \(\langle\text{LLM}, \text{dataset}\rangle\) configurations using a difference-in-differences (DiD) regression, evaluating Normalized Interaction Effect (NIE) and Success Rate (SR):
| Steering Direction | Dose Multiplier (\(d'\)) | Normalized Interaction Effect (NIE) | 95% CI | Heterogeneity (\(\tau^2\)) | Success Rate (SR) | Significant Runs (\(k/n\)) |
|---|---|---|---|---|---|---|
| sAwMIL (is-true) | 3 | 0.439 | (0.357, 0.520) | 0.0768 | 85.4% | 41 / 48 |
| sAwMIL (is-true) | 1 | 0.350 | (0.271, 0.428) | 0.0722 | 68.8% | 33 / 48 |
| TTPD (is-true) | 3 | 0.411 | (0.332, 0.491) | 0.0735 | 83.3% | 40 / 48 |
| TTPD (is-true) | 1 | 0.329 | (0.261, 0.397) | 0.0517 | 66.7% | 32 / 48 |
| SVM (is-true) | 3 | 0.334 | (0.267, 0.402) | 0.0511 | 70.8% | 34 / 48 |
| SVM (is-true) | 1 | 0.253 | (0.198, 0.309) | 0.0334 | 58.3% | 28 / 48 |
| MD (mean-difference) | 3 | 0.155 | (0.116, 0.193) | 0.0152 | 50.0% | 24 / 48 |
| MD (mean-difference) | 1 | 0.068 | (0.050, 0.086) | 0.0017 | 14.6% | 7 / 48 |
Key Findings¶
- Binary probes collapse under full-bag evaluation: While mean-difference (MD+CP) and polarity probes (TTPD+CP) achieve moderate MCC (0.40โ0.68) on the last token, their performance collapses to near-zero (0.03โ0.11) across full token bags. Confusion matrix inspection confirms that binary probes consistently misclassify neither statements into high-confidence true or false regions.
- Truth and falsehood are not polar opposites: Measuring cosine similarity between learned
is-trueandis-falsevectors yields values significantly above -1.0, with an effective matrix rank of 1.93, proving that truthfulness and falsehood occupy distinct representational dimensions. - Causal relevance and robust transferability: In CTI steering experiments, sAwMIL achieves the highest causal impact (NIE = 0.439, SR = 85.4%), demonstrating that its isolated veracity vectors directly modulate token probabilities on correct factual completions while maintaining an average cross-domain generalization MCC of 0.85.
Highlights & Insights¶
- Three-valued epistemic grounding: Expanding veracity beyond binary truth values to explicitly incorporate unknown/unsupported states captures the intrinsic boundary of LLM parameter knowledge.
- Weakly supervised MIL with actualization priors: Coupling bag-level constraints with quantile-based token relabeling resolves the tension between prohibitive per-token annotation costs and last-token over-reliance.
- Unified diagnostic and intervention paradigm: Demonstrating that probe directions successfully steer model generations under Difference-in-Differences analysis validates their causal authenticity for model auditing and knowledge editing.
Limitations & Future Work¶
- Relational statement restriction: Evaluated stimuli are limited to simple two-entity relational statements (e.g., city-country, drug-disease) with single negation, leaving multi-hop and compositional claims for future study.
- Synthetic entity proxies: The neither class relies on Markov-generated fictitious entities; while mitigating pre-training data leakage, they may not fully capture human temporal uncertainty or contextual ambiguity.
- Linearity boundary: The study explores linear hyperplanes; whether non-linear representations or dynamic attention head routing provide richer geometric veracity structures remains open.
Related Work & Insights¶
- vs. Mean-Difference Probe (Marks & Tegmark, 2024): Presumes a single bidirectional axis and last-token sufficiency; sAwMIL exposes the multi-dimensional subspace of truth and provides explicit abstention for unlearned concepts.
- vs. TTPD Probe (Bรผrger et al., 2024): Decouples truth and grammatical polarity but remains vulnerable to pre-actualized token noise; sAwMIL enforces actualization filtering and conformal calibration.
- vs. Zero-Shot Prompting (Abbasi-Yadkori et al., 2024): Prompting suffers from strong label skews and surface phrasing sensitivity; sAwMIL extracts robust, calibrated veracity signals directly from intermediate activations.
Rating¶
- Novelty: โญโญโญโญโญ Formulates the trilemma of truth in LLMs, successfully integrating three-valued logic, sparse MIL, and conformal calibration.
- Experimental Thoroughness: โญโญโญโญโญ Rigorous evaluation across 16 LLMs from 4 families, 3 custom datasets, and causal Difference-in-Differences steering.
- Writing Quality: โญโญโญโญโญ Crisp problem framing, clear mathematical exposition, and disciplined empirical execution.
- Value: โญโญโญโญโญ High foundational value for hallucination detection, representation engineering, model auditing, and factual reliability assurance.