TruthLens: Object Hallucination Detection via Self-Evaluating Truthfulness Scores in LVLMs¶
Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/wyqstan/TruthLens
Area: Multimodal VLM
Keywords: object hallucination detection, large vision-language models, representation separability, truthfulness score, zero-overhead self-evaluation
TL;DR¶
TruthLens reveals that real and hallucinated object tokens are clearly separable in LVLM hidden states but lose this separability at the language modeling head, and repurposes rarely-used special tokens to expose per-token truthfulness scores with zero additional inference latency.
Background & Motivation¶
Large vision-language models (LVLMs) have achieved unprecedented breakthroughs across image description, visual question answering, and complex multimodal reasoning tasks. Nonetheless, object hallucination (OH)—where models hallucinate nonexistent visual entities in generated outputs—remains an enduring vulnerability that jeopardizes reliable deployment in safety-critical domains such as healthcare and autonomous driving. Existing detection paradigms primarily approach this vulnerability via external verification pipelines: one branch deploys auxiliary teacher LVLMs or external evaluator LLMs (e.g., GAIVE, HaLEM), inevitably incurring heavy computational overhead and latency; another branch inspects visual grounding cues through attention maps (e.g., SVAR) or global-local feature similarities (e.g., GLSIM). However, these methods largely overlook the internal confidence representations implicitly embedded within the underlying language models.
Studies on large language model uncertainty indicate that internal hidden representations preserve factual correctness and predictive uncertainty. Because modern LVLMs perform autoregressive decoding in text token space under cross-modal visual conditioning, their internal feature representations should inherently reflect visual grounding veracity. Diagnostic probes conducted with Linear Discriminant Analysis (LDA) confirm this intuition: the final hidden representations of representative LVLMs (including LLaVA-1.5, Qwen2.5-VL, and LLaVA-OneVision) exhibit remarkable separability between visually grounded and hallucinated object tokens (achieving 79%–84% AUROC). However, this discriminative capability sharply collapses when measuring output probability distributions via negative log-likelihood (NLL) or predictive entropy (dropping to 63%–64% AUROC). Further projection subspace analysis demonstrates that the truthfulness discriminative direction maintains an alignment ratio below 0.025 with the effective subspace of the LM head projection matrix, exposing a systematic representation-projection mismatch.
This disparity reveals a promising path forward: instead of appending heavy external judges or designing hand-crafted feature metrics, one can teach the native LM head to preserve and surface its internal truthfulness signals directly. Core idea: repurpose the output log-probability of a designated low-frequency special token at each object token step as a self-evaluating truthfulness score, fine-tuning the model with an MSE reward objective and KL divergence constraint to achieve zero-inference-overhead hallucination detection and closed-loop mitigation.
Method¶
Overall Architecture¶
The TruthLens pipeline comprises three distinct operational stages: probe formulation and special token repurposing, balanced self-evaluation fine-tuning, and inference detection with closed-loop mitigation. During autoregressive decoding, the model generates output probability distributions across the vocabulary. TruthLens leaves core vocabulary token probabilities intact to protect generative fluency, designating a low-frequency special token (such as <unk> in LLaVA or <|image_pad|> in Qwen2.5-VL) as a dedicated truthfulness probing channel. At each object token position, the model extracts the log-probability assigned to this special token and subtracts a reference constant to compute a normalized truthfulness score \(S(o)\). During training, captions are sampled over densely annotated training images, and an adaptive class-reweighted MSE objective trains the truthfulness score toward 1 for grounded objects and 0 for hallucinations, while a sequence-level KL divergence constraint regularizes the base model against language degradation. At test time, the model performs fine-grained hallucination detection via thresholding without auxiliary tools and can iteratively refine its own output via confidence-guided self-revision.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input Image and Text Prompt<br/>Multimodal autoregressive candidate generation"] --> B["Special-Token Truthfulness Scoring<br/>Extract special token log-prob with constant shift"]
B --> C["Balanced Reweighted MSE and KL Regularization<br/>Class-adaptive MSE + frozen reference KL constraint"]
C --> D["Self-Evaluating Detection and Closed-Loop Mitigation<br/>Threshold-based detection + self-revision prompt"]
D --> E["Output Hallucination-Free Text and Confidence Scores"]
Key Designs¶
1. Special-Token Truthfulness Scoring: Non-intrusive projection of internal separability onto output logits
Directly recalibrating standard vocabulary token probabilities for hallucination detection risks destabilizing next-token prediction and degrading language fluency. TruthLens addresses this challenge by repurposing special placeholder tokens that are naturally assigned near-zero generation probabilities during normal inference (such as <unk> in LLaVA models or <|image_pad|> in Qwen2.5-VL and LLaVA-OneVision). Adjusting the logits of these low-probability tokens imposes negligible perturbation on the overall sampling distribution. For each token \(o\) corresponding to an object mention, the truthfulness score \(S(o, X, I)\) is formulated by measuring the log-probability assigned to the designated special token \(z_c\) minus a constant shift \(c_{\mathrm{ref}}\):
$\(S(o, X, I) = \log \pi_\theta(z_c \mid X, I, y_{<o}) - c_{\mathrm{ref}}\)$
where \(c_{\mathrm{ref}} = -28.0\) normalizes the negative log-probability scale into a well-behaved range near \([0, 1]\). For multi-sub-token objects, empirical analysis confirms that truthfulness signals are evenly distributed throughout sub-tokens; thus, supervising the first sub-token offers consistent performance and robust detection. This design introduces zero additional parameters and requires no extra inference forward passes.
2. Balanced Reweighted MSE and KL Regularization: Mitigating class imbalance while safeguarding general capabilities
In natural multimodal outputs, visually grounded objects heavily outnumber hallucinated objects, producing severe label imbalance that could bias scoring functions toward false confidence. Furthermore, fine-tuning risks catastrophic forgetting of general reasoning capabilities. TruthLens introduces a dual-objective training strategy to resolve these issues. Within each optimization step, the model counts the numbers of real object tokens \(N_r\) and hallucinated object tokens \(N_h\), computing dynamic class weights \(w_r = \frac{N_r + N_h}{2N_r}\) and \(w_h = \frac{N_r + N_h}{2N_h}\). To preserve core conversational abilities, the objective incorporates a sequence-level KL divergence penalty against a frozen reference model \(\pi_{\mathrm{ref}}\): $\(\mathcal{L} = \frac{1}{N_r + N_h} \sum_{(X, I)} \sum_{y \sim \pi_\theta} \sum_{o \in \mathcal{O}(y)} \left( w_r \mathbb{I}_{\{\hat{r}=1\}} + w_h \mathbb{I}_{\{\hat{r}=0\}} \right) \cdot \beta \left( S(o, X, I) - \hat{r}(o, I) \right)^2 + D_{\mathrm{KL}}(\pi_\theta(y \mid X, I) \parallel \pi_{\mathrm{ref}}(y \mid X, I))\)$ where the target reward \(\hat{r}(o, I)\) equals 1 for visually grounded objects and 0 otherwise, and \(\beta = 0.1\) stabilizes gradient updates. Optimized via LoRA (\(r=8, \alpha=16\)) on 4,000 MSCOCO images with 8 stochastic rollouts per image, the model aligns truthfulness scores within 6–12 GPU hours without degrading general multimodal benchmarks.
3. Self-Evaluating Detection and Closed-Loop Mitigation: Seamless transition from verification to text refinement
Once fine-tuned, the LVLM acts as its own autonomous verifier. At inference time, binary classification applies a threshold \(\mu\): tokens with \(S(o) \le \mu\) are classified as hallucinations, while those above are verified as grounded. Beyond post-hoc detection, TruthLens establishes an efficient detection-to-mitigation feedback loop. Operating at a 95% True Positive Rate (TPR) operating threshold, flagged hallucinated tokens are collected and passed back to the base model alongside the original draft caption, prompting the model to eliminate ungrounded entities while preserving factual observations. This plug-and-play revision reduces both sentence-level and instance-level hallucinations substantially without retraining the generative backbone.
Loss & Training¶
Parameter-efficient fine-tuning is conducted via LoRA on visual projection and language model weights, with a learning rate of \(5 \times 10^{-5}\) and batch size of 16 across 4 NVIDIA A800 GPUs. For each training image, 8 candidate captions are sampled. Object mentions are parsed, lemmatized, and matched against ground-truth category synsets to assign binary supervision labels \(\hat{r} \in \{0, 1\}\). The class-reweighted MSE loss is computed on the initial sub-token of each detected object, while the KL divergence constraint operates over full token sequences.
Key Experimental Results¶
Main Results¶
Evaluation spans MSCOCO (in-domain 80 categories) and Objects365 (365 categories, testing zero-shot generalization), benchmarking against probability-based, attention-based, and representation-based baselines across five open-source LVLMs.
| Dataset | Model | Metric | TruthLens (Ours) | Prev. SOTA (GLSIM / SVAR) | Gain |
|---|---|---|---|---|---|
| MSCOCO | LLaVA-1.5-7B | AUROC / AUPR | 90.57 / 97.25 | 83.70 / 94.20 (GLSIM) | +6.87% / +3.05% |
| MSCOCO | LLaVA-1.5-13B | AUROC / AUPR | 90.99 / 97.59 | 84.97 / 95.15 (GLSIM) | +6.02% / +2.44% |
| MSCOCO | LLaVA-NeXT-13B | AUROC / AUPR | 93.07 / 98.91 | 78.37 / 95.35 (GLSIM) | +14.70% / +3.56% |
| MSCOCO | Qwen2.5-VL-7B | AUROC / AUPR | 91.46 / 98.59 | 74.05 / 94.40 (GLSIM) | +17.41% / +4.19% |
| MSCOCO | LLaVA-OneVision-1.5-8B | AUROC / AUPR | 93.04 / 98.93 | 76.94 / 95.42 (GLSIM) | +16.10% / +3.51% |
| Objects365 | LLaVA-1.5-7B | AUROC / AUPR | 77.89 / 84.56 | 72.60 / 74.60 (GLSIM) | +5.29% / +9.96% |
| Objects365 | LLaVA-NeXT-13B | AUROC / AUPR | 80.15 / 87.58 | 70.33 / 79.02 (SVAR) | +9.82% / +8.56% |
| Objects365 | Qwen2.5-VL-7B | AUROC / AUPR | 71.54 / 84.56 | 65.11 / 80.75 (SVAR) | +6.43% / +3.81% |
| Objects365 | LLaVA-OneVision-1.5-8B | AUROC / AUPR | 82.14 / 88.31 | 72.59 / 80.88 (SVAR) | +9.55% / +7.43% |
Note: Across all models, original captioning recall is fully maintained after fine-tuning (e.g., LLaVA-1.5-7B transitions from 76.10% to 77.34%, Qwen2.5-VL-7B from 66.51% to 66.80%), confirming no degradation in visual coverage.
Ablation Study¶
Ablation experiments on MSCOCO validate the necessity of each optimization component in TruthLens.
| Config | LLaVA-1.5-7B AUROC | LLaVA-1.5-7B AUPR | Qwen2.5-VL-7B AUROC | Qwen2.5-VL-7B AUPR | Note |
|---|---|---|---|---|---|
| Raw Model (w/o fine-tuning) | 34.80 | 70.71 | 35.04 | 80.98 | Raw special token distributions lack calibrated signal |
| MSE Reward + KL Constraint | 89.02 | 96.36 | 85.69 | 97.49 | Activating truthfulness readout yields massive boost |
| Full Model (+ Class Reweighting) | 90.57 | 97.25 | 91.46 | 98.59 | Balances sparse positive/negative gradients (+5.77% on Qwen) |
| w/o KL Constraint | Collapsed | Collapsed | Collapsed | Collapsed | Severe distribution collapse into repetitive strings |
Token choice ablation on Qwen2.5-VL reveals minor differences across special tokens: <|image_pad|> (91.46), <|vision_start|> (91.54), and <|vision_end|> (91.20) all yield comparable AUROC, confirming that the framework's effectiveness stems from feature-space alignment rather than specific token artifacts.
Key Findings¶
- Component Contributions: The MSE regression objective provides the primary driving force for exposing internal confidence (elevating AUROC from ~35% to >85%), while dynamic class reweighting prevents model bias on imbalanced data, offering a 5.77% AUROC boost on Qwen2.5-VL. The KL regularizer is indispensable to prevent catastrophic generative collapse.
- Out-of-Domain Generalization: Although supervised on only 80 MSCOCO categories, TruthLens transfers smoothly to the 365 diverse categories of Objects365, outperforming all baselines by 5%–10% AUROC. This demonstrates that the model learns general grounding confidence rather than category-specific memorization.
- Layer Emergence and Projection Bottleneck: Layer-wise LDA reveals that separability steadily builds across intermediate layers (normalized depth 0.3–0.7) during cross-modal fusion. Untuned LM heads fail to expose this signal because the discriminative vector aligns with less than 2.5% of the readout projection subspace.
- Detection-to-Mitigation Efficacy: Using TruthLens scores in a self-revision prompt on LLaVA-1.5-7B decreases instance-level hallucination (CHAIRi) from 14.81% to 12.23% and sentence-level hallucination (CHAIRs) from 49.40% to 43.78%, with negligible impact on object recall (76.74% to 76.31%).
Highlights & Insights¶
- Repurposing Inactive Special Tokens: Exploiting rarely-sampled special tokens as calibrated truthfulness indicators circumvents standard vocabulary disruption, achieving native token-level scoring with zero additional forward passes or memory footprint.
- Insightful Representation-Projection Diagnostics: Through layer-wise LDA and output-null projection analysis, the paper pinpoints why existing models "know more than they show," elucidating the mathematical foundation behind LM head miscalibration.
- Closed-Loop Self-Mitigation: The high-precision token scores directly empower models to audit and rewrite their own hallucinations without relying on specialized external re-writers.
Limitations & Future Work¶
- Supervision Dependency: Fine-tuning still requires ground-truth object category annotations (e.g., from MSCOCO) to construct positive and negative labels. Exploring self-supervised or consistency-driven pseudo-labeling remains an open direction.
- Two-Stage Correction vs. Real-Time Decoding: The current mitigation relies on post-generation prompt-based rewriting. Future work could integrate truthfulness scores directly into autoregressive decoding (e.g., via confidence-steered sampling or contrastive decoding) to suppress hallucinations during generation.
- Compound Entity Granularity: While evaluating the first sub-token is empirically sufficient, fine-grained multi-token aggregation strategies could be further tailored for domain-specific long terminology.
Related Work & Insights¶
- vs External Judge Models (GAIVE, HaLEM): Prior methods query external LLMs/LVLMs, which inflates system latency and deployment cost; TruthLens leverages the model's own parameters for zero-overhead internal self-evaluation.
- vs Attention/Feature Baselines (SVAR, GLSIM): Previous internal methods measure visual attention weights or multimodal embedding distances; TruthLens diagnoses and resolves the LM head projection mismatch, outperforming GLSIM by over 16% AUROC on advanced architectures.
- vs Sequence-Level Self-Reward (LASER): While LASER evaluates complete solutions for math/reasoning tasks, TruthLens extends the formulation to token-level multimodal grounding, addressing fine-grained object hallucinations.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ [Identifies the representation-projection mismatch at the LM head and repurposes inactive special tokens for zero-overhead truthfulness scoring]
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Evaluates five LVLM architectures across in-domain, out-of-domain, and attribute hallucination benchmarks, accompanied by deep mathematical diagnostic analyses]
- Writing Quality: ⭐⭐⭐⭐⭐ [Cohesive problem motivation, clean narrative progression, rigorous mathematical formulations, and clear visual diagrams]
- Value: ⭐⭐⭐⭐⭐ [Delivers immediate practical utility for real-world deployments by requiring no extra parameters or inference latency]