Large Language Models Systematically Favor Popular Options: Evidence, Mechanisms, and Mitigation¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/DataScienceUIBK/PopMCQ
Area: NLP Understanding
Keywords: popularity bias, multiple-choice evaluation, confidence calibration, inference-time debiasing, PopMCQ
TL;DR¶
The paper reveals that LLMs systematically favor popular but incorrect options in multiple-choice questions (popularity bias), introduces PopMCQ across six controlled strategies, and proposes PopDebiasโa lightweight, inference-time prior-removal method that boosts accuracy by up to 31.9 percentage points without retraining.
Background & Motivation¶
Multiple-choice questions (MCQs) serve as the foundation of modern LLM capability evaluation (such as MMLU, ARC, AGIEval, and C-Eval), relying on the fundamental assumption that a model's relative probability over candidate options faithfully mirrors its parametric knowledge and factual reasoning capability. Empirical studies have demonstrated that language models exhibit severe long-tail distribution characteristics in memorization: entities frequently encountered during pretraining are recalled with substantially higher accuracy than lesser-known entities. However, when this familiarity prior encounters multiple-choice settings, a fundamental and overlooked conflict arises: when popular but factually incorrect entities appear as distractors, does the model's decision mechanism remain faithful to factual truth, or is it hijacked by the superficial popularity of candidate entities?
Prior research on MCQ evaluation robustness predominantly focuses on surface-level, non-semantic artifacts such as option position or token order bias, while the causal impact of option entity popularity remains largely unexamined. In standard benchmarks, distractor popularity is left uncontrolled and sampled arbitrarily, preventing researchers from disentangling whether model failures stem from an actual lack of knowledge (knowledge blindness) or active temptation by well-known distractors at decision time. Compounding the issue, model confidence remains stubbornly high when selecting famous incorrect entities, generating a severe confidenceโaccuracy miscalibration that threatens the reliability of high-stakes benchmark rankings.
To resolve this confounding tension, evaluation protocols must explicitly isolate option popularity as an independent variable rather than conflating it with item difficulty. Core idea: isolate popularity bias as a decision-time attraction phenomenon via the PopMCQ benchmark across six controlled distractor strategies, and formulate popularity as a separable multiplicative prior that can be estimated and stripped at inference time via linear reweighting and adaptive confidence tempering without any model retraining.
Method¶
Overall Architecture¶
PopDebias is designed to decouple and remove the entity popularity prior from the uncalibrated prediction distribution of black-box or white-box language models. Given an MCQ instance with \(K\) options (where \(K=4\)), the base model outputs a raw probability distribution \(p = (p_1, \dots, p_K)\) over the option tokens, while each option is mapped to its normalized Wikipedia page-view scalar \(Pop(o_i) \in [0, 1]\). PopDebias first extracts a multi-dimensional feature vector capturing item-level popularity variance and model confidence to dynamically estimate the bias strength. It then applies a linearized inverse-prior weight to penalize hyper-popular entities, and computes a popularity pressure coefficient derived from the entropy of the option popularity profile to perform adaptive confidence tempering, producing the final debiased distribution \(\tilde{p}\).
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
In["MCQ Input and Base Prediction<br/>Option entities and probability p"] --> BiasEst["Popularity Feature Extraction and Bias Estimation<br/>Multi-dimensional features + lightweight regressor"]
BiasEst --> Reweight["Linearized Inverse-Prior Reweighting<br/>Suppress high-frequency entities and renormalize"]
Reweight --> Tempering["Adaptive Confidence Tempering<br/>Entropy-based pressure factor and dynamic scaling"]
Tempering --> Out["Debiased Prediction Distribution<br/>Calibrated probability p_tilde and final choice"]
Key Designs¶
1. Popularity Feature Extraction and Bias Estimation: Global penalty degrades helpful popularity -> Local item feature regression -> Adaptive bias strength determination In natural questions where the ground-truth answer happens to be the most prominent entity, a naive uniform penalty on popular entities would severely degrade legitimate correct answers. To address this risk, PopDebias extracts five label-free statistical features available strictly at inference time: $\(f = \left[ \text{Var}(Pop),\, \max_i Pop(o_i),\, Pop(o_{\hat{\imath}}),\, \text{rank}_{pop}(o_{\hat{\imath}}),\, \max_i p_i \right]\)$ where \(\hat{\imath} = \arg\max_i p_i\) represents the model's top prediction. Intuitively, popularity bias intensifies when the model exhibits high confidence on a common option while the option popularities have low variance. On a small 10% calibration split, a lightweight linear regressor \(f_\theta\) is trained to predict the popularity gap between the model's choice and the gold answer (\(Pop(o_{\hat{\imath}}) - Pop(o^\star)\)). At test time, the bias strength is computed label-free as \(b = \text{clip}(f_\theta(f), 0, b_{\max})\); if no calibration set is accessible, a bounded analytical heuristic can serve as a zero-shot proxy.
2. Linearized Inverse-Prior Reweighting: Multiplicative prior confounding truth -> Bounded inverse-weighting -> Smooth removal of popularity premium Observed model probability is modeled as a product of popularity-free belief and a popularity prior (\(p_i \propto \pi_i \cdot \phi_i\)). Directly dividing by the prior (\(\phi_i \propto p_i / \pi_i\)) is numerically unstable in small-option settings where tail options approach zero popularity. To avoid extreme division artifacts, PopDebias implements a bounded first-order linearization of \(\pi_i^{-1}\): $\(w_i = \text{clip}\left(1 - b \cdot \frac{Pop(o_i)}{\max_j Pop(o_j) + \varepsilon},\, 0,\, 1\right)\)$ This formulation normalizes the option popularity against the within-item maximum and applies the estimated bias strength \(b\) to temper over-activated entities. The reweighted intermediate distribution is renormalized as \(q_i = \frac{p_i w_i}{\sum_j p_j w_j}\), successfully curbing inflated distractor probabilities while avoiding numerical degeneracy.
3. Adaptive Confidence Tempering: Celebrity options cause miscalibration -> Entropy and confidence coupling -> Global probability recalibration When models err toward famous distractors, their output confidence remains persistently high, causing huge Expected Calibration Error (ECE). To dynamically modulate correction intensity and prevent overcorrection when the model is already uncertain, PopDebias introduces an entropy-driven tempering factor. Let \(r_i \propto Pop(o_i)\) represent the normalized within-item popularity profile; the popularity pressure factor is defined as: $\(\psi = 1 - \frac{H(r)}{\log K}, \quad H(r) = -\sum_{i=1}^K r_i \log(r_i + \varepsilon)\)$ When one or two options heavily dominate in page views, the popularity entropy is low, driving \(\psi \to 1\) and signaling a sharp popularity trap; when option popularities are evenly matched, \(\psi \to 0\). Combining the base confidence \(c = \max_i p_i\), the estimated bias \(b\), and scalar \(\beta = 1\), the tempering coefficient is \(\gamma = \text{clip}(1 - \beta b c \psi, 0, 1)\), yielding the final smoothed distribution: $\(\tilde{p}_i = \gamma q_i + \frac{1 - \gamma}{K}\)$ When faced with severe popularity traps under high model overconfidence, \(\gamma\) decreases, flattening the overconfident distribution toward uniform to enable safe argmax flipping; when bias is absent, \(\gamma \approx 1\) and \(\tilde{p} \approx p\), faithfully preserving the original prediction.
Formal Definition and Evaluation Protocol¶
To rigorously analyze popularity bias, the paper operationalizes metrics and intervention strategies as follows: - External Entity Popularity Proxy: Each option entity is linked to Wikipedia to collect monthly page views from January 2015 to December 2024, Min-Max normalized to \([0, 1]\), and relativized within each item: \(Pop(o_i) \leftarrow Pop(o_i) / \max_j Pop(o_j)\). - Correlation (\(\rho\)): Spearman rank correlation between binary correctness (\(\mathbf{1}\{\hat{o} = o^\star\}\)) and chosen option popularity \(Pop(\hat{o})\). A strongly negative \(\rho\) signifies that selecting popular options reliably predicts being wrong. - High-Popularity Selection Rate (HPSR) and POPGAP: \(\text{HPSR} = \mathbb{E}\left[\mathbf{1}\{\text{rank}_{pop}(\hat{o}) \le \lfloor K/2 \rfloor\}\right]\) measures the fraction of selections in the top-50% popularity rank (chance = 50%). \(\text{POPGAP} = \mathbb{E}[Pop(\hat{o}) - Pop(o^\star)]\) measures the average popularity deviation from the gold answer. - PopMCQ Strategy Design: Keeping question text and ground truth fixed, 20 candidates across low/medium/high popularity tiers are generated via GPT-4o, and assembled deterministically into six variants: - S1 (Baseline): 3 distractors sampled uniformly at random with no popularity constraints; - S2 (Popular Trap): Top-3 most popular candidates with \(Pop > Pop(o^\star)\), maximizing adversarial popularity pressure; - S3 (Gradient): 3 distractors with strictly decreasing popularity, all exceeding \(Pop(o^\star)\); - S4 (Direct Contest): 1 highest-popularity distractor + 2 medium-popularity distractors; - S5 (Reverse Control): 3 distractors with \(Pop < Pop(o^\star)\), where truth is most popular (positive control); - S6 (None of the Above): Gold option is "None of the above" (\(Pop = 0\)) while distractors include celebrity entities, testing abstention under popularity pressure. - Open-Recall Diagnostic Protocol: To eliminate knowledge deficit as a confound, models first answer questions in free-form generation without options. S2 is then evaluated strictly on the subset of items the model already answered correctly in open recall ("known-answer" subset).
Key Experimental Results¶
Main Results¶
The authors evaluate 22 open-source LLMs (0.5B to 32B parameters across Llama, Qwen2.5, Gemma, Phi, DeepSeek, Falcon, and Mistral families) across four benchmark QA datasets (EntityQuestions, Natural Questions, MuSiQue, and WebQuestions, encompassing 12,210 items). The table below summarizes aggregate metrics across all 22 models and 4 datasets before and after applying PopDebias.
| Strategy | Accuracy Acc (%) [Before] |
Accuracy Acc (%) [After] |
Accuracy Gain \(\Delta\text{Acc}\) (pp) |
High-Pop Selection HPSR (%) [Before \(\to\) After] |
Correlation Spearman \(\rho\) [Before \(\to\) After] |
|---|---|---|---|---|---|
| S1 (Baseline) | 42.5 | 49.6 | +7.2 | 46.4 \(\to\) 32.5 | -0.288 \(\to\) -0.113 |
| S2 (Popular Trap) | 43.8 | 75.8 | +31.9 | 53.6 \(\to\) 21.6 | -0.879 \(\to\) -0.679 |
| S3 (Gradient) | 42.4 | 64.5 | +22.1 | 49.4 \(\to\) 23.9 | -0.754 \(\to\) -0.560 |
| S4 (Direct Contest) | 40.1 | 55.9 | +15.7 | 48.6 \(\to\) 28.7 | -0.570 \(\to\) -0.422 |
| S5 (Reverse Control) | 43.3 | 45.8 | +2.5 | 55.9 \(\to\) 48.8 | +0.193 \(\to\) +0.266 |
| S6 (None of the Above) | 14.5 | 25.9 | +11.4 | 56.3 \(\to\) 32.6 | -0.475 \(\to\) -0.641 |
Ablation Study & Mechanism Analysis¶
The table below presents the open-recall diagnostic on MuSiQue under S2 (Popular Trap), alongside a direct comparison between PopDebias and the position-debiasing baseline (PrideDebias).
| Model | Open-Recall Acc (%) |
Known Items \(N\) |
S2 Acc on Known (%) |
Errors Choosing More-Popular \(W \to MP\) (%) |
PopDebias Net Delta \(\Delta\text{Acc}\) / \(\Delta\text{HPSR}\) |
PrideDebias Net Delta \(\Delta\text{Acc}\) / \(\Delta\text{HPSR}\) |
|---|---|---|---|---|---|---|
| Llama-3-8B | 38.5 | 336 | 61.0 | 99.6 | +21.5 pp / -22.3 pp | -0.2 pp / +0.3 pp |
| Qwen2.5-7B | 35.8 | 313 | 56.5 | 99.8 | +34.1 pp / -35.1 pp | -1.5 pp / +1.1 pp |
| Gemma-2-9B | 26.2 | 229 | 33.5 | 100.0 | +20.7 pp / -21.2 pp | +0.9 pp / -1.1 pp |
| Phi-4 (14B) | 45.0 | 393 | 65.0 | 99.5 | +21.4 pp / -21.4 pp | +0.3 pp / -0.9 pp |
| DeepSeek-V2-Lite | 29.5 | 258 | 42.0 | 100.0 | +54.1 pp / -54.7 pp | -0.8 pp / +0.6 pp |
| Pooled (All 22 Models) | 34.9 | 1,529 | 52.0 | 99.8 | +31.9 pp / -32.0 pp | +0.2 pp / -0.1 pp |
Key Findings¶
- Decision-Time Vulnerability over Knowledge Deficit: In the open-recall diagnostic, even when evaluated strictly on items the models can answer correctly in open generation, S2 accuracy plummets to 52.0%, and 99.8% of these errors choose an option more popular than the truth. Furthermore, 29.1% of all S2 errors across 22 models are answered correctly under reverse control S5, proving that popularity directly alters decision selection.
- Familiarity-Induced Overconfidence: Partitioning selections by popularity shows that under low popularity (\(\le 0.3\)), confidence is 0.526 with accuracy at 0.464 (gap = 0.062). Under high popularity (\(> 0.7\)), confidence remains high at 0.524 while accuracy drops to 0.035, yielding an overconfidence gap of 0.489. PopDebias cuts Expected Calibration Error (ECE) by up to 67% (e.g., Llama-3-8B from 0.194 to 0.064).
- Scale and Alignment Do Not Solve Popularity Bias: Scaling Qwen2.5 from 0.5B to 32B improves accuracy from 37.4% to 57.8%, yet \(\rho\) remains strictly negative (\(-0.895 \to -0.845\)). RLHF-tuned Zephyr-7B shows identical bias (\(\rho = -0.87\)) to base Mistral-7B. Advanced reasoning models (o1-mini, QwQ-32B) exhibit attenuated but persistent bias under S2 (\(\rho = -0.41\) and \(-0.55\)).
- Prompting and Permutation Approaches Fail: Adding explicit instructions ("avoid choosing popular options") or Chain-of-Thought prompts yields negligible accuracy gains (0โ4 pp) with \(\rho\) remaining around -0.85. Position-permutation debiasing (PrideDebias) produces negligible gains (0.2 pp) because popularity bias is content-driven rather than order-driven.
Highlights & Insights¶
- Causal Decoupling of Familiarity in Evaluation: By combining long-term Wikipedia page views with synthetic controlled distractor sets (S1โS6), the paper successfully turns entity familiarity from an informal heuristic into an actionable, quantifiable causal variable.
- Inference-Time, Label-Free Prior Stripping: PopDebias operates as a plug-and-play post-processing step requiring only a single forward pass (\(O(K)\) overhead), successfully correcting predictions without costly fine-tuning or permutation combinatorial explosion.
- Crucial Caution for Benchmark Validity: Widespread benchmarks relying on MCQ formats (MMLU, AGIEval) inadvertently reward models that align with popular distractors, emphasizing the urgent need to control for option popularity when assessing real LLM capabilities.
Limitations & Future Work¶
- Dependency on External Wikipedia Proxy: Wikipedia page views serve as an external, model-agnostic proxy; however, they may fail to capture niche entity frequencies in non-English corpora, specialized codebases, or proprietary pretraining distributions.
- Restriction to White-Box Output Logits: PopDebias relies on access to token probabilities, making it difficult to directly deploy on commercial closed-source APIs that provide only generated text without probability outputs.
- Future Directions: Developing model-internal, zero-resource familiarity proxies (e.g., context-free prompt perplexity) and extending popularity debiasing to open-ended generative QA and dialogue systems.
Related Work & Insights¶
- vs PrideDebias (Zheng et al., 2024): While PrideDebias addresses positional bias via permutation averaging across \(K!\) permutations, PopDebias targets content-driven entity popularity in a single pass, delivering +21 to +34 pp accuracy gains where PrideDebias yields near-zero improvements.
- vs PopQA / Long-tail Knowledge (Mallen et al., 2023; Kandpal et al., 2023): Prior work showed that models struggle to recall tail facts during generation; this paper demonstrates the converse in discriminative settings: high-frequency head entities actively sabotage decision-making even when the underlying fact is known.
Rating¶
- Novelty: โญโญโญโญโญ [Systematically establishes popularity bias in MCQs, introduces the controlled PopMCQ benchmark, and proposes an elegant inference-time debiasing algorithm]
- Experimental Thoroughness: โญโญโญโญโญ [Evaluates 22 open-source models across 4 families, 4 datasets, closed-source frontier models, and comprehensive open-recall diagnostics]
- Writing Quality: โญโญโญโญโญ [Exceptionally clear argumentation, rigorous formalization, and seamless transition from empirical evidence to algorithmic resolution]
- Value: โญโญโญโญโญ [Provides an essential reality check and actionable mitigation tool for the entire NLP benchmark evaluation ecosystem]