Stability vs. Manipulability: Evaluating Robustness Under Post-Decision Interaction in LLM Judges

Conference: ACL2026
arXiv: 2606.05384
Code: None
Area: LLM Evaluation
Keywords: LLM Evaluation, Post-Decision Manipulability, Robustness, Conversational Influence

TL;DR

This paper reveals a critical vulnerability of LLM evaluators: while highly stable under repeated evaluation, they undergo significant reversals (49% flip rate, 74% under authoritative framing) when subjected to subsequent conversational challenges. This indicates that stability does not equate to robustness and that confidence levels fail to predict actual reliability.

Background & Motivation

Background: The LLM-as-judge paradigm has become mainstream for benchmarking, using LLMs to automatically compare and rank model outputs in systems like MT-Bench and AlpacaEval. These methods are widely adopted due to their low cost, scalability, and high alignment with human evaluation.

Limitations of Prior Work: Current evaluation pipelines implicitly assume that for a given input (a prompt and two candidate responses), the LLM's judgment should be stable and reproducible. However, this assumption collapses in interactive scenarios. If humans are allowed to continue the dialogue, challenge, or persuade the evaluator after the initial judgment, the results can be altered. This vulnerability is particularly problematic because LLMs are inherently conversational systems capable of explaining, reconsidering, and revising decisions—a flexibility beneficial for tasks but a loophole for evaluators.

Key Challenge: Stability (consistency under repetition or neutral reconsideration) and robustness (resistance to targeted conversational influence) are two independent dimensions. Existing methods only measure stability, completely ignoring post-decision interaction robustness as an independent failure mode.

Goal: To systematically measure the vulnerability of LLM evaluation under post-decision conversational interactions, distinguishing between the failure modes of "whether it changes" and "whether it changes toward a specific direction," and to propose quantitative metrics.

Key Insight: Utilizing a causal isolation design—fixing the two candidate responses being evaluated while only varying the post-decision interaction with the evaluator (repeated evaluation vs. neutral reconsideration vs. targeted questioning). This controlled design directly reveals the effects of conversational influence.

Core Idea: Defining two metrics, "Persuasive Susceptibility" (PS) and "Directed Steering" (DS), and constructing the ERS (Evaluation Robustness Score) to synthesize both, thereby simultaneously measuring the risks of "being easily changed" and "being steered in a certain direction."

Method

Overall Architecture

This paper uses causal isolation in controlled experiments to deconstruct the vulnerabilities of LLM evaluators: the two candidate responses under evaluation are fixed, and only the interaction mode the evaluator experiences after making the initial judgment is varied. This clearly observes the effect of "conversational influence" as a single variable. Each evaluation instance (100 pairs from MT-Bench and AlpacaEval) sequentially undergoes four conditions: first, a baseline evaluation \(z^{(0)}\), where the model outputs binary preference and confidence in \([0,100]\); next, a repeated evaluation \(B2\) with the exact same prompt to measure intrinsic stability; followed by a neutral, non-persuasive follow-up dialogue \(z^{(n)}\) to control for the influence of the "dialogue context itself"; and finally, targeted questioning \(z^{(c)}\) involving skepticism, authority, or evidence. These questions are applied via two protocols, and flips are recorded into diagnostic and composite metrics. Two GPT-4o series models (GPT-4o and GPT-4o-mini) completed a total of 1440 evaluations under deterministic decoding (temperature 0), with each pair sampled repeatedly across multiple conditions to enable independent measurement of stability and manipulability.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400, 'subGraphTitleMargin': {'top': 8, 'bottom': 16}}}}%%
flowchart TD
    A["100 Candidate Pairs (Fixed)"] --> B["Baseline Evaluation z^(0)<br/>Binary Preference + [0,100] Confidence"]
    B --> C["Repeated Evaluation B2<br/>Same Prompt, Measurement of Intrinsic Stability"]
    B --> D["Neutral Follow-up z^(n)<br/>Control for Contextual Effects"]
    B --> E["Targeted Challenges z^(c)<br/>Skepticism / Authority / Evidence"]
    subgraph DUAL["Dual Protocol Design"]
        direction TB
        F1["Anti-baseline Protocol<br/>Target = Opposite of Baseline"]
        F2["Balanced Target Audit<br/>Target and Baseline assigned independently"]
    end
    E --> DUAL
    DUAL --> G["Confidence & Reason Overlap Diagnosis<br/>True Correction vs. Post-hoc Rationalization"]
    G --> H["Evaluation Robustness Score ERS<br/>1 − (α·PS + β·DS)"]

Key Designs

1. Dual Protocol Design: Separating "Whether it changes" from "Which direction it changes"

Flip rates alone cannot distinguish if a model is being randomly overturned or deliberately manipulated. Thus, this paper designs two complementary protocols. In the Anti-baseline protocol, the questioning target is always the opposite of the baseline judgment; any change naturally aligns with the direction of the challenge. This serves as a stress test for how easily a judgment can be overturned. In the Balanced Target Audit, the questioning target is assigned independently of the baseline judgment. Here, Directed Steering \(DS_{\text{signed}} = \Pr(z^{(c)}=t) - \Pr(z^{(n)}=t)\) (where \(t\) is the target) truly measures "intentional steering" rather than "random flipping." Using both protocols allows the risks of "reversal susceptibility" and "intentional manipulation" to be separated.

2. Confidence and Reasoning Overlap Diagnosis: Distinguishing True Correction from Post-hoc Rationalization

When a judgment flips, it is crucial to determine if this is a legitimate correction based on new evidence or a post-hoc rationalization after being pressured. The authors record the overlap ratio of reasons before and after the change, finding an average of only 0.23, with overlap below 20% in 37–42% of cases. Meanwhile, authoritative challenges caused the most flips (74%) but the largest drop in confidence (−7.1), which contradicts the pattern of "finding new evidence should increase confidence." True correction should involve clear statements of "where I was wrong before," but low overlap combined with decreased confidence strongly suggests the model is weaving a new argument after changing its mind, rather than identifying a specific error.

3. Evaluation Robustness Score ERS: Penalizing Both Failure Modes with a Single Metric

Looking only at persuasive susceptibility does not characterize the full risk. The authors synthesize susceptibility and directed steering as \(ERS = 1 - (\alpha\, PS + \beta\, DS)\), where \(PS = \Pr(z^{(c)} \neq z^{(0)})\) is the persuasive susceptibility and \(DS = \max(0, DS_{\text{signed}})\) is the non-negative directed steering, using \(\alpha = \beta = 0.5\). Under the Anti-baseline protocol, \(ERS=0.51\), exposing high vulnerability; under the Balanced Audit, \(ERS=0.903\), indicating that this vulnerability primarily stems from "reversal susceptibility" rather than "directed control." This single metric penalizes both failure modes and can be adjusted based on scenario-specific weights.

This work is diagnostic; it does not design training targets. All evaluations use off-the-shelf GPT-4o models for deterministic inference. To ensure reliable causal inference, it employs template paraphrasing verification, McNemar tests, and Generalized Estimating Equations (GEE) significance tests clustered at the prompt level.

Key Experimental Results

Main Results

Property	Metric	Baseline/Control	Persuasion/Audit	Key Meaning
Stability	Flip Rate	1.0% (Repeated) / 0% (Neutral)	49% (Anti-baseline)	Extremely stable under repetition, major flips under challenge
Persuasive Susceptibility PS	Change Probability	0% (Neutral)	19.4% (Balanced)	Persuasive challenges trigger changes beyond baseline
Directed Steering DS	\(DS_{\text{signed}}\)	—	-0.018 (Balanced)	Anti-baseline high alignment (49%), but no net steering in balanced audit
Human Consistency	Agreement	67% (Baseline)	48% (Auth Anti-baseline) / 60.5% (Auth Balanced)	Drops 19.8 pp under Anti-baseline; drops 3.3 pp under Balanced
Harmful Flip Rate	Harmful Ratio	—	64%	Most flips move away from human preference
Rank Stability	Kendall \(\tau\)	1.00	0.50 (Anti-baseline) / 1.00 (Balanced Aggregated)	Ranks shift drastically (6/8 changed) under Anti-baseline
Authority Effect	Flip Rate	—	74% (Anti-baseline) / 31.7% (Balanced)	Authority framing is the strongest instability factor
Confidence	Mean	89%	82% (Post-flip Anti-baseline)	Significant flips occur even at high confidence; calibration fails
Reason Overlap	Overlap Ratio	—	0.23 (Anti-baseline)	37-42% of flipped reasons have <20% overlap; post-hoc rationalization
ERS Robustness	ERS	—	0.51 (Anti-baseline) / 0.903 (Balanced)	Vulnerability stems from reversals rather than directed control

Difficulty and Multi-step Ablation

Condition	Flip Rate	Interpretation
Baseline Agreement (83 pairs)	43%	Initial judgments where both sides agreed
Baseline Disagreement (17 pairs)	75%	Vulnerability 1.7x higher; evaluations needing robustness most are the most fragile
Skepticism Challenge	41%	First-round flip rate in multi-step sequence is 10.2%
Authoritative Challenge	74% / 31.7%	Strongest intervention; Anti-baseline significantly higher than Balanced
Evidence-based Challenge	~25%	Logical argument type; effect is relatively weaker
Multi-step Non-monotonicity	27/59 flipped at least once	Avg 1.89 steps to first flip; rises to 39% after Authority, falls to 18.6% after Evidence

Key Findings

• Stability \(\neq\) Robustness: This is the core insight. Repeated evaluation has a 1% flip rate and neutral reconsideration 0%, yet Anti-baseline challenges yield a 49% flip rate (74% for authority). Existing pipelines only measure stability and completely miss the dimension of conversational robustness.
• Authority is Most Dangerous: Among the three types (skepticism, authority, evidence), the authority framing ("Experts disagree") is the most effective. This suggests LLMs are more susceptible to social pressure than logical arguments.
• Confidence Calibration Failure: All evaluations show confidence between 70-100, yet the most flips occur under authority (74%) where confidence drops the most (-7.1). This indicates confidence is misaligned with actual knowledge—a deep alignment issue.
• Reason Regeneration vs. Correction: The reason overlap is only 0.23 during flips, with <20% overlap in 42% of cases. This strongly suggests that LLMs fabricate reasons post-hoc rather than identifying specific errors.
• Ambiguity Amplifies Fragility: Evaluators are 1.7x more fragile when there is initial baseline disagreement (75% vs 43%). The cases most requiring robust evaluation are the most vulnerable.
• Ranking Pollution: Over 6 out of 8 model rankings changed positions (\(\tau=0.50\)) under the Anti-baseline protocol, directly threatening benchmark validity.
• Mostly Harmful: 64% of flips moved away from human preference. Although the baseline was 68% correct, changes were mostly detrimental.

Highlights & Insights

• Discovery of Overlooked Failure Mode: Unlike prior work on prompt sensitivity or initial bias, this work is the first to systematically study post-decision manipulability. This is a practically relevant new dimension, as LLM evaluation is inherently conversational.
• Sophisticated Dual-Protocol Design: The Anti-baseline protocol tests "whether it changes," while the Balanced Audit isolates "true directed steering" by independently assigning targets. This causal isolation is rigorous and reproducible.
• Universality of ERS Metric: Synthesizing susceptibility and directed steering into a single formula allows future work to weight different challenge types and adapt to specific scenario tolerances.
• Implications of Calibration Failure: High confidence across the board fails to predict vulnerability. The drop in confidence under authoritative challenge highlights a disconnect between "surface consistency" and "actual robustness."
• Diagnostic Value of Low Reason Overlap: The low overlap (0.23) is strong evidence of post-hoc rationalization. This provides an actionable detection metric: if a new reason cannot coherently explain "what was wrong before," the change is untrustworthy.

Limitations & Future Work

Ours (Authors' limitations): - Only two models from the GPT-4o series were used; vulnerabilities may differ in other architectures or future versions. - Only covered MT-Bench and AlpacaEval; generalization to other domains/tasks/modalities is unknown. - Sample size of 100 pairs is relatively small, though 1440 samples were taken across conditions; large-scale validation is valuable. - Experiments assume a controlled interactive scenario; actual pipes may have safeguards (multi-evaluator aggregation, etc.) that mitigate these risks.

Self-identified limitations: - Low overlap might not always mean "fabrication"; human annotation is needed to judge the semantic significance of reason changes. - The Balanced Audit shows no net directed steering (\(DS=0\)), yet \(PS=19.4\%\) flips remain non-zero; the drivers of these non-steered flips need further diagnosis. - Multi-step challenge mechanisms are non-monotonic and not deeply explored; they may relate to context length, memory decay, or conflict resolution.

Future Directions: - Interaction Security: Limiting post-decision dialogue rounds, disabling authoritative framing, using fixed grading rubrics, and separating initial from revised judgments. - Multi-evaluator Aggregation: Testing whether ensembles of LLM evaluators can reduce individual vulnerability and cross-model correlation. - Mechanistic Diagnosis: Using causal intervention to diagnose internal causes—instruction tuning, RLHF, evaluation prompts, and dialogue handling. - Robust Evaluator Design: Training or prompting specialized evaluators to resist social pressure, calibrate confidence, and explicitly identify errors.

Related Work & Insights

• vs. Bias Research: Prior work found prompt sensitivity and stylistic biases. This work reveals an independent failure mode—conversational compliance. This is not traditional bias but a robustness issue in interaction.
• vs. Adversarial/Red-teaming: Red-teaming usually targets initial task outputs. This work focuses on the "changeable even after decision" threat scenario.
• vs. Self-refinement: While self-refinement is beneficial for reasoning tasks, it becomes a loophole in evaluation. This suggests "improvability" has different values depending on the role.
• Insight: Evaluation design requires a trade-off between "flexibility" and "trustworthiness." Revision shouldn't be banned but ensured for legitimacy through structured protocols and robust metrics.

Rating

• Novelty: ⭐⭐⭐⭐⭐ First systematic study of post-decision manipulability; the dual-protocol and ERS are universally valuable and challenge the LLM-as-judge paradigm.
• Experimental Thoroughness: ⭐⭐⭐⭐ Rigorous causal design and full statistical analysis; however, lacks large-scale validation across many different model families.
• Writing Quality: ⭐⭐⭐⭐⭐ Logical, precise definitions, intuitive charts, and strong alignment between problem statement and conclusions.
• Value: ⭐⭐⭐⭐⭐ Directly threatens the credibility of current evaluation pipelines, with serious implications for model rankings and benchmark validity.

Related Papers

• [ACL 2026] Fin-Bias: Comprehensive Evaluation for LLM Decision-Making under human bias in Finance Domain
• [ACL 2026] Aggregate vs. Personalized Judges in Business Idea Evaluation: Evidence from Expert Disagreement
• [NeurIPS 2025] On Evaluating LLM Alignment by Evaluating LLMs as Judges
• [ECCV 2024] R²-Bench: Benchmarking the Robustness of Referring Perception Models under Perturbations
• [ACL 2026] Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations