Skip to content

InterPol: De-anonymizing LM Arena via Interpolated Preference Learning

Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning / Model Evaluation & Safety
Keywords: LLM Evaluation, Anonymous Leaderboard, Model De-anonymization, Model Interpolation, Preference Learning

TL;DR

This paper presents INTERPOL, a model-driven LLM identification framework that extracts target-specific response signatures from interpolated preference data by synthesizing controllable hard negatives via weight interpolation and training a detector with adaptive triplet preference learning, substantially outperforming statistical baselines and demonstrating that strategic voting can manipulate leaderboard rankings with up to 69.9% fewer interactions.

Background & Motivation

The rapid advancement of Large Language Models (LLMs) has rendered traditional static benchmarks such as MMLU and HumanEval increasingly vulnerable to evaluation data contamination and poorly aligned with open-ended, real-world user interactions. In response, preference-based crowdsourced platforms—most prominently Chatbot Arena (LM Arena)—have emerged as the de facto gold standard for frontier model evaluation. The foundational operational pillar of this paradigm is strict response anonymity: human judges evaluate double-blind pairwise responses to identical prompts purely based on intrinsic output quality, and these judgments are subsequently aggregated via Bradley-Terry modeling into public Elo rating leaderboards.

While initial investigations have suggested that surface-level text statistics (e.g., Bag-of-Words, TF-IDF, or length distributions) can inadvertently leak model provenance, such superficial signals become brittle when distinguishing stylistically aligned models, close competitors with narrow Elo margins, or variants within the same architectural family (such as GPT-4o versus GPT-4o-mini). Furthermore, modern leaderboard aggregation protocols increasingly deploy statistical sentiment- and style-debiasing filters that down-weight suspicious, feature-based voting patterns. This raises an urgent and consequential security question: after discounting shallow statistical heuristics, do anonymous responses continue to harbor deep, model-specific behavioral fingerprints that can be systematically exploited by advanced machine learning techniques to de-anonymize arena duels?

To address this challenge, the authors reframe leaderboard de-anonymization as a supervised preference learning task, where a target-specific detector is trained to assign systematically higher scalar preference scores to a target LLM's responses over competing outputs without requiring privileged platform telemetry. However, naively collecting negative samples from unrelated open models yields trivial discrimination tasks with sparse gradient signals. Core idea: Synthesize continuously controllable hard negatives by interpolating a target-mimicking copy model with its base backbone in weight space, and optimize a target-specific detector through boundary-aware adaptive triplet preference learning across progressive difficulty curricula.

Method

Overall Architecture

The INTERPOL architecture comprises two primary decoupled stages: (1) controllable hard-negative synthesis combining off-policy distillation and weight-space parameter interpolation, and (2) boundary-aware adaptive curriculum training under a triplet preference learning objective. Starting from an offline-collected query set, the framework first fine-tunes a lightweight open-source base model on black-box target model API outputs to obtain a target-mimicking copy model. Next, it linearly interpolates parameters between the base model and the copy model to synthesize intermediate-difficulty negative responses with explicitly dialed similarity. Finally, these synthetic responses are combined with the target output and the semantically closest non-target response into training triplets, optimizing a reward-model-style detector via a multi-stage curriculum that progressively escalates discrimination difficulty.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Query q"] --> B["Off-Policy Knowledge Distillation<br/>Supervised fine-tuning of base model to mimic target"]
    B --> C["Weight-Space Model Interpolation<br/>Linear parameter mixing with controllable factor α"]
    C --> D["Hard Negative Retrieval via Semantic Similarity<br/>Sentence encoder finds nearest real competitor output"]
    D --> E["Boundary-Aware Adaptive Triplet Learning<br/>Dynamic switching between pairwise and triplet objectives"]
    E --> F["Iterative Curriculum Learning Schedule<br/>Progressive difficulty scaling from α1=0.5 to α2=0.75"]
    F --> G["Target Model Detector fθ<br/>Assigns robust scalar preference scores"]

Key Designs

1. Off-Policy Knowledge Distillation: Eliminating Internal Model Dependencies via Black-Box Mimicking

Because realistic adversaries and external auditors only have black-box API access to proprietary target models \(M^*\) without access to hidden states or model weights, directly learning continuous decision boundaries around target generations is inherently challenging. To construct a controllable generator, INTERPOL first initializes a compact open-source base language model (Phi-4-mini-instruct) and performs standard supervised fine-tuning (SFT) over collected query–response pairs: $\(\hat{M}^* = \arg\min_\phi \frac{1}{|D|} \sum_{(q_i, y_i^*) \in D} \mathcal{L}_{\text{CE}}(M_\phi(q_i), y_i^*)\)$ The resulting copy model parameter set \(\hat{\phi}_{\text{copy}}\) reliably captures the target model's signature lexical patterns, reasoning structure, and stylistic conventions, providing a functional proxy for generating adversary-controlled target-like outputs.

2. Weight-Space Model Interpolation: Overcoming Discrete Negative Boundaries with Controllable Hard Negatives

Conventional contrastive or preference learning setups rely on responses from external non-target models as negative samples, which often differ radically in capability, length, and domain distribution, providing uninformative, easily separable supervisory signals. Capitalizing on the smooth functional properties of weight-space parameter arithmetic, INTERPOL linearly interpolates between the initial un-finetuned base model parameters \(\phi_{\text{init}}\) and the target-mimicking copy parameters \(\hat{\phi}_{\text{copy}}\): $\(\tilde{\phi}_\alpha = (1 - \alpha) \phi_{\text{init}} + \alpha \hat{\phi}_{\text{copy}}, \quad \alpha \in [0, 1]\)$ As the interpolation ratio \(\alpha\) monotonically increases from 0 toward 1, the synthesized response \(\tilde{y}_i^\alpha = \tilde{M}_\alpha(q_i)\) continuously transitions from the base initializer toward the target copy, exhibiting monotonically decreasing target-response perplexity (PPL) and increasing semantic embedding similarity. This allows the framework to synthesize controllable intermediate negatives satisfying the strict target-preference condition \(0 < \Delta(q, \tilde{y}^\alpha) < \Delta(q, y^o)\) at zero marginal API query cost.

3. Hard Negative Retrieval via Semantic Similarity: Anchoring Real-World Decision Boundaries

To prevent the detector from overfitting solely to synthetic artifacts of the interpolated model, INTERPOL grounds the learning objective in real-world candidate outputs. For each query, given the set of outputs from all non-target candidate LLMs \(Y_i = \{y_i^k \mid y_i^k \sim M_k(q_i), \forall M_k \in \mathcal{M} \setminus \{M^*\}\}\), the framework computes high-dimensional semantic representations using an off-the-shelf sentence transformer (all-MiniLM-L6-v2) and selects the non-target response with the highest cosine similarity to the target response: $\(y_i^o = \arg\max_{y_i^k \in Y_i} \text{sim}(e(y_i^*), e(y_i^k))\)$ Pairing this hardest real competitor \(y_i^o\) with the synthesized intermediate response \(\tilde{y}_i^\alpha\) forms an informative response triplet \(\mathcal{T}_\alpha = \{(q_i, y_i^*, \tilde{y}_i^\alpha, y_i^o)\}_{i=1}^N\) spanning both empirical and synthetic semantic boundaries.

4. Boundary-Aware Adaptive Triplet Learning: Stabilizing Optimization via Margin-Gated Objectives

Directly enforcing full triplet ranking constraints on queries where the detector cannot even distinguish coarse target-versus-non-target differences can destabilize gradient dynamics. INTERPOL introduces a sample-wise discrimination margin \(m_i = f_\theta(q_i, y_i^*) - f_\theta(q_i, y_i^o)\). When \(m_i < 0\), indicating that the detector currently misidentifies the target relative to the empirical hard negative, the training objective automatically falls back to standard binary pairwise Bradley-Terry cross-entropy \(\mathcal{L}_{\text{pref}}(q_i, y_i^*, y_i^o)\). Only when \(m_i \ge 0\) does the model apply the full composite triplet loss \(\mathcal{L}_{\text{trp}}^i\): $\(\mathcal{L}_{\text{trp}}^i = \lambda_1 \mathcal{L}_{\text{pref}}(q_i, y_i^*, \tilde{y}_i^\alpha) + \lambda_2 \mathcal{L}_{\text{pref}}(q_i, \tilde{y}_i^\alpha, y_i^o) + \lambda_3 \mathcal{L}_{\text{pref}}(q_i, y_i^*, y_i^o)\)$ With hyperparameter weights set to \((\lambda_1, \lambda_2, \lambda_3) = (0.3, 0.3, 1.0)\), this adaptive gating ensures the detector master the primary ranking boundary before enforcing fine-grained separation against the synthetic intermediate negative.

5. Iterative Curriculum Learning Schedule: Multi-Stage Escalation of Identification Hardness

To guide the detector toward robust, non-trivial attribution features without training instability, INTERPOL organizes optimization across a two-stage iterative curriculum. Stage 1 trains the detector backbone (GRM-Llama-3.2-3B) for 5 epochs using easier synthetic negatives constructed at \(\alpha_1 = 0.5\) with an initial learning rate of \(1 \times 10^{-5}\), establishing clear separation in score distributions. Stage 2 initializes from the optimal checkpoint of Stage 1 and continues for 5 additional epochs using substantially harder negatives generated at \(\alpha_2 = 0.75\) with a halved learning rate of \(5 \times 10^{-6}\). This sequential refinement prevents detector collapse on subtle signatures, forcing it to look past shallow lexical distributions to cohesive discourse-level patterns.

Loss & Training

  • Optimization Objective: For triplet dataset \(\mathcal{T}_\alpha\), minimize \(\mathcal{L}_{\text{fin}} = \frac{1}{|\mathcal{T}_\alpha|} \sum_{i \in \mathcal{T}_\alpha} \mathcal{L}_{\text{final}}^i\), where \(\mathcal{L}_{\text{final}}^i = \mathcal{L}_{\text{trp}}^i\) if \(m_i \ge 0\), else \(\mathcal{L}_{\text{pref}}(q_i, y_i^*, y_i^o)\). The pairwise preference loss follows the Bradley-Terry formulation: \(\mathcal{L}_{\text{pref}}(q, y_w, y_l) = -\log \sigma(f_\theta(q, y_w) - f_\theta(q, y_l))\).
  • Hardware & Hyperparameters: Detector backbone is GRM-Llama-3.2-3B; copy backbone is Phi-4-mini-instruct. Training uses AdamW (\(\beta_1=0.9, \beta_2=0.999\), weight decay 0.01), cosine learning rate schedule with 100 warmup steps, effective batch size 8 (batch size 2 with gradient accumulation steps 4), FP16 mixed precision, and DeepSpeed ZeRO-2 on a single NVIDIA RTX 6000 Ada (96GB).

Key Experimental Results

Main Results

The framework is evaluated across three primary frontier target models (GPT-4o, Gemini-2.5-Pro, and Claude-Sonnet-4) against 11 non-target candidate LLMs spanning six major model families, utilizing both the Chatbot Arena human preference dataset and the Alpaca instruction dataset. Each evaluation duel consists of one target response and one competing non-target response to the same prompt.

Target Model Method Arena Acc. Arena AUROC Alpaca Acc. Alpaca AUROC
GPT-4o Length-Word 0.556 0.611 0.493 0.551
Length-Char 0.564 0.610 0.495 0.552
TF-IDF 0.697 0.766 0.655 0.704
Bag-of-Words (BoW) 0.709 0.772 0.631 0.692
Neural-Based (DeBERTa-v3) 0.898 0.888 0.726 0.767
LLM-as-a-Judge (Gemma-2-27B) 0.822 — 0.679 —
INTERPOL (Ours) 0.908 0.920 0.887 0.875
Gemini-Pro Length-Word 0.792 0.830 0.618 0.647
TF-IDF 0.844 0.926 0.780 0.851
BoW 0.887 0.948 0.798 0.911
Neural-Based 0.908 0.940 0.828 0.894
LLM-as-a-Judge 0.958 — 0.620 —
INTERPOL (Ours) 1.000 0.979 0.946 0.967
Sonnet-4 Length-Word 0.639 0.621 0.525 0.529
TF-IDF 0.810 0.902 0.763 0.834
BoW 0.851 0.929 0.767 0.861
Neural-Based 0.883 0.908 0.825 0.899
LLM-as-a-Judge 0.902 — 0.631 —
INTERPOL (Ours) 0.946 0.976 0.982 0.954

Ablation Study

A component ablation on negative sampling strategy (N), triplet loss (T), adaptive margin gating (A), and iterative multi-stage curriculum (I) evaluates the contribution of each design module, averaged across all three target LLMs.

Negative Type (N) Triplet Loss (T) Iterative (I) Adaptive (A) Avg. Acc. Avg. AUROC Configuration Note
Hard — — — 0.863 0.882 Baseline pairwise cross-entropy
Hard ✓ — — 0.914 0.931 Single-stage triplet preference loss
Easy ✓ — ✓ 0.887 0.896 Adaptive triplet using easy negatives
Random ✓ — ✓ 0.928 0.920 Adaptive triplet using random negatives
Hard ✓ — ✓ 0.935 0.951 Single-stage full adaptive triplet
Hard (Ours Full) ✓ ✓ ✓ 0.951 0.958 Two-stage iterative curriculum (α=0.5 → 0.75)

To quantify the downstream manipulation risk under reconstructed Bradley-Terry / Elo dynamics on the 140k LM Arena battle dataset, the table below records the total interactions (and actual adversarial votes in parentheses) required to elevate target models to Rank 1 under the enemy-list promotion policy:

Identification Detector o3-2025-04-16 (Rank 5 → 1) deepseek-v3-0324 (Rank 15 → 1) llama-4-scout-17b (Rank 45 → 1)
Length-Char 26,400 (2,188) 76,900 (6,425) 109,100 (9,142)
TF-IDF 21,800 (2,468) 72,200 (8,131) 85,300 (9,686)
BoW 22,200 (2,631) 69,500 (8,188) 85,200 (10,082)
Neural-Based 22,400 (2,911) 62,000 (8,011) 70,600 (9,124)
LLM-Judge 22,500 (2,916) 62,600 (8,073) 70,600 (9,100)
INTERPOL (Ours) 22,200 (3,066) 59,600 (8,137) 64,700 (8,871)

Key Findings

  • Resilience Against Within-Family Confounding: While shallow lexical baselines experience catastrophic degradation when differentiating closely related models (e.g., TF-IDF accuracy collapses to 0.645 on GPT-4o vs. GPT-4o-mini and Neural-Based drops to 0.590), INTERPOL maintains 0.740 and 0.915 respectively, proving that it captures deep behavioral invariants beyond shared architecture and pre-training corpora.
  • Robustness in Close-Elo Regimes: In a specialized evaluation pool restricted to tightly clustered high-ranked competitors on LM Arena (Grok-4-Fast, DeepSeek-V3.1, GLM-4.6, Kimi-K2, Qwen3-Max; Elo 1416–1428), INTERPOL achieves 0.949 accuracy on GPT-4o and 0.989 on Sonnet-4, confirming that performance gains are not driven by broad capability disparity.
  • Intrinsic Response Attribution without Query Conditioning: In an ablation removing prompt queries entirely at test time, the response-only detector achieves 0.989 accuracy / 0.991 AUROC on Gemini-Pro and 0.981 accuracy / 0.983 AUROC on Sonnet-4, demonstrating that model identity signatures reside intrinsically within generated response text rather than prompt-matching artifacts.
  • Drastic Reduction in Attack Budget via Competitor-Aware Voting: In simulated arena rank promotion, pairing target upvoting with downvoting identified top-3 competitors reduces the required interaction budget for promoting o3 from Rank 5 to Rank 1 from 45,800 to 22,200 (a 51.5% reduction), with reductions reaching 88.1%–90.0% for Rank 2 and Rank 3 promotions.

Highlights & Insights

  • Interpolation-Induced Continuous Difficulty Control: By leveraging weight-space arithmetic rather than discrete data scraping, INTERPOL synthesizes smooth, intermediate difficulty spectra at zero incremental API cost, transforming a brittle classification task into a well-behaved continuous preference optimization problem.
  • Interpretability via Erasure Exposes Discourse Patterns: Erasure-based local attribution demonstrates that deleting the top-10 words prioritized by INTERPOL causes a margin drop of 11.567 (compared to 6.452 for TF-IDF and 6.195 for random deletion); qualitative analysis confirms these words are not isolated anomaly tokens, but constituent components of coherent explanatory spans characteristic of frontier reasoning models.
  • Broad Transferability Across Query Distributions and Decoding Settings: Detectors trained on Alpaca generalize seamlessly to Arena duels (AUROC 0.838–0.976), and altering target test-time decoding from greedy to stochastic sampling (temperature 0.7, top-p 0.9) produces a negligible shift of only +0.26% Accuracy and -0.29% AUROC.

Limitations & Future Work

  • Text-Only, Single-Turn Scope: The current investigation focuses on English, single-turn textual interactions; future work should explore multi-turn conversations, code execution artifacts, and multimodal (vision-language) evaluation arenas where identity signatures may manifest differently.
  • Offline Simulation vs. Deployed Platform Defenses: Ranking manipulation experiments rely on reconstructed Bradley-Terry dynamics under uniform random pairing; live platforms feature non-stationary user traffic, dynamic matchmaking heuristics, and proprietary fraud detection mechanisms that could increase the practical execution cost.
  • Capacity Scaling of Copy Models: The framework currently employs a compact 4B-parameter copy model (Phi-4-mini / Qwen3-4B); as frontier target LLMs advance toward multi-modal agentic architectures, copy models may require commensurate scaling to synthesize faithful intermediate representations.
  • vs. Lexical & Statistical Attribution Baselines (Huang et al., 2025; Salton & Buckley, 1988): Traditional approaches rely on n-gram frequencies, TF-IDF, or length distributions that fail under stylistic harmonization or within-family model comparisons, and are easily mitigated by length- and sentiment-controlled leaderboard aggregation. INTERPOL achieves 32.3% higher accuracy by learning non-linear, semantic preference boundaries.
  • vs. Multi-Class Neural Attribution Classifiers (Tay et al., 2020; Antoun et al., 2024): Direct \(K\)-class neural classifiers overfit to closed candidate sets and exhibit high variance under unseen model distribution shifts. INTERPOL reformulates attribution as target-centric pairwise/triplet preference scoring, demonstrating superior out-of-distribution transfer to unseen candidate pools.
  • vs. Active Watermarking and Fingerprinting (Kirchenbauer et al., 2023; Xu et al., 2024): Active techniques require white-box decoding access to embed imperceptible statistical watermarks into generated logits. INTERPOL demonstrates that standard, unwatermarked model responses already exhibit passive, indelible behavioral fingerprints that can be decoded without provider consent.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Pioneering framework that reframes leaderboard de-anonymization via weight-space interpolation and adaptive preference learning.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluation across 3 frontier targets, 12 comparison models, close-Elo pools, within-family audits, erasure attribution, and downstream manipulation simulations.
  • Writing Quality: ⭐⭐⭐⭐⭐ Rigorous methodology, crisp technical narrative, and seamless theoretical-to-empirical alignment.
  • Value: ⭐⭐⭐⭐⭐ Exposes a profound vulnerability at the heart of crowdsourced LLM evaluation, providing critical insights for designing manipulation-resistant evaluation platforms.