Language Model Maps for Prompt-Response Distributions via Log-Likelihood Vectors¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: GitHub
Area: LLM Reasoning
Keywords: Model maps, conditional log-likelihood vectors, prompt shift, additive compositionality, pointwise mutual information
TL;DR¶
This paper constructs language model maps for conditional prompt-response distributions via conditional log-likelihood and PMI vectors, proves that squared Euclidean distances in this space approximate conditional KL divergence, discovers that prompt modifications induce approximately additive shift vectors (Prompt Shift), and accurately predicts downstream task scores and composite prompt effects without autoregressive generation.
Background & Motivation¶
With the rapid proliferation and architectural diversification of open-source and proprietary large language models, establishing a rigorous, shared geometric space to compare behavioral similarities, lineage evolution, and fine-tuning dynamics has become a foundational challenge. Prior research pioneered the concept of language model maps by embedding models as points in Euclidean space based on unconditional log-likelihood vectors evaluated over context-free text chunks. While that representation established a direct theoretical equivalence between squared Euclidean distance and the Kullback-Leibler (KL) divergence between probability distributions, it was inherently restricted to unconditional probability distributions \(p(y)\), overlooking the interactive nature of modern LLMs.
In real-world deployment, large language models operate predominantly as conditional probability generators \(p(y|x)\), where downstream tasks such as multi-turn dialogue, question answering, and chain-of-thought reasoning are strictly governed by user prompts. Recent attempts to quantify distances across conditional generation models rely largely on computing text embedding distances over autoregressively sampled outputs (e.g., Artificial Hivemind). However, text-generation-based metrics suffer substantially from decoding randomness, sampling variance, and representation biases introduced by external sentence encoders. Concurrently, model-internal approaches based on weight matrices or activation profiles cannot readily establish a common metric space across heterogeneous model families, parameter dimensions, or Dense versus MoE architectures.
To overcome these fundamental hurdles, this paper investigates evaluating conditional probability models directly on a pre-defined set of fixed prompt-response pairs, eliminating the need for text generation while preserving the conditional dependency. Core idea: Represent each language model as a conditional log-likelihood vector \(\log p(y|x)\) and a Pointwise Mutual Information (PMI) vector \(\log p(y|x) - \log p(y)\) over fixed prompt-response pairs, establishing a model map where squared Euclidean distance approximates conditional KL divergence and uncovering that prompt modifications induce approximately additive geometric shift vectors (Prompt Shift).
Method¶
Overall Architecture¶
The proposed framework takes as input a pre-defined reference text set consisting of \(N\) prompt-response pairs \(\mathcal{D} = \{(x_s, y_s)\}_{s=1}^N\) and a collection of \(K\) autoregressive language models \(\{p_i\}_{i=1}^K\). The pipeline proceeds through four main stages: First, for each model, conditional log-likelihood scalars across all \(N\) pairs are computed via the model's native chat template to construct an \(N\)-dimensional conditional log-likelihood vector. Second, by subtracting an empty-prompt baseline proxy of the model marginal distribution, Pointwise Mutual Information (PMI) vectors are derived to filter out unconditional language priors. Third, applying global centering to the conditional log-likelihood vectors aligns them with a theoretical second-order Taylor approximation of the conditional KL divergence. Finally, the induced shared metric space is used to trace systematic vector displacements under prompt transformations (Prompt Shift), establish their additive compositionality, and train linear ridge regressors that predict downstream task performance as well as the effects of unobserved composite prompt transformations.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input: Evaluation Dataset D={(xs,ys)} and Models {pi}"] --> B["Conditional Log-Likelihood Vector Extraction<br/>lis = log pi(ys|xs) assembled into RN"]
B --> C["PMI Vector & Mutual Information Decomposition<br/>Δlis = log pi(ys|xs) - log pi(ys)"]
C --> D["Global Centering & Conditional KL Divergence Mapping<br/>||ξi - ξj||² ≈ 2N · KL(pi, pj)"]
D --> E["Prompt Shift & Additive Compositionality Modeling<br/>v_rep+cot,i ≈ v_cot,i + v_rep,i"]
E --> F["Downstream Task Prediction & Composite Prompt Synthesis<br/>RidgeCV on ξ to predict scores and delta shifts"]
Key Designs¶
1. Conditional Log-Likelihood Vector Extraction: Grounding conditional distributions without autoregressive generation
Unconditional log-likelihood vectors fail to measure model behavior in instruction-following scenarios because they omit the prompt context entirely. To ground conditional distributions \(p_i(y|x)\) without incurring the high variance and computational cost of autoregressive generation, this design feeds each pre-defined prompt-response pair \((x_s, y_s)\) into model \(p_i\) using its canonical chat template. The conditional log-likelihood assigned to the target response tokens \(y_s = (z_1, \dots, z_{|y_s|})\) given prompt \(x_s\) is accumulated: $\(\ell_{is} = \log p_i(y_s|x_s) = \sum_{t=1}^{|y_s|} \log p_i(z_t | x_s, z_{<t})\)$ Assembling these entries yields the conditional log-likelihood vector \(\boldsymbol{\ell}_i = (\ell_{i1}, \dots, \ell_{iN})^\top \in \mathbb{R}^N\). To remove arbitrary constant offsets caused by differing vocabulary sizes or overall log-probability calibrations across distinct model architectures, the global mean \(\bar{\ell}_i = \frac{1}{N} \sum_{s=1}^N \ell_{is}\) is subtracted from each component, producing the centered log-likelihood vector \(\boldsymbol{\xi}_i = \boldsymbol{\ell}_i - \bar{\ell}_i \mathbf{1}\).
2. PMI Vector & Mutual Information Decomposition: Isolating prompt-response dependencies from background language priors
A model's conditional probability \(\log p(y|x)\) conflates the unconditional plausibility of response \(y\) with the specific associative dependency between \(x\) and \(y\). When comparing models with disparate pre-training corpora, background lexical priors can dominate the representation and obscure fine-tuning distinctions. To eliminate this background prior, the method introduces the Pointwise Mutual Information (PMI) vector: $\(\Delta \ell_{is} = \log p_i(y_s|x_s) - \log p_i(y_s)\)$ While the theoretical definition of the marginal denominator requires the reference marginal \(p_i^{\text{ref}}(y_s) = \mathbb{E}_{x \sim p_0(x)}[p_i(y_s|x)]\), computing this expectation via shuffled-prompt Monte Carlo permutations is prohibitively expensive across hundreds of models. Instead, the authors use an empty-prompt evaluation \(p_i^{\text{model}}(y_s) \approx p_i(y_s | \emptyset)\) under the model's native chat template as a practical proxy. Subtracting \(\log p_i(y_s)\) yields a feature representation that isolates the prompt-response coupling, where the sample mean \(\Delta \bar{\ell}_i = \frac{1}{N}\sum_{s=1}^N \Delta \ell_{is}\) approximates the mutual information \(\text{MI}(p_i)\), sharpening clustering along instruction-tuning datasets such as Tulu3 versus Infinity-Instruct.
3. Global Centering & Conditional KL Divergence Mapping: Establishing information-theoretic equivalence for Euclidean distances
In arbitrary high-dimensional vector embeddings, geometric Euclidean distance frequently lacks an intrinsic physical interpretation. By conducting a local expansion around a reference data-generating distribution \(p_0(x, y) = p_0(y|x)p_0(x)\) where log-likelihood ratios \(r_{ij}(x, y) = \log p_i(y|x) - \log p_j(y|x) = O_p(\lambda)\) are small, the average conditional KL divergence decomposes into variance: $\(\mathrm{KL}(p_i, p_j) = \mathbb{E}_{x \sim p_0(x)} [\mathrm{KL}(p_i, p_j | x)] \approx \frac{1}{2} \mathrm{Var}_{(x, y) \sim p_0(x, y)} \left( \log \frac{p_i(y|x)}{p_j(y|x)} \right)\)$ Approximating this variance with the empirical sample variance across dataset \(\mathcal{D}\) directly yields: $\(\mathrm{KL}(p_i, p_j) \approx \frac{1}{2N} \|\boldsymbol{\xi}_i - \boldsymbol{\xi}_j\|_2^2\)$ The authors rigorously demonstrate in the appendix that uncentered, globally centered, and per-prompt centered quadratic forms differ only by \(O(\lambda^4)\) terms, meaning they share the identical leading-order \(O(\lambda^2)\) accuracy with an error of \(O(\lambda^3 + \lambda^2 / \sqrt{N})\). Consequently, global centering establishes a direct equivalence between Euclidean distances on the model map and directional conditional KL divergence.
4. Prompt Shift & Additive Compositionality Modeling: Capturing prompt edit algebra and synthesis in metric space
When an input prompt \(x\) undergoes a transformation \(T(x)\)—such as appending a chain-of-thought suffix \(T_{\text{cot}}(x) = x + c\) or repeating the prompt \(T_{\text{rep}}(x) = x + x\)—the model's conditional generation distribution moves across the map. This displacement is captured by the shift vector \(\mathbf{v}_{T, i} = \boldsymbol{\ell}_{T, i} - \boldsymbol{\ell}_{\text{base}, i}\). Crucially, the authors discover that composite prompt transformations \(T_{\text{rep+cot}}(x) = x + x + c\) exhibit remarkable additive compositionality: $\(\mathbf{v}_{\text{rep+cot}, i} \approx \mathbf{v}_{\text{cot}, i} + \mathbf{v}_{\text{rep}, i}\)$ This linear structure mirrors word vector arithmetic (e.g., King - Man + Woman ≈ Queen). It enables constructing synthetic log-likelihood feature vectors for unobserved composite prompts: \(\hat{\boldsymbol{\xi}}_{\text{rep+cot}, i} = \boldsymbol{\xi}_{\text{cot}, i} + \boldsymbol{\xi}_{\text{rep}, i} - \boldsymbol{\xi}_{\text{base}, i}\). Feeding these synthetic vectors into pre-trained linear regression models enables zero-inference prediction of performance changes under composite prompts.
Benchmark & Evaluation Protocol¶
The primary empirical analysis leverages \(K = 552\) open-weight language models (\(\le 70\text{B}\)) with \(\ge 1,000\) downloads on the Open LLM Leaderboard v2. Reference text datasets include \(N = 10,000\) prompt-response pairs from AllenAI's tulu-3-sft-mixture and \(N = 10,000\) pairs from BAAI's Infinity-Instruct-7M, with UltraChat (\(N = 5,000\)) serving as an out-of-domain robustness check. All forward passes are executed in bfloat16 with numerical stability clipping applied at the bottom 2nd percentile across the model-text matrix. For downstream task evaluation, lm-evaluation-harness is deployed in a zero-shot setting across five reasoning and QA benchmarks: ARC-Challenge, ARC-Easy, OpenBookQA, PIQA, and TruthfulQA. Linear ridge regression models (RidgeCV with 5-fold CV, \(\alpha \in [10^{-3}, 10^5]\)) are trained on an 8:2 model-wise split (\(K_{\text{train}} = 388, K_{\text{test}} = 97\)), evaluating test-set Pearson \(r\) and Spearman \(\rho\) with 2,000 bootstrap resamplings.
Key Experimental Results¶
Main Results¶
The primary evaluation assesses whether centered conditional log-likelihood vectors \(\boldsymbol{\xi}\) can accurately predict downstream task performance across held-out models (\(K_{\text{test}} = 97\), Table 3). Table 8 further benchmarks conditional versus unconditional representations on the subset of 438 models where both feature sets are available.
| Task | Pearson \(r\) | Spearman \(\rho\) | Conditional \(r\) (438 Models) | Unconditional \(r\) (438 Models) |
|---|---|---|---|---|
| ARC-Challenge | 0.829 [0.783, 0.867] | 0.821 [0.765, 0.869] | 0.914 [0.864, 0.950] | 0.914 [0.868, 0.950] |
| ARC-Easy | 0.841 [0.803, 0.872] | 0.824 [0.778, 0.863] | 0.864 [0.791, 0.914] | 0.839 [0.757, 0.901] |
| OpenBookQA | 0.586 [0.363, 0.807] | 0.797 [0.728, 0.854] | 0.879 [0.819, 0.929] | 0.862 [0.733, 0.943] |
| PIQA | 0.906 [0.881, 0.931] | 0.945 [0.926, 0.960] | 0.945 [0.904, 0.972] | 0.920 [0.867, 0.965] |
| TruthfulQA | 0.842 [0.790, 0.890] | 0.869 [0.820, 0.906] | 0.902 [0.830, 0.950] | 0.884 [0.808, 0.935] |
| Mean | 0.801 | 0.851 | 0.901 | 0.884 |
Ablation Study¶
The additive compositionality of prompt shift vectors was evaluated across different reference text sets (Table 5) by calculating the relative prediction error \(\|\mathbf{v}_{\text{cot}, i} + \mathbf{v}_{\text{rep}, i} - \mathbf{v}_{\text{rep+cot}, i}\| / \|\mathbf{v}_{\text{rep+cot}, i} + \mathbf{c}_i\|\) in both \(\boldsymbol{\ell}\) and \(\boldsymbol{\xi}\) spaces, alongside the Pearson correlation between actual and summed scalar shifts \(\delta\). In addition, Table 2 examines fine-tuning domain alignment via mean PMI (\(\Delta \bar{\ell}_i\)) and conditional log-likelihood (\(\bar{\ell}_i\)).
| Text Set | Models \(K\) | Texts \(N\) | Rel. Error in \(\boldsymbol{\ell}\) (Median) | Rel. Error in \(\boldsymbol{\ell}\) (Mean) | Rel. Error in \(\boldsymbol{\xi}\) (Median) | Pearson \(r\) of \(\delta\) |
|---|---|---|---|---|---|---|
| Tulu3 (Full) | 552 | 10,000 | 0.062 [0.058, 0.066] | 0.081 [0.075, 0.088] | 0.074 [0.070, 0.080] | 0.987 [0.959, 0.995] |
| Tulu3 (Matched) | 100 | 5,000 | 0.064 [0.054, 0.070] | 0.070 [0.062, 0.079] | 0.070 [0.064, 0.081] | 0.984 [0.970, 0.992] |
| Infinity-Instruct | 100 | 5,000 | 0.083 [0.073, 0.102] | 0.097 [0.086, 0.109] | 0.100 [0.090, 0.109] | 0.986 [0.974, 0.993] |
| UltraChat | 100 | 5,000 | 0.061 [0.048, 0.081] | 0.084 [0.072, 0.098] | 0.087 [0.074, 0.111] | 0.975 [0.964, 0.990] |
Key observations from post-training domain alignment (Table 2):
- Llama-3.1-Tulu-3-8B-SFT shows a massive \(\Delta \text{MI}\) gain of \(+97.3\) on in-domain Tulu3, compared to only \(+38.1\) on out-of-domain Infinity-Instruct. Its conditional log-likelihood changes by \(+2.4\) on Tulu3 but collapses by \(-32.1\) on Infinity-Instruct.
- SmolTulu-1.7b-Instruct gains \(+108.4\) \(\Delta \text{MI}\) on Tulu3, versus \(+34.3\) on Infinity-Instruct.
- Infinity-Instruct-3M-0625-Qwen2-7B improves \(\Delta \text{MI}\) by \(+5.4\) on its training source Infinity-Instruct, but only \(+0.4\) on Tulu3.
Key Findings¶
- High linear additivity of prompt shift vectors: Shift vectors induced by CoT suffixing and prompt repetition compose additively with median relative prediction errors as low as 6.2% to 8.3%. The scalar change in log-likelihood \(\delta_{\text{rep+cot}}\) correlates almost perfectly (\(r = 0.987\)) with \(\delta_{\text{cot}} + \delta_{\text{rep}}\).
- Zero-inference prediction of composite prompt effects: Constructing synthetic feature vectors \(\hat{\boldsymbol{\xi}}_{\text{rep+cot}} = \boldsymbol{\xi}_{\text{cot}} + \boldsymbol{\xi}_{\text{rep}} - \boldsymbol{\xi}_{\text{base}}\) produces downstream performance predictions that strongly correlate with true task scores under composite prompting (\(r = 0.819\) to \(0.913\) across ARC and PIQA), without observing model responses under the composite prompt.
- PMI vectors effectively isolate fine-tuning lineage: While conditional log-likelihood \(\bar{\ell}\) is dominated by pre-training base capabilities, PMI vectors subtract unconditional response plausibility, highlighting clear clustering corresponding to instruction-tuning datasets (Tulu3 vs. Infinity-Instruct).
- Model lineage dictates KL approximation fidelity: Comparing vector-based KL estimates to Monte Carlo sampled KL across 24 diverse models (Table 10) demonstrates high correlation for same-family model pairs (\(\rho = 0.908\), \(r = 0.909\)), which moderates for cross-family pairs (\(\rho = 0.781\), \(r = 0.732\)). This confirms that the local Taylor expansion holds with high precision when models are close in distributional space.
Highlights & Insights¶
- Generation-Free Representation of Conditional Models: By evaluating token log-likelihoods over static prompt-response pairs, the method avoids sampling variance, decoding parameters (temperature, top-p), and sentence-encoder embedding biases, enabling objective, deterministic comparison of conditional distributions.
- Prompt Vector Arithmetic: Mirroring the classic word embedding arithmetic (
v(King) - v(Man) + v(Woman) ≈ v(Queen)), prompt operations act as linear translation operators in log-likelihood space (v(Repeat+CoT) ≈ v(Repeat) + v(CoT)), opening new avenues for analytic prompt engineering. - Extremely Low Carbon Footprint: Constructing the entire model map across 552 models over 5 prompt conditions requires only forward pass evaluations of target tokens, consuming roughly 317 kWh (\(\approx 153\ \text{kgCO}_2\text{eq}\)), orders of magnitude cheaper than autoregressively generating text corpora.
Limitations & Future Work¶
- Reliance on White-Box Log-Likelihoods: The methodology requires token-level conditional log-likelihood access, making it inapplicable to closed-source commercial APIs that do not return full sequence log-probabilities.
- Scope of Prompt Transformations: Experiments focus on two prompt modifications (CoT suffixing and prompt repetition). Whether similar additive compositionality generalizes to semantic transformations, rephrasing, few-shot demonstrations, or language switching remains an open question.
- Approximation Error in Empty-Prompt Marginal: The empty-prompt baseline \(p(y|\emptyset)\) serves as a practical substitute for the model marginal \(p^{\text{model}}(y)\), but it deviates from the formal reference marginal \(p^{\text{ref}}(y)\). While empirical correlations are strong (\(r = 0.806\)), discrepancies may arise for models highly sensitive to system prompt formatting.
Related Work & Insights¶
- Evolution from Unconditional to Conditional Model Maps: Extends the unconditional log-likelihood framework of Oyama et al. (2025b) to prompt-conditioned distributions \(p(y|x)\), bridging the gap between statistical probability maps and modern instruction-following LLM paradigms.
- Comparison to Generative and Parameter-Based Distances: Unlike Artificial Hivemind (Jiang et al., 2025), which measures cosine distance between MiniLM embeddings of generated texts, this approach operates directly on exact probability mass functions without decoding bias; unlike weight-space analysis (Horwitz et al., 2025), it natively compares models across disparate architectures, layer widths, and MoE configurations.
- Implications for Test-Time Scaling & Alignment Auditing: The ability to additively combine shift vectors provides an analytic foundation for predicting test-time compute scaling. Furthermore, PMI vectors offer an effective diagnostic fingerprint for identifying data contamination and tracking post-training adaptation.
Rating¶
- Novelty: 4.5 / 5.0 (Extends model mapping to conditional spaces, formalizing prompt shifts as additive vector arithmetic)
- Experimental Thoroughness: 4.8 / 5.0 (552 models evaluated across multiple text sets, backed by rigorous Monte Carlo KL validation and CarbonTracker accounting)
- Writing Quality: 4.7 / 5.0 (Clean theoretical derivations with comprehensive appendix proofs for variance decompositions and quadratic approximations)
- Value: 4.6 / 5.0 (Provides an efficient, generation-free metric space for model diagnostics, performance prediction, and prompt analysis)