Skip to content

LEVEE: Layer-wise Evaluation of Vector Encoding Effects: Bias Localization and Functional Separation across 11 LLMs

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/GDM1nu/EMNLP-2026-LEVEE
Area: LLM Safety
Keywords: Task Vectors, Bias Suppression, Weight-Space Decomposition, Functional Separation, Active Suppression

TL;DR

LEVEE conducts a systematic layer-wise interventional analysis of alignment task vectors \(\Delta W\) across 11 LLMs, proving that social bias suppression strictly localizes to the mid-third of transformer layers, functionally separates from capability-critical early layers in weight space, and actively neutralizes latent bias amplification residing in the outer thirds.

Background & Motivation

Weight merging and task arithmetic have emerged as an increasingly prevalent paradigm for transferring capabilities across large language models without retraining. By defining the alignment task vector as the difference between instruction-tuned and base weights (\(\Delta W = W_{\text{instruct}} - W_{\text{base}}\)), practitioners can directly modify model behaviors through linear addition. However, existing literature overwhelmingly treats \(\Delta W\) as a monolithic black box, focusing exclusively on resolving parameter interference and sparsification. This leaves a fundamental mechanistic question unresolved: alignment fine-tuning simultaneously instills instruction-following, bias mitigation, and safety refusal, yet the internal layer-wise organization of these disparate behavioral signals remains completely opaque.

Prior mechanistic interpretability investigations predominantly operate in activation space or inspect single static models. For example, causal tracing localizes factual associations to mid-layer MLP activations, and representation engineering identifies steering directions for refusal. However, correlational representation analysis cannot substitute for causal transmission in weight modifications, nor do the layers that initially store stereotypical associations in base models necessarily coincide with the layers modified by alignment task vectors \(\Delta W\). Without knowing the spatial distribution of safety versus general capability inside \(\Delta W\), arbitrary model merging, parameter pruning, or task vector arithmetic risks destroying safety guardrails or inadvertently producing deployed models that are drastically more biased than the unaligned base.

This work introduces LEVEE, a systematic layer-wise interventional framework that interrogates the internal structure of \(\Delta W\) across 11 instruction-tuned models spanning the Qwen2.5, Gemma-2/3, and Llama-3 families (0.5B to 14B parameters, with further extensions up to 32B). Through single-layer sufficiency sweeps, block removal ablations, sign-flip tests, and capability-controlled evaluations, the authors establish a definitive causal map of alignment encodings in weight space. Core idea: by treating the alignment task vector \(\Delta W\) as an ordered, depth-structured composite object rather than a monolith, this paper demonstrates that social bias suppression is causally localized to the mid-third of transformer layers, exhibits clean functional double-dissociation from the capability-critical early layers in weight space, and actively counteracts latent bias-amplifying components introduced by the outer thirds during fine-tuning.

Method

Overall Architecture

The LEVEE pipeline measures the behavioral impact of isolating and perturbing specific layer subsets of the alignment update. It extracts the layer-wise difference tensors \(\Delta W_i\) between paired instruct and base models, systematically constructs intervened model weights \(\hat{W}\) across five experimental paradigms (single-layer sufficiency sweep, block removal necessity, cumulative prefix injection, sign-flip directionality, and norm-matched controls), evaluates social bias on the full BBQ benchmark alongside general capability on a domain-stratified 14-subject MMLU subset, and applies a layer-selective safety auditing protocol to detect latent bias amplification.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Extract Paired Weight Difference<br/>W_instruct - W_base"] --> B["Layer-Wise Task Vector Decomposition<br/>Isolate per-block tensors ΔW_i"]
    B --> C["Single-Layer Sufficiency Sweep<br/>Evaluate W_base + ΔW_i per layer"]
    C --> D["Block Removal & Functional Dissociation<br/>Intervene on -first / -mid / -last"]
    D --> E["Direction-Specificity & Geometry Validation<br/>Sign-flip -ΔW_i and norm-matched controls"]
    E --> F["Layer-Selective Safety Auditing<br/>Detect outer-third bias amplification"]
    F --> G["Safe Model Merging & Targeted Debiasing"]

Key Designs

1. Layer-Wise Task Vector Decomposition & Sufficiency Sweep: Dissecting monolithic updates into isolated single-layer injections
To overcome the blind spots of monolithic evaluation, this design explicitly isolates the weight update for each individual transformer block: $\(\Delta W_i = W_{\text{instruct}, i} - W_{\text{base}, i}, \quad i \in \{0, \dots, N-1\}\)$ The extraction spans all shared weight matrices within block \(i\), including multi-head attention projections (\(W_q, W_k, W_v, W_o\)), feed-forward network weights, and internal layer norms, while holding token embeddings, the output classification head, and final RMSNorm layers fixed at their unaligned base values. Under the single-layer sufficiency experiment, only the \(i\)-th block's task vector is added into the base model (\(\hat{W}_i = W_{\text{base}} + \Delta W_i\)), keeping all remaining \(N-1\) layers identical to the base architecture. By evaluating downstream bias using the \(L\text{-BBQ}\) metric across 58,492 items, this intervention directly tests whether single-layer parameter updates are sufficient to induce meaningful bias suppression in isolation.

2. Block Removal & BBQ-MMLU Double Dissociation: Establishing spatial necessity and functional independence in weight space
To examine whether safety alignment and general capabilities compete for the same parameter subspace across network depth, the model layers are partitioned into three contiguous blocks of equal size: First-third (\(0 \sim \lfloor N/3 \rfloor - 1\)), Mid-third (\(\lfloor N/3 \rfloor \sim \lfloor 2N/3 \rfloor - 1\)), and Last-third (\(\lfloor 2N/3 \rfloor \sim N - 1\)). The protocol applies the complete alignment vector while selectively ablating exactly one contiguous block: $\(\hat{W} = W_{\text{base}} + \sum_{i \notin \text{block}} \Delta W_i\)$ Evaluating the resulting models across conditions—Full, -first, -mid, -last, and mid-only—on both BBQ and a pre-frozen, domain-stratified 14-subject MMLU subset (spanning STEM, Humanities, Social Sciences, and Professional disciplines) establishes a double dissociation. Removing the mid-third (-mid) consistently precipitates severe degradation in bias mitigation across all models, whereas ablating the first-third (-first) triggers massive drops in general capability (up to 35.0 percentage points on MMLU in Gemma-2-9B) while leaving bias metrics almost completely undisturbed (\(|\Delta L\text{-BBQ}| \le 0.006\)).

3. Direction-Specificity & Static Geometry Contrast: Ruling out OOD perturbations and disproving static weight heuristics
To verify that mid-layer debiasing reflects genuine directional semantics rather than arbitrary out-of-distribution (OOD) activation disruptions, the framework executes sign-flip interventions (\(-\Delta W_i\) injection) and norm-matched random Gaussian noise controls calibrated to identical Frobenius norms \(\|\Delta W_i\|_F\). In structured-signal models, injecting \(+\Delta W_i\) suppresses bias while \(-\Delta W_i\) symmetrically amplifies bias (exhibiting up to 100% sign-flip symmetry in Qwen2.5 models), while norm-matched controls yield substantially weaker suppression ratios (0.535x–0.548x). Crucially, evaluating static weight statistics reveals that Frobenius norm, sign coherence, and participation ratio all peak in the last third of the network, diverging completely from behavioral localization and proving that interventional analysis is indispensable for uncovering functional alignment geometry.

4. Layer-Selective Safety Auditing: Identifying latent amplifiers and verifying active suppression
To safeguard against catastrophic safety failures during selective model merging or pruning, an empirical auditing protocol evaluates models under three conditions: Full \(\Delta W\), Outer-only (-mid), and Mid-only. An audit failure is triggered whenever outer-only injection results in higher social bias than the unaligned base model (\(\text{BBQ}_{\text{outer-only}} > \text{BBQ}_{\text{base}}\)). This diagnostic establishes that mid-third layers do not merely host passive bias-neutral updates; rather, they exert an active suppression mechanism that dynamically cancels out latent bias-amplifying weight changes induced in the outer thirds during instruction tuning. If the mid-third is stripped away, these latent amplifiers are unleashed, rendering the model substantially more biased than if it had never been aligned at all.

Benchmark & Evaluation Protocol

Social bias is quantified using a normalized ambiguous-context metric, denoted \(L\text{-BBQ}\): $\(S_{L\text{-BBQ}} = (1 - \text{Acc}_{\text{amb}}) \cdot \frac{1}{N} \sum_{i=1}^N b_i\)$ where \(\text{Acc}_{\text{amb}}\) represents ambiguous-context accuracy, \(b_i \in \{+1, -1, 0\}\) classifies answers into biased, anti-biased, or neutral/unknown categories based on BBQ's target metadata, and \(N\) denotes the total count of resolved ambiguous responses. The scaling factor \((1 - \text{Acc}_{\text{amb}})\) modulates signed stereotyping by the frequency of non-neutral commitments (closer to 0 indicates less bias). Capability is benchmarked using 5-shot MMLU across 14 stratified subjects. All experiments are executed in native bf16 precision without quantization using lm-evaluation-harness on NVIDIA A100 GPUs with deterministic greedy decoding.

Key Experimental Results

Main Results

The single-layer sufficiency sweep and block removal experiments across 11 core instruction-tuned models demonstrate clear localization of bias suppression to the mid-third.

Model Params Layers \(N\) Best Single Layer Best Third Base \(L\text{-BBQ}\) -mid \(\Delta L\text{-BBQ}\) (vs Full) -first \(\Delta L\text{-BBQ}\) (vs Full) -last \(\Delta L\text{-BBQ}\) (vs Full)
Qwen2.5-0.5B‡ 0.5B 24 L10 Mid 0.252 +0.078 +0.119 -0.003
Qwen2.5-1.5B‡ 1.5B 28 L23 Last 0.273 +0.118 +0.045 +0.004
Qwen2.5-3B 3B 36 L18 Mid 0.050 +0.019 <0.001 <0.001
Qwen2.5-14B 14B 48 L21 Mid 0.013 +0.007 <0.001 <0.001
Gemma-2-2B 2B 26 L11 Mid 0.160 +0.169* <0.001 -0.006
Gemma-2-9B 9B 42 L20 Mid 0.116 +0.057 +0.001 <0.001
Gemma-3-12B 12B 48 L18 Mid 0.095 +0.069 +0.020 <0.001
Llama-3.2-1B 1B 16 L9 Mid 0.251 +0.111 +0.014 +0.013
Llama-3.2-3B 3B 28 L10 Mid 0.176 +0.215* -0.011 -0.002
Llama-3-8B 8B 32 L16 Mid 0.125 +0.115* -0.011 -0.005
Llama-3.1-8B 8B 32 L11 Mid 0.118 +0.030 +0.028 -0.001

(Notes: ‡ denotes diffuse-signal models lacking localized convergence; * denotes models where post-removal bias surpasses the unaligned base, confirming active suppression. Under exact binomial testing across all 9 structured-signal models, mid-third localization is statistically significant at \(p \approx 5.1 \times 10^{-5}\).)

Ablation Study: BBQ-MMLU Double Dissociation & Safety Audit

The table below contrasts bias degradation and general capability shifts (percentage point change in MMLU accuracy relative to Full \(\Delta W\)) alongside the layer-selective safety audit verdicts.

Model Condition / Evaluation Base Bias Full \(\Delta W\) Bias Outer-only (-mid) Bias \(\Delta L\text{-BBQ}\) (vs Full) \(\Delta \text{MMLU}\) (pp vs Full) Audit Verdict
Qwen2.5-3B -mid removal 0.050 <0.001 0.019 +0.019 -0.8 pp Pass
Qwen2.5-3B -first removal 0.050 <0.001 - +0.000 +0.3 pp -
Gemma-2-2B -mid removal 0.160 0.016 0.184 +0.169 -6.7 pp Fail (+15% above base)
Gemma-2-2B -first removal 0.160 0.016 - -0.006 -12.4 pp -
Gemma-2-9B -mid removal 0.116 <0.001 0.058 +0.057 -3.9 pp Pass
Gemma-2-9B -first removal 0.116 <0.001 - +0.001 -35.0 pp -
Llama-3.2-3B -mid removal 0.176 0.035 0.249 +0.215 -4.8 pp Fail (+42% above base)
Llama-3.2-3B -first removal 0.176 0.035 - -0.011 -1.0 pp -
Llama-3-8B -mid removal 0.125 0.067 0.182 +0.115 +2.0 pp Fail (+46% above base)

(Notes: Audit fails when outer-only BBQ score exceeds unaligned base BBQ. Mid-third removal is universally bias-critical, whereas first-third removal is capability-critical in Gemma architectures.)

Key Findings

  • Universal Mid-Third Localization: In 9 out of 11 models (100% of structured-signal architectures), the single layer achieving minimum bias belongs to the mid-third. In block removal, omitting the mid-third (-mid) causes the largest increase in bias in 10 of 11 models (\(p \approx 1.3 \times 10^{-4}\)).
  • Functional Double Dissociation in Weight Space: Bias suppression and general capability occupy distinct depth intervals. Mid-third removal destroys bias mitigation with negligible MMLU loss (e.g., -0.8 pp in Qwen2.5-3B, -3.9 pp in Gemma-2-9B). Conversely, first-third removal causes massive MMLU collapse in Gemma models (up to -35.0 pp) while altering bias scores by less than 0.006. The last-third carries negligible influence on both fronts.
  • Empirical Validation of Active Suppression: In 3 of 11 models (Llama-3.2-3B, Llama-3-8B, Gemma-2-2B), injecting only the outer thirds produces a model with +15% to +46% higher bias than the unaligned base. This demonstrates that alignment fine-tuning introduces latent bias-amplifying updates in outer layers that are actively counterbalanced by the mid-third suppressor.
  • Capacity-Modulated Localization Threshold: Model scale governs the emergence of localized encoding. Small models like Qwen2.5-0.5B and 1.5B show diffuse, flat per-layer sensitivity across the entire network, whereas sharp mid-third localization emerges at 3B in Qwen and below 1B in Llama, pointing to an architecture-dependent capacity threshold.

Highlights & Insights

  • De-monolithizing the Task Vector: Demonstrates that task vectors are not homogeneous transformations but possess a rich functional topology along network depth, establishing that safety and utility are modularly separable in weight space.
  • Unveiling the Hidden Threat of Partial Merging: Highlights that selective model merging or pruning based solely on capability metrics will inadvertently discard safety-critical mid-layers, unleashing latent bias amplifiers and yielding models worse than unaligned baselines.
  • Enabling Targeted, Zero-Retraining Debiasing: Proves that injecting solely the mid-third of \(\Delta W\) achieves 89%–99% of full bias suppression while preserving over 95% of general utility, providing a practical blueprint for surgical alignment editing.

Limitations & Future Work

  • Architectural and Scale Horizons: The primary controlled study is bounded by models up to 14B parameters; while Appendix extensions validate 24B–32B models, empirical testing on frontier scale models (\(\ge 70\text{B}\)) remains unverified, where representations might either sharpen further or disperse.
  • Absence of Circuit-Level Granularity: The method operates at the block level and confirms physical localization, but does not isolate specific attention heads, induction circuits, or MLP key-value memory slots that govern debiasing mechanics.
  • Format and Task Generalization: Benchmark analysis centers primarily on multiple-choice ambiguous QA formats (BBQ); extending layer-wise interventional mapping to free-form generation toxicity, safety refusal dynamics, and hallucination calibration represents an open challenge.
  • vs Ilharco et al. (Task Arithmetic) & TIES-Merging: Prior task vector merging methods treat \(\Delta W\) as a uniform tensor and resolve cross-task conflicts via magnitude pruning; this paper proves that pruning without depth-aware auditing can catastrophically sever safety-critical mid-layers.
  • vs Meng et al. (ROME / MEMIT) & Geva et al.: Mechanistic studies on pre-trained models locate factual associations in mid-layer MLP modules; this work establishes that alignment fine-tuning strategically modifies this exact mid-layer semantic substrate to neutralize stereotypical associations.
  • vs Zou et al. (Representation Engineering) & Arditi et al.: Representation steering manipulates hidden activations at inference time; LEVEE operates directly on parameter delta matrices, providing an unshakeable weight-space map that persists across execution states.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ [Pioneers the layer-wise interventional decomposition of task vectors and discovers active suppression]
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Covers 15 models, 5 interventional designs, 58k+ BBQ items, and extensive robustness checks]
  • Writing Quality: ⭐⭐⭐⭐⭐ [Exceptional clarity, rigorous statistical framing, and candid treatment of architectural anomalies]
  • Value: ⭐⭐⭐⭐⭐ [Directly informs safe model merging, parameter pruning, and targeted safety alignment without retraining]