Skip to content

Debias-SparseGPT: Bias-Aware Pruning for Large Language Models

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/upunaprosk/debias-llm-compressor
Area: LLM Reasoning
Keywords: LLM Compression, Weight Pruning, Second-Order Reconstruction, Algorithmic Fairness, Bias Mitigation

TL;DR

Debias-SparseGPT introduces a bias-aware second-order formulation into post-training pruning by augmenting the layer-wise reconstruction objective with paired demographic contrasts, directly counteracting stereotype amplification across unstructured and 2:4 semi-structured sparsity regimes without compromising computational throughput or baseline language modeling accuracy.

Background & Motivation

Weight pruning has emerged as a cornerstone technique for deploying large language models (LLMs) at scale, enabling substantial throughput acceleration and memory footprint reduction. Second-order post-training pruning methods, notably SparseGPT, formulate parameter sparsification as a sequence of layer-wise quadratic reconstruction problems. By computing an empirical input Hessian on calibration sequences and executing block-wise parameter compensation, these approaches achieve remarkable sparsity-to-perplexity trade-offs without requiring expensive retraining.

However, recent investigations into compressed LLM trust have uncovered a severe degradation in output fairness. While pruned models maintain near-lossless performance on coarse benchmarks like MMLU or HellaSwag, their behavioral safety deteriorates significantly in open-ended and causal question-answering tasks involving demographic attributes. When presented with underspecified contexts where the objective truth is "Unknown" or "Not stated", pruned models systematically abandon neutral abstention and produce answers skewed toward stereotypical group associations.

The root cause of this failure mode lies in the objective function of standard second-order pruning: it seeks exclusively to preserve raw layer output activations under typical inputs, treating all parameter coordinates as neutral contributors to global reconstruction error. In doing so, standard pruning disproportionately strips away subtle parameters responsible for suppressing representational disparities between sensitive demographic counterparts. Core idea: incorporate a representational divergence penalty over paired counterfactual inputs into the second-order reconstruction objective, deriving a bias-aware Hessian that jointly directs saliency mask selection and error compensation without increasing the computational complexity of SparseGPT.

Method

Overall Architecture

Debias-SparseGPT targets the mitigation of pruning-induced demographic bias during one-shot post-training model compression. Given a pretrained weight matrix \(W \in \mathbb{R}^{n \times d}\), a target sparsity ratio \(s\), and paired contrastive calibration sequences \((X_0, X_1)\) (e.g., minimal pro-stereotypical and anti-stereotypical sentence pairs differing only by demographic tokens), the method executes in three interconnected phases. First, it computes a bias-aware input Hessian matrix \(H\) by combining reconstruction outer products with a cross-pair difference penalty. Second, it utilizes the inverse Cholesky factor of \(H\) to compute second-order parameter saliencies and select the retention mask \(M\). Finally, it applies block-wise second-order parameter compensation and lazy batched tail updates across unpruned columns.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: Layer Weights W<br/>Paired Contrastive Data (X0, X1) & Target Sparsity s"] --> B["Paired Difference Extraction & Bias-Aware Hessian Accumulation<br/>Compute H = Hacc + Hbias = X0 X0^T + X1 X1^T + 2 dX dX^T"]
    B --> C["Numerically Stable Cholesky Inversion<br/>Calculate C^T C = H^-1 and inverse diagonal entries"]
    C --> D["Bias-Sensitive Second-Order Saliency & Mask Construction<br/>Prune lowest saliency weights eps_p within stride blocks"]
    D --> E["Second-Order Block Compensation & Lazy Batched Tail Updates<br/>Local residual buffer E propagation and unpruned weight updates"]
    E --> F["Output: Pruned Fair Weight Matrix W_hat"]

Key Designs

1. Paired Difference Extraction & Bias-Aware Hessian Accumulation: Preserving relative geometry between demographically contrasting inputs

Standard SparseGPT approximates the parameter-level Hessian solely through the empirical covariance \(X X^\top\) of unconstrained calibration text, which fails to penalize shifts in sensitive attribute representations. Debias-SparseGPT reformulates the layer-wise reconstruction objective over paired contrastive representations \(X_0, X_1 \in \mathbb{R}^{d \times N_t}\) with difference \(\Delta X = X_0 - X_1\):

\[\min_{\hat{W}} \frac{1}{2} \sum_{i \in \{0,1\}} \| W X_i - \hat{W} X_i \|_2^2 + \| W \Delta X - \hat{W} \Delta X \|_2^2\]

The first term enforces output fidelity across both demographic inputs, while the second term explicitly penalizes alterations in their representational distance \(\Delta X\). Because the model parameters \(w = \text{vecr}(W)\) are pretrained and satisfy first-order stationarity (\(J_w = 0\)), expanding this objective into a second-order Taylor series yields the weight-space Hessian \(H_w = I_n \otimes H\), where the input-space Hessian \(H \in \mathbb{R}^{d \times d}\) is given analytically by:

\[H = X_0 X_0^\top + X_1 X_1^\top + 2 \Delta X \Delta X^\top\]

By directly accumulating the outer product \(2 \Delta X \Delta X^\top\) into the running Hessian sum, the formulation requires no additional \(d \times d\) memory storage and preserves the \(O(d^2)\) layer-wise space complexity.

2. Bias-Sensitive Second-Order Saliency & Mask Construction: Incorporating fairness penalties into parameter removal decisions

Pruning a single weight coordinate \(w_p\) imposes the hard constraint \((\hat{w})_p = 0\), or \(e_p^\top \Delta w = -w_p\). Under the constrained quadratic minimization problem \(\min_{\Delta w} \frac{1}{2} \Delta w^\top H_w \Delta w\) subject to \(e_p^\top \Delta w = -w_p\), the Lagrangian formulation yields the exact saliency metric:

\[\varepsilon_p = \frac{1}{2} (\Delta w^*)^\top H_w \Delta w^* = \frac{w_p^2}{2 [H_w^{-1}]_{pp}} = \frac{w_{ij}^2}{2 [H^{-1}]_{jj}}\]

Here, \([H^{-1}]_{jj}\) represents the \(j\)-th diagonal element of the inverted input-space Hessian. Because the bias term \(2 \Delta X \Delta X^\top\) scales the curvature along directions sensitive to demographic substitutions, weights operating on critical debiasing axes exhibit smaller diagonal inverse values \([H^{-1}]_{jj}\), driving up their saliency \(\varepsilon_p\). During mask construction, for any target sparsity \(s\) under unstructured or semi-structured \(N:M\) patterns, the algorithm retains the \((1-s)\) parameters with the largest \(\varepsilon_p\) values, preventing the inadvertent elimination of bias-mitigating connections.

3. Second-Order Block Compensation & Lazy Batched Tail Updates: Efficient error propagation without scaling runtime overhead

Once a parameter column is pruned, the Optimal Brain Surgeon (OBS) analytic solution compensates remaining unpruned parameters via:

\[\Delta w^* = - \frac{w_p}{[H_w^{-1}]_{pp}} H_w^{-1} e_p\]

To maintain high numerical stability and computational throughput, Debias-SparseGPT computes the Cholesky factorization \(C^\top C = H^{-1}\) with upper-triangular matrix \(C\). Inside each column block of size \(B\) (defaulting to 128), local column errors \(E_{:, j-i} = (1 - M_{:, j}) \odot (W_{:, j} / [C]_{jj})\) immediately adjust the weights within the current block: \(W_{:, j:(i+B)} \leftarrow W_{:, j:(i+B)} - E_{:, j-i} \cdot C_{j, j:(i+B)}\). Once all columns in block \(B\) are processed, the accumulated residual buffer \(E \in \mathbb{R}^{n \times B}\) updates the remaining unpruned weight columns \((i+B):\) in a single matrix multiplication: \(W_{:, (i+B):} \leftarrow W_{:, (i+B):} - E \cdot C_{i:(i+B), (i+B):}\). This block-lazy execution model keeps the total computational complexity at \(O(d_{\text{hidden}}^3)\), identical to standard SparseGPT.

Loss & Training

Debias-SparseGPT is a training-free, post-training pruning algorithm executed in a single forward pass without backpropagation. - Calibration Datasets: StereoSet development set containing 4,212 minimally paired pro- and anti-stereotypical sentences. In aggressive 2:4 structured sparsity regimes, the calibration set is augmented with 256 UltraChat dialogue examples (UltraChat tokens update only the standard reconstruction terms without \(\Delta X\)). - Hyperparameters: Hessian diagonal damping fraction \(\lambda = 0.01\); block size \(B = 128\); saliency stride \(B_s = M\) for \(N:M\) semi-structured settings. - Coverage: Pruning applies uniformly to all linear projections across transformer layers: attention projections (q_proj, k_proj, v_proj, o_proj) and feed-forward MLP projections (gate_proj, up_proj, down_proj).

Key Experimental Results

Main Results

Evaluations span nine open-source LLMs (seven instruction-tuned models and two base models). Key metrics include WikiText-2 perplexity (PPL), zero-shot reasoning (HellaSwag, MMLU), generative stereotype benchmarks (UnQover, BBQ, evaluating accuracy in selecting "Unknown/Not stated"), CrowS-Pairs preference (ideal at 50%), and Distance-to-Optimum (DTO, measuring Euclidean distance from perfect (1.0, 1.0) performance on MMLU and UnQover).

Below are the primary results under 1:4 semi-structured sparsity (directly excerpted from Table 2 in the paper):

Model + Method PPL โ†“ HellaSwag โ†‘ MMLU โ†‘ UnQover โ†‘ BBQ โ†‘ CP (CrowS-Pairs) โ†“ DTO โ†“
Llama-3.1-8B-IT (Dense) 6.99 57.49 63.17 30.10 76.40 61.72 0.559
+ Magnitude 9.48 56.74 62.46 17.93 69.10 61.36 0.638
+ Wanda 8.10 55.25 56.54 41.98 68.20 62.75 0.513
+ SparseGPT 8.17 55.56 59.11 35.60 67.10 60.64 0.539
+ Debias-SparseGPT (Ours) 8.19 55.45 59.76 60.46โ€  70.70โ€  59.81 0.399
Vicuna-1.5-7B (Dense) 6.92 56.58 48.60 17.44 41.60 66.43 0.688
+ Magnitude 7.39 57.09 47.37 12.21 35.50 65.12 0.724
+ Wanda 7.45 54.58 45.62 20.56 37.80 65.83 0.681
+ SparseGPT 7.66 54.13 46.14 17.82 37.00 65.30 0.695
+ Debias-SparseGPT (Ours) 7.64 54.30 45.81 21.54โ€  37.90 65.89 0.674
Qwen-2.5-7B-IT (Dense) 7.14 56.99 68.71 73.17 86.30 60.82 0.291
+ Magnitude 9.62 54.46 65.25 51.42 85.50 61.30 0.422
+ Wanda 7.75 55.23 67.23 61.51 87.50 61.18 0.357
+ SparseGPT 7.92 55.15 67.35 70.60 85.40 61.24 0.311
+ Debias-SparseGPT (Ours) 7.92 55.06 67.73 74.41โ€  86.60โ€  59.99โ€  0.291

(Note: โ€  denotes statistically significant improvements over SparseGPT under a paired t-test with \(p < 0.01\).)

Ablation Study

To investigate the performance under aggressive 2:4 structured hardware sparsity, the authors ablate the impact of calibration data diversity on Qwen-2.5-7B-IT by augmenting StereoSet (SS) with 256 UltraChat dialogue samples (data directly excerpted from Table 4 and Table 12):

Pruning Method Calibration Data Configuration PPL โ†“ MMLU โ†‘ UnQover โ†‘ BBQ โ†‘ DTO โ†“ Note
SparseGPT StereoSet Only 15.89 50.41 28.45 64.10 0.616 Severe degradation under aggressive 2:4 constraint
SparseGPT StereoSet + 256 UltraChat 13.84 53.84 42.46 75.20 0.522 Rich context provides general recovery (+14.01 UnQover)
Debias-SparseGPT StereoSet Only 16.17 48.16 24.94 66.70 0.645 Short sentences alone lack context diversity under 2:4
Debias-SparseGPT StereoSet + 256 UltraChat 13.37 54.17 47.26 78.60 0.494 Paired term + long context yields highest accuracy & lowest DTO

Further category-specific calibration ablations (Table 15) reveal that religion-specific contrastive calibration transfers exceptionally well across categories (achieving 52.50% average UnQover and a 0.453 DTO), while full StereoSet calibration delivers the lowest overall perplexity (13.37).

Key Findings

  • Reversal of Pruning-Induced Bias Amplification: On LLaMA-3.1-8B, Debias-SparseGPT drives UnQover accuracy to 60.46% (compared to 35.60% for SparseGPT and 41.98% for Wanda), while restoring BBQ accuracy from 67.10% to 70.70%, approaching the dense model baseline of 76.40%.
  • Role of Baseline Predictive Uncertainty: Analysis of prediction entropy on UnQover (Table 10) demonstrates that LLaMA-3.1-8B exhibits high entropy on correct predictions (0.938 with confidence 0.545), whereas Qwen-2.5-7B is highly confident (entropy 0.109 with confidence 0.960). Models with higher baseline uncertainty are far more vulnerable to compression distortion, explaining why LLaMA benefits most dramatically from bias-aware pruning.
  • Hardware Acceleration and Reduced Carbon Footprint: Benchmarking Qwen-2.5-7B on an NVIDIA A100 GPU using vLLM under 2:4 sparsity (Table 5) confirms that Debias-SparseGPT boosts inference throughput from 27.54 to 73.05 tokens/s while lowering emissions from 0.0998 to 0.0376 kgCO2e per million tokens, achieving a superior DTO of 0.494 versus SparseGPT's 0.522.

Highlights & Insights

  • Zero-Memory Cost Hessian Augmentation: By formulating the debiasing objective as a quadratic difference penalty, the algorithm directly integrates \(2 \Delta X \Delta X^\top\) into the existing Hessian accumulator, preserving the closed-form OBS update without adding extra optimization phases.
  • Localization of Bias-Sensitive Weights: Layer-wise mask disagreement analysis (Table 17) demonstrates that 16 of the top 20 weight matrices with highest pruning pattern divergences belong to self-attention output projections (self_attn.o_proj), pinpointing attention output aggregation as the primary locus of bias propagation in LLMs.
  • Broad Safety Synergies: Supplementary evaluations on RealToxicityPrompts and HarmBench (Table 16) reveal that Debias-SparseGPT consistently decreases the unsafe response rate compared to SparseGPT (e.g., reducing HarmBench unsafe rate from 0.265 to 0.190 on LLaMA-3.1-8B and from 0.510 to 0.425 on Vicuna-1.5-7B).

Limitations & Future Work

  • Monolingual English Scope: All calibration datasets and evaluation benchmarks are restricted to English, leaving cross-lingual bias mitigation under compression unverified.
  • Calibration Sensitivity in Aggressive Regimes: Under strict 2:4 structured sparsity, the method requires augmentation with rich multi-turn conversational data (UltraChat) to prevent perplexity spikes; theoretical bounds on calibration sample complexity remain to be established.
  • Synergy with Quantization: The paper focuses on weight pruning; extending the contrastive Hessian formulation to joint post-training quantization (PTQ) and parameter-efficient fine-tuning (PEFT) represents an open avenue.
  • vs. SparseGPT: SparseGPT relies exclusively on standard input covariances, which exacerbates demographic bias during weight compensation; Debias-SparseGPT incorporates contrastive difference terms, significantly boosting fairness without adding computational overhead.
  • vs. Wanda / Wanda++: Wanda scores parameter significance solely through weight magnitude and input activation norms without global curvature compensation; Debias-SparseGPT maintains full second-order inverse Hessian compensation, achieving superior perplexity and safety.
  • vs. Post-Hoc Model Editing (BiasEdit / FairSteer): Inference-time steering and localized model editing incur recurring runtime latency or require parameter fine-tuning; Debias-SparseGPT permanently embeds fairness into the sparse weight topology during one-shot compression.

Rating

  • Novelty: โญโญโญโญ [Elegant integration of counterfactual contrastive outer products into the classical OBS inverse Hessian framework]
  • Experimental Thoroughness: โญโญโญโญโญ [Evaluated across nine open models, four sparsity patterns, uncertainty profiling, and carbon throughput metrics]
  • Writing Quality: โญโญโญโญโญ [Clear mathematical derivations, cohesive narrative flow, and well-structured comparative tables]
  • Value: โญโญโญโญโญ [Directly resolves a critical roadblock in deploying accelerated 2:4 sparse LLMs for trustworthy real-world applications]