Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivity¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: LLM quantization, mixed-precision quantization, calibration-free quantization, mechanistic interpretability, singular value decomposition
TL;DR¶
Addressing the flaw of existing calibration-free mixed-precision quantization methods that treat modules uniformly and rely solely on scalar outlier statistics, NSDS mechanistically decomposes layers into Detectors and Writers, estimates sensitivity through both Numerical Vulnerability and role-aware Structural Expressiveness, and aggregates scores via MAD-Sigmoid and Soft-OR for robust, data-free bit allocation.
Background & Motivation¶
Large Language Models have demonstrated exceptional proficiency across complex reasoning, multilingual comprehension, and domain-specialized problem solving, yet their tremendous parameter scale imposes formidable memory and computational bottlenecks upon real-world deployment. Post-Training Quantization (PTQ) has emerged as an indispensable compression paradigm that circumvents resource-heavy re-training. Nonetheless, when pushing models into extreme low-bit regimes (such as sub-4-bit averages), standard uniform quantization triggers catastrophic performance collapse and perplexity surges. Layer-wise Mixed-Precision Quantization (LMPQ) addresses this deficiency by strategically granting higher bit-widths (e.g., 4-bit) to sensitive layers while assigning fewer bits (e.g., 2-bit) to less critical layers, thus preserving hardware-friendly tensor contiguity and GPU kernel regularity unlike irregular group-wise quantization schemes.
Although criterion-based sensitivity estimation offers high efficiency, existing methods exhibit severe limitations depending on whether they require calibration data. Calibration-based schemes demand expensive forward-backward passes and inevitably suffer from catastrophic overfitting to specific calibration datasets, which destabilizes generalization across diverse downstream tasks. Conversely, existing calibration-free methods suffer from two critical conceptual oversights: first, they homogenously lump all intra-layer weight matrices together without considering the functional heterogeneity of distinct operations; second, they rely on single-faceted numerical statistics, such as weight outliers or simple parameter variance, completely ignoring the structural information capacity of the weight subspace. Empirical inspections reveal that layers exhibiting minimal numerical outliers can still suffer disastrous perplexity degradation if their underlying structural expressiveness is damaged during quantization.
To transcend the blind spots of conventional outlier-based metrics, this work approaches quantization sensitivity through the lens of mechanistic interpretability to analyze how functional subcomponents process information. The authors uncover that layer sensitivity is governed by two orthogonal yet complementary properties: parameter numerical distribution tails and subspace operational expressiveness. Core idea: This paper proposes NSDS, a calibration-free LMPQ framework that mechanistically decomposes each Transformer layer into Detectors and Writers, quantifies sensitivity through excess kurtosis (Numerical Vulnerability) and vocabulary-projected spectral capacity (Structural Expressiveness), and fuses them via MAD-Sigmoid normalization and Soft-OR aggregation to guide closed-form data-free bit allocation.
Method¶
Overall Architecture¶
NSDS operates as an entirely data-free sensitivity evaluation and bit-allocation framework for Transformer layer-wise mixed-precision quantization. The pipeline proceeds through four sequential phases: first, mechanistic layer decomposition partitions multi-head attention and feed-forward sublayers into feature-activating Detectors and residual-writing Writers; second, dual-sensitivity estimation quantifies Numerical Vulnerability (NV) via excess kurtosis and Structural Expressiveness (SE) via truncated SVD spectral energy reweighted by component operational roles; third, robust score aggregation aligns divergent metric scales through Median Absolute Deviation (MAD) standardization and fuses them using a Soft-OR operator to emphasize peak component vulnerability; fourth, deterministic data-free bit allocation solves a closed-form budget equation to greedily assign 4-bit precision to the most sensitive layers and 2-bit precision to the remaining layers under a specified average bit budget.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Pre-trained Model Weight Matrices"] --> B["Stage 1: Mechanistic Role Decomposition<br/>Partition into Detectors and Writers"]
B --> C["Stage 2: Dual-Sensitivity Estimation<br/>Excess Kurtosis for NV + Role-Aware SVD Spectral Entropy for SE"]
C --> D["Stage 3: Robust Score Aggregation<br/>MAD-Sigmoid Standardization + Soft-OR Peak Vulnerability Fusion"]
D --> E["Stage 4: Data-Free Closed-Form Bit Allocation<br/>Rank-Ordered Greedy 4-bit / 2-bit Precision Assignment"]
E --> F["Mixed-Precision Quantized Model"]
Key Designs¶
1. Mechanistic Role Decomposition: Decoupling Functional Information Flows Prior calibration-free sensitivity estimators treat attention projections and feed-forward blocks indiscriminately, masking the operational heterogeneity of weights in modifying the residual stream. Drawing from mechanistic interpretability circuits, NSDS decomposes each Transformer layer into two operational roles: Detectors and Writers. In Multi-Head Attention, the combined query-key matrix \(W_{QK}^{(l,h)} = W_Q^{(l,h)} (W_K^{(l,h)})^T\) detects attention patterns determining which tokens should attend to one another, acting as a Detector; conversely, the output-value matrix \(W_{OV}^{(l,h)} = W_V^{(l,h)} W_O^{(l,h)}\) extracts and transforms attended information and writes it into the residual stream, functioning as a Writer. In Feed-Forward Networks (including SwiGLU architectures), the up-projection \(W_{\text{in}}\) (and gate projection \(W_{\text{gate}}\)) detects relevant neuron feature activations, operating as Detectors; the down-projection \(W_{\text{out}}\) subsequently writes the activated features back into the residual stream, operating as a Writer. This role distinction grounds sensitivity estimation in actual circuit dynamics.
2. Dual-Sensitivity Estimation: Unifying Numerical Outliers and Subspace Capacity Conventional heuristics rely exclusively on scalar distributions, routinely missing layers whose weights are numerically uniform but structurally fragile in low-rank manifolds. NSDS models sensitivity along two orthogonal dimensions: Numerical Vulnerability (NV) and Structural Expressiveness (SE). NV assesses the heavy-tailed outlier distribution by computing excess kurtosis over the flattened weight matrix \(w\): $$ \kappa(w) = \frac{\mathbb{E}[(w - \mu)^4]}{(\mathbb{E}[(w - \mu)^2])^2} - 3 $$ A high excess kurtosis signals severe outlier concentrations that aggressively stretch the quantization grid, inducing drastic quantization noise. Meanwhile, SE quantifies representational energy and entropy by performing SVD on the weight matrix (\(W = U \Sigma V^T\)) truncated to the top singular components accounting for 90% cumulative variance to filter tail noise. The raw structural capacity \(E_{\text{base}} = \|\sigma\|_1 \times \exp(H(\sigma))\) is then modulated using operational role reweighting: Detectors rely on input singular vectors \(V\) to identify residual patterns, reweighted by the sub-linear transformed excess kurtosis of singular vectors \(\beta_{DS}^{(i)} = \log(1 + \text{ReLU}(\kappa(v_i)))\) (with Query and Key kurtosis multiplied for QK components); Writers rely on output singular vectors \(U\) to write into the residual stream, reweighted by the \(L_1\) projection norm onto the vocabulary unembedding matrix \(W_U\): \(\beta_{WD}^{(i)} = \|W_U^T u_i\|_1\). Replacing raw singular values \(\sigma_i\) with \(\sigma_i \cdot \beta^{(i)}\) produces the role-aware structural expressiveness \(E_{\text{role}}\).
3. Robust Score Aggregation: Reconciling Metric Divergence via Soft-OR Because Numerical Vulnerability and Structural Expressiveness exhibit divergent distributions, variances, and numeric scales, linear combination would cause one metric to arbitrarily suppress the other. NSDS standardizes raw scores into robust Z-scores using the Median Absolute Deviation (MAD): $$ z^{(l,c)} = \frac{r^{(l,c)} - \text{Median}(R)}{1.4826 \cdot \text{MAD}(R) + \epsilon} $$ where \(1.4826\) scales the MAD to match the standard deviation under normality. The standardized score is passed through a Sigmoid function to obtain a bounded probabilistic score \(P^{(l,c)} \in (0, 1)\). To ensure that a layer is identified as sensitive whenever either its numerical or structural dimension is endangered—rather than diluting acute vulnerabilities via arithmetic averaging—NSDS applies the Soft-OR operator: \(\text{Soft-OR}(\{P_i\}) = 1 - \prod (1 - P_i)\). Intra-layer components are first aggregated into layer-level NV (\(S_{\text{NV}}^l\)) and SE (\(S_{\text{SE}}^l\)) scores, and then merged into the final layer sensitivity metric \(S_{\text{NSDS}}^l = \text{Soft-OR}(S_{\text{NV}}^l, S_{\text{SE}}^l)\).
4. Data-Free Closed-Form Bit Allocation: Zero-Calibration Budget Mapping Given a target average bit budget \(\bar{b} \in [2, 4]\) (typically set to 3.0), the proportion of layers allocated to 4-bit precision is derived analytically under the equal-layer assumption: \(\rho = \frac{\bar{b} - 2}{4 - 2}\). The required number of 4-bit layers is given directly by \(L_4 = \text{round}(\rho \cdot L)\), with the remaining \(L_2 = L - L_4\) layers assigned to 2-bit precision. The model layers are sorted in descending order according to their unified \(S_{\text{NSDS}}^l\) scores; the top \(L_4\) layers are deterministically assigned 4 bits, and all others receive 2 bits. This procedure requires zero forward passes and zero calibration data.
Benchmark & Evaluation Protocol¶
The default quantization engine is the calibration-free HQQ framework, alongside experiments using GPTQ. Evaluations cover Llama-3.1-8B, Qwen2.5-7B, Llama-2-13B, and Qwen2.5-14B. Benchmarks span language modeling perplexity (PPL on WikiText-2 and C4, lower is better) and multi-task language reasoning accuracy (ARC-Challenge, HellaSwag, PIQA, BoolQ, WinoGrande, and TruthfulQA, higher is better). Baselines include calibration-free metrics (MSE, EWQ, ZD, KurtBoost) and calibration-based metrics (LIM, LSAQ, LLM-MQ, LieQ, SliM-LLM).
Key Experimental Results¶
Main Results¶
Under a target average bit budget of \(\bar{b} = 3\) with the default HQQ backend, Table 1 details the comparative performance on Llama-3.1-8B and Qwen2.5-7B across six reasoning tasks and two language modeling benchmarks:
| Model | Method | ARC-C (↑) | Hellaswag (↑) | PIQA (↑) | BoolQ (↑) | Winogrande (↑) | TruthfulQA (↑) | WikiText-2 (↓) | C4 (↓) |
|---|---|---|---|---|---|---|---|---|---|
| Llama-3.1-8B | FP16 Base | 57.76 | 81.97 | 80.09 | 82.11 | 77.35 | 28.40 | 6.24 | 8.95 |
| MSE | 38.73 | 64.21 | 68.58 | 60.73 | 65.82 | 23.95 | 9.22 | 12.35 | |
| EWQ | 40.17 | 67.16 | 71.49 | 64.52 | 67.43 | 23.34 | 8.38 | 11.27 | |
| ZD | 39.45 | 66.94 | 69.63 | 62.88 | 67.51 | 24.49 | 8.42 | 11.51 | |
| KurtBoost | 41.87 | 68.22 | 70.76 | 67.76 | 69.69 | 25.15 | 7.74 | 10.65 | |
| NSDS (Ours) | 43.16 | 69.67 | 73.31 | 66.89 | 72.28 | 26.43 | 7.25 | 9.97 | |
| Qwen2.5-7B | FP16 Base | 63.82 | 80.22 | 78.73 | 84.65 | 76.01 | 39.05 | 6.85 | 10.44 |
| MSE | 53.68 | 67.35 | 72.92 | 69.85 | 65.18 | 27.82 | 11.65 | 14.22 | |
| EWQ | 51.85 | 67.52 | 72.10 | 67.96 | 64.35 | 26.95 | 11.48 | 14.05 | |
| ZD | 55.82 | 70.45 | 73.01 | 72.95 | 65.25 | 28.94 | 9.23 | 12.15 | |
| KurtBoost | 54.25 | 69.92 | 72.48 | 70.40 | 65.72 | 28.35 | 9.84 | 13.68 | |
| NSDS (Ours) | 57.08 | 71.56 | 75.25 | 73.18 | 68.56 | 31.15 | 8.61 | 11.83 |
On larger-scale architectures (13B/14B), NSDS maintains decisive superiority across both perplexity and average reasoning accuracy:
| Model | Method | Reasoning Average Acc (↑) | WikiText-2 PPL (↓) | C4 PPL (↓) |
|---|---|---|---|---|
| Llama-2-13B | FP16 Base | 67.23 | 4.88 | 6.47 |
| MSE | 59.91 | 8.55 | 10.21 | |
| EWQ | 61.68 | 6.77 | 8.43 | |
| ZD | 60.99 | 7.08 | 8.89 | |
| KurtBoost | 62.08 | 6.62 | 8.41 | |
| NSDS (Ours) | 63.01 | 6.23 | 7.99 | |
| Qwen2.5-14B | FP16 Base | 73.25 | 5.29 | 9.15 |
| MSE | 64.29 | 8.93 | 12.31 | |
| EWQ | 63.36 | 9.37 | 12.88 | |
| ZD | 65.42 | 8.54 | 11.97 | |
| KurtBoost | 65.13 | 8.46 | 12.36 | |
| NSDS (Ours) | 66.15 | 8.28 | 11.65 |
Ablation Study¶
Ablation experiments evaluate the isolated contributions of Numerical Vulnerability, Structural Expressiveness, role reweighting, and the aggregation operators:
| Ablation Setting | Configuration Description | Llama-3.1-8B Avg Acc (↑) | Qwen2.5-14B Avg Acc (↑) | Perplexity Trend |
|---|---|---|---|---|
| Full NSDS | Unified dual-sensitivity with MAD & Soft-OR | 58.62 | 66.15 | Lowest across all models (optimal) |
| w/o SE | Removing Structural Expressiveness (NV only) | 56.40 (-2.22) | 64.82 (-1.33) | Noticeable PPL surge across tasks |
| w/o NV | Removing Numerical Vulnerability (SE only) | 56.95 (-1.67) | 65.01 (-1.14) | Moderate PPL increase |
| w/o \(\beta_{DS}\) & \(\beta_{WD}\) | Removing role-aware SVD reweighting factors | 57.21 (-1.41) | 65.20 (-0.95) | Moderate accuracy drop |
| w/o MAD-Sigmoid & Soft-OR | Replacing with linear average pooling | 54.85 (-3.77) | 62.07 (-4.08) | Severe performance drop and severe PPL spike |
Key Findings¶
- Aggregation via MAD-Sigmoid & Soft-OR is the Most Pivotal Component: Removing the normalization and Soft-OR operator degrades accuracy by up to 4.08% on Qwen2.5-14B and triggers severe perplexity spikes. Because NV and SE originate from disparate statistical distributions, naive averaging obscures acute, localized layer vulnerabilities.
- Superior Robustness in Severe Low-Bit Regimes: Sweeping average bit budgets from 3.8 down to 2.2 reveals that competing baselines (EWQ, KurtBoost) experience sharp drops starting at 3.2 bits, whereas NSDS retains stable reasoning performance down to 2.6 bits before showing degradation.
- Cross-Model Consistency Beats Calibration-Based Methods: While calibration-based metrics exhibit erratic behavior depending on dataset alignment (e.g., LIM ranks first on Llama-3.1 but degrades to last on Qwen2.5-7B), NSDS consistently ranks within the top two across all tested model families without using any calibration tokens.
- Orthogonal Compatibility with Advanced PTQ Engines: When pairing NSDS with GPTQ instead of HQQ, model accuracy improves uniformly across all benchmarks, demonstrating that NSDS acts as a modular, plug-and-play layer selector that synergizes effectively with error-compensating quantization backends.
Highlights & Insights¶
- First Framework to Infuse Mechanistic Circuit Roles into Quantization: Instead of treating Transformer parameters as isotropic matrices, NSDS grounds layer sensitivity in operational functionality by formalizing attention and MLP components into Detectors and Writers.
- Logit Lens-Inspired Output Reweighting for Writers: Leveraging the vocabulary projection norm of singular vectors effectively measures the downstream impact of Writer components on token generation, providing an elegant, calibration-free measure of semantic agency.
- Hardware-Friendly Layer-wise Mixed Precision: By maintaining uniform bit-widths within each layer, NSDS achieves sub-3-bit compression while completely preserving tensor layout and GPU kernel regularity, avoiding the execution overhead of group-wise quantization.
Limitations & Future Work¶
- Evaluation Limited to Models up to 14B: Due to computational constraints, the empirical evaluation in the paper does not extend to 30B, 70B, or mixture-of-experts (MoE) architectures where sensitivity distributions across deep layers may exhibit different dynamics.
- Discrete Bimodal Bit Allocation: The framework is formulated around binary 2-bit/4-bit partitioning; expanding the closed-form allocation to continuous or multi-precision spaces (e.g., dynamic 2/3/4/8-bit mixes) remains an avenue for future work.
Related Work & Insights¶
- vs KurtBoost / ZD: Traditional calibration-free metrics rely purely on parameter outlier ratios or raw kurtosis, failing to capture subspace collapse when weights have low outlier counts; NSDS incorporates SVD-derived spectral entropy to protect structurally irreplaceable layers.
- vs LIM / LieQ: Calibration-based methods require forward inference over calibration tokens and are prone to distribution mismatch; NSDS achieves superior multi-task robustness purely by inspecting pre-trained weight matrices with zero calibration overhead.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Elegant integration of mechanistic interpretability principles with structural spectral analysis for quantization.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluation across 4 model families, 6 reasoning tasks, and 2 language modeling benchmarks with exhaustive ablations.
- Writing Quality: ⭐⭐⭐⭐⭐ Rigorous mathematical formulation, clear theoretical motivation, and high-density presentation.
- Value: ⭐⭐⭐⭐⭐ Delivers a practical, reliable, and completely calibration-free guideline for extreme low-bit LLM deployment.