OCGQuant: Outlier-Companion Grouping for NVFP4 Quantization¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/Eshamont/OCGQuant
Area: Model Compression
Keywords: NVFP4 Quantization, Activation Outliers, Companion Grouping, Collateral Error, PTQ
TL;DR¶
Addressing the challenge where activation outliers dominate block scales in native NVFP4 microscaling formats, OCGQuant introduces Outlier-Companion Grouping to adaptively pair outlier channels with low-magnitude companion channels, eliminating collateral quantization error while sustaining RTN-level prefill speedups via fused kernel optimization.
Background & Motivation¶
The operational cost of deploying large language models (LLMs) across high-throughput cloud clusters and resource-constrained edge hardware has intensified the demand for ultra-low-bit post-training quantization (PTQ). With recent GPU architectures introducing native hardware support for microscaling floating-point arithmetic, the NVFP4 format has emerged as an efficient standard for native sub-byte inference. NVFP4 stores tensor elements in non-uniform FP4-E2M1, assigns an FP8-E4M3 scaling factor to every contiguous block of 16 elements, and applies a tensor-wide FP32 scale to expand the dynamic range. Compared with conventional uniform INT4 quantization, NVFP4 provides a finer representation grid near zero and localized adaptive dynamic range. However, when performing low-bit weight-activation quantization, activation outliers remain the primary obstacle causing catastrophic performance degradation.
Existing outlier mitigation techniques designed for integer quantization or coarse-grained scaling exhibit severe design mismatches when directly transferred to NVFP4. Mixed-precision approaches such as LLM.int8() and Atom retain unquantized or higher-precision calculation branches, introducing heterogeneous memory access and kernel dispatch overhead. Transformation-based rotation methods like QuaRot utilize orthogonal or Hadamard matrices to disperse outlier energy across all channels; yet in microscaling formats, this global mixing disrupts the natural sparsity of local blocks and frequently amplifies overall reconstruction error. Meanwhile, NVFP4-tailored solutions such as ARCQuant introduce augmented residual channels to compensate for outlier truncation, which inevitably incurs extra runtime memory bandwidth and arithmetic operations. The root tension stems from the structural granularity of NVFP4 itself: every contiguous group of 16 channels must share a single FP8 block scale. When an extreme outlier emerges within a block, its magnitude dictates the entire block scale, forcing the remaining normal-magnitude channels to be scaled by this inflated factor and rounded onto coarse, widely spaced FP4 grid intervals.
To break this scale-dominance bottleneck, this work revisits NVFP4 from the perspective of native channel grouping and block composition: rather than suppressing outliers or relying on auxiliary compensation paths, why not intelligently reconfigure the channel composition within each 16-element block? If an outlier is deliberately grouped alongside companion channels whose magnitudes are near zero, the shared large scale will not distort the companions because their values comfortably fall within the dense zero-adjacent FP4 codebook, thereby shielding ordinary informational channels from collateral degradation. Core idea: formalize the reducible error imposed on non-maximal block elements as Collateral Quantization Error, and propose Outlier-Companion Grouping (OCG) to adaptively pair outlier channels with low-RMS companion channels, coupled with offline GPTQ weight reconstruction and fused kernel scheduling to achieve near-FP16 fidelity with zero online runtime overhead.
Method¶
Overall Architecture¶
The OCGQuant execution pipeline coordinates three interconnected phases: offline calibration and channel grouping, offline weight permutation and reconstruction, and online fused kernel inference. First, layer-wise activation statistics are profiled over a lightweight calibration set to extract the root-mean-square (RMS) magnitude of each channel. Utilizing the persistent channel locality of LLM activations, OCG identifies extreme outlier candidates and low-magnitude companion channels, constructing a global channel permutation matrix \(P\). Second, the weight matrices undergo identical static channel permutation \(WP\), followed by offline second-order GPTQ reconstruction in the permuted space to absorb weight-side quantization residuals without adding runtime latency. Finally, during online serving, channel reordering is completely absorbed into a fused RMSNorm and NVFP4 activation quantization kernel, utilizing a contiguous reorder-segment fast path to eliminate global memory round-trips.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400, 'subGraphTitleMargin': {'top': 8, 'bottom': 16}}}}%%
flowchart TD
A["Calibration Activation Input<br/>X ∈ R^(N×C)"] --> B["Outlier-Companion Grouping<br/>Pair 1 outlier channel with 15 low-RMS companions"]
B --> C["Adaptive Outlier-Companion Packing<br/>Dynamic margin stopping rule preserves contiguous segments"]
C --> D["Weight Permutation and Offline GPTQ<br/>Minimize reconstruction error in WP space"]
D --> E["Fused Kernel and Contiguous Fast Path<br/>Single-kernel RMSNorm + Reorder + NVFP4 Quantization"]
E --> F["Low-Error NVFP4 Packed Output<br/>Dispatched to Tensor Core GEMM"]
Key Designs¶
1. Outlier-Companion Grouping: Mitigate Scale Domination -> Pair Extreme Channels with Near-Zero Companions -> Minimize Collateral Quantization Error In standard contiguous channel layouts, arbitrary 16-element blocks frequently group severe activation outliers together with ordinary informational activations. For any quantization block \(B\) with group size \(G=16\), let \(m = \arg\max_{j\in B} |x_j|\) denote the maximum-magnitude element that dictates the effective dequantization block scale \(s_B\). The remaining \(G-1\) values must be quantized under this enlarged scale despite having much smaller magnitudes. The paper formalizes this induced distortion as Collateral Quantization Error: $\(E_{\text{col}}(B, m) = \sum_{j\in B, j\neq m} \left( x_j - s_B \left\lfloor \frac{x_j}{s_B} \right\rceil_{\text{FP4}} \right)^2\)$ When ordinary values are quantized under an outlier-induced scale, their normalized values fall into coarse rounding regions with wide quantization steps. OCG addresses this by constructing specialized blocks \(B_o = F_o \cup \{o\}\), each pairing one outlier candidate \(o\) with \(G-1=15\) companion channels \(F_o\) exhibiting the smallest RMS magnitudes across the unassigned channel pool \(U\): $\(F_o = \arg\min_{F \subseteq U \setminus \{o\}, |F|=G-1} \sum_{f \in F} r_f^2\)$ Because these companion channels maintain values tightly concentrated around zero, their scaled representations fall safely into the dense zero-adjacent codebook of FP4-E2M1 (values in \(\{0, \pm0.5, \pm1\}\)), bounding their absolute reconstruction error to near-zero levels. As proven in Theorem 1, minimizing the sum of squared companion RMS values minimizes the probabilistic upper bound of collateral error exceedance, effectively isolating outliers and shielding ordinary channels.
2. Adaptive Outlier-Companion Packing: Prevent Over-Grouping -> Dynamic Margin Stopping Criterion -> Preserve Natural Contiguity for Ordinary Channels Different transformer layers exhibit vastly different activation profiles: down-projection layers in MLP blocks often contain massive outliers spanning thousands of units, whereas attention projections display milder distributions. Forcibly permuting all channels across all layers yields diminishing accuracy gains while destroying memory layout locality. OCG therefore incorporates an adaptive stopping mechanism. Before packing begins, the \(G-1\) lowest-RMS channels in the initial pool are designated as a reference companion set \(F_{\text{ref}}\). For each successive top outlier candidate \(o\), a margin metric is evaluated: $\(\Delta(o) = r_o^2 - \sum_{f \in F_{\text{ref}}} r_f^2\)$ If \(\Delta(o) > 0\), the candidate's magnitude is confirmed to dominate the local block scale significantly, and an outlier-companion block \(B_o\) is formed and removed from \(U\). The moment \(\Delta(o) \le 0\), the remaining channels no longer exhibit severe scale disparity, and the packing loop terminates immediately. The remaining unassigned channels preserve their original relative physical order. This selective packing confines permutations strictly to pathological outlier channels while keeping the majority of regular channels in contiguous order.
3. Weight Permutation and Offline GPTQ Compensation: Reordered Channel Misalignment -> Symmetrical Weight Permutation with Hessian Reconstruction -> Zero-Overhead Weight Error Absorption To maintain the mathematical equivalence of linear transformations under activation channel permutation \(P\) (\(XW^\top = XP P^\top W^\top = (XP)(WP)^\top\)), the corresponding weight matrices must undergo identical column permutation \(W' = WP\). However, applying standard round-to-nearest (RTN) quantization to the permuted weights degrades performance due to altered cross-channel correlation structures. OCGQuant resolves this by treating the permuted weight matrix \(W'\) as the optimization target in an offline GPTQ reconstruction routine. Using inverse Hessian information computed over calibration data, GPTQ sequentially compensates for weight-side quantization errors in the exact channel order dictated by OCG. This calibration and compensation phase runs entirely offline (requiring only ~5 minutes per 8B model) and adds zero latency or memory overhead to online serving.
Quantization Grouping and Hardware Scheduling Strategy¶
In native NVFP4 inference on modern GPU architectures, Tensor Cores fetch and dequantize 16-element vectors via contiguous hardware micro-scaling logic. A naive runtime implementation that explicitly materializes the reordered activation tensor via a separate torch.gather kernel would induce substantial DRAM round-trip memory traffic and kernel launch latency, easily eroding the computational speedup of low-bit execution. OCGQuant overcomes this through two hardware-conscious scheduling strategies:
First, it deploys a Fused CUDA Kernel. The entry normalization (LayerNorm or RMSNorm), online channel index gathering, and NVFP4 block quantization are consolidated into a single unified kernel. Inside thread-block SRAM and register files, normalized activation elements are gathered according to the precomputed permutation table, dynamically evaluating the 16-element maximum absolute value, calculating the FP8-E4M3 block scale, and packing four 4-bit elements into standard sub-byte byte containers. The intermediate permuted full-precision tensor is never written to global memory.
Second, it introduces a Contiguous Reorder-segment Fast Path. Benefiting from the adaptive packing design, extensive subsets of ordinary channels maintain their pristine relative ordering. For quantization blocks composed of such consecutive channels, the CUDA kernel bypasses per-element gather lookup tables entirely. Instead, threads load a single base channel pointer and increment index offsets consecutively (base + offset), drastically reducing instruction counts, register pressure, and divergent memory transactions across warps. Non-contiguous outlier-companion blocks seamlessly fall back to the gather path, maximizing hardware throughput.
Key Experimental Results¶
Main Results¶
Evaluation was conducted on a single NVIDIA RTX 5090 GPU across Llama3 (Llama3.1-8B, Llama3.2-3B, Llama3.2-3B-Instruct) and Qwen3 (Qwen3-1.7B, Qwen3-8B, Qwen3-32B) model families using native NVFP4 quantization. Calibration was standardized using 128 sequences of 2048 tokens sampled from the WikiText-2 training split. Evaluation metrics include WikiText-2 test perplexity (lower is better) and zero-shot accuracy averaged across six LM-Eval common-sense reasoning benchmarks (ARC-Easy, ARC-Challenge, BoolQ, HellaSwag, LAMBADA, PIQA).
Table 1: Perplexity comparison on WikiText-2 under NVFP4 quantization (QuaRot is unavailable on Qwen3-32B due to unsupported Hadamard matrix dimensions) | Method | Llama3.2-3B | Llama3.1-8B | Qwen3-1.7B | Qwen3-8B | Qwen3-32B | | :--- | :---: | :---: | :---: | :---: | :---: | | FP16 (Full Precision) | 7.81 | 6.24 | 16.67 | 9.72 | 7.60 | | RTN (Baseline NVFP4) | 8.62 | 6.95 | 19.25 | 10.07 | 8.01 | | SmoothQuant | 8.57 | 6.88 | 17.45 | 10.10 | 7.89 | | QuaRot | 8.92 | 6.92 | 18.12 | 10.25 | - | | GPTQ | 10.48 | 6.79 | 18.46 | 10.10 | 7.86 | | MR-GPTQ | 8.50 | 6.86 | 18.70 | 10.27 | 7.96 | | ARCQuant | 8.54 | 6.87 | 17.45 | 10.35 | 7.90 | | OCGQuant (Ours) | 8.37 | 6.75 | 16.71 | 9.90 | 7.82 |
Table 2: Average zero-shot accuracy (%) across ARC-C, ARC-E, BoolQ, HellaSwag, LAMBADA, and PIQA under NVFP4 quantization | Method | Llama3.2-3B | Llama3.1-8B | Qwen3-1.7B | Qwen3-8B | | :--- | :---: | :---: | :---: | :---: | | FP16 | 68.75 | 75.42 | 62.29 | 73.53 | | RTN | 66.16 | 72.65 | 56.82 | 71.73 | | SmoothQuant | 66.07 | 73.29 | 57.22 | 72.21 | | QuaRot | 64.07 | 72.74 | 56.20 | 71.09 | | GPTQ | 65.20 | 72.57 | 58.26 | 72.44 | | MR-GPTQ | 66.46 | 72.84 | 55.90 | 71.75 | | ARCQuant | 65.86 | 73.30 | 57.53 | 71.71 | | OCGQuant (Ours) | 67.12 | 73.54 | 58.63 | 72.56 |
On instruction-tuned reasoning tasks evaluated with Llama3.2-3B-Instruct, OCGQuant achieves 56.96% on 5-shot MMLU (vs. 56.66% for RTN and 56.83% for ARCQuant) and 73.69% on 8-shot CoT GSM8K (vs. 71.65% for RTN and 71.87% for ARCQuant), proving that minimizing collateral error delivers substantial advantages in numerical and multi-step reasoning.
Ablation Study¶
To verify the individual contributions of each architectural module, an ablation was conducted on WikiText-2 perplexity (Table 3), accompanied by an analysis of quantization group size scalability (Table 4).
Table 3: Ablation study of OCGQuant components on WikiText-2 perplexity | Configuration | Llama3.2-3B | Qwen3-1.7B | Note | | :--- | :---: | :---: | :--- | | RTN Baseline | 8.62 | 19.25 | Standard unpermuted NVFP4 quantization | | + GPTQ | 10.48 | 18.46 | Weight-only reconstruction without activation grouping | | + OCG w/o AP | 8.64 | 17.20 | OCG without adaptive packing (all channels regrouped) | | + OCG (Activation only) | 8.63 | 17.18 | Full OCG activation permutation without weight GPTQ | | OCGQuant (Full Model) | 8.37 | 16.71 | Full OCG grouping + weight GPTQ in permuted space |
Table 4: Perplexity comparison across different group sizes under NVFP4 pseudo/real quantization | Group Size | Method | Llama3.2-3B | Llama3.1-8B | Qwen3-1.7B | Qwen3-8B | | :---: | :--- | :---: | :---: | :---: | :---: | | G=8 | RTN / OCGQuant | 8.44 / 8.21 | 6.78 / 6.63 | 18.50 / 16.91 | 9.94 / 9.93 | | G=16 (Native NVFP4) | RTN / OCGQuant | 8.62 / 8.37 | 6.95 / 6.75 | 19.25 / 16.71 | 10.07 / 9.90 | | G=32 | RTN / OCGQuant | 8.82 / 8.50 | 7.11 / 6.88 | 20.05 / 17.63 | 10.13 / 10.01 | | G=64 | RTN / OCGQuant | 9.09 / 8.72 | 7.31 / 7.04 | 19.76 / 18.12 | 10.22 / 10.21 |
Key Findings¶
- Unprecedented Recovery on Outlier-Heavy Models: On Qwen3-1.7B, where activation outliers reach extreme magnitudes of \(\sim 10^4\), standard RTN degrades perplexity severely from 16.67 to 19.25. OCGQuant restores perplexity to 16.71, nearly matching full FP16 precision and outperforming all competing PTQ methods by a wide margin.
- Synergistic Activation-Weight Co-design: On Llama3.2-3B, applying GPTQ alone degrades perplexity from 8.62 to 10.48 due to fitting unconditioned activation outliers. However, when GPTQ is executed in the OCG-permuted channel space where collateral activation errors have been neutralized, perplexity drops to the lowest score of 8.37.
- Hardware-Level Speedup with Zero Memory Inflation: In Llama3.1-8B prefill measurements with sequence length 2048, OCGQuant delivers 1.68× (batch size 1) to 2.29× (batch size 16) speedups over FP16, attaining 20,159 to 26,439 tokens/s. This represents only a negligible 1.5%–2.9% drop relative to the unpermuted RTN ceiling (20,752 to 26,838 tokens/s) while consistently outperforming ARCQuant (18,457 to 22,784 tokens/s). In decoding peak memory, OCGQuant matches RTN bit-for-bit (6.02 GB at BS=1; 10.52 GB at BS=16), requiring zero auxiliary buffers.
Highlights & Insights¶
- Novel Problem Formulation: Defining "Collateral Quantization Error" accurately pinpoints that the primary source of low-bit degradation is not the outlier itself, but the collateral damage inflicted on neighbor elements forced to share an inflated scaling factor.
- Optimal Exploration of NVFP4 Codebook Asymmetry: Intelligently pairing one large outlier with 15 near-zero channels capitalizes on the dense representation interval of FP4-E2M1 around zero, transforming a mathematical error bound into an elegant greedy grouping heuristic.
- Hardware-Algorithm Co-optimization: The adaptive packing margin maintains contiguous index blocks for the majority of regular channels, enabling the contiguous fast-path CUDA kernel to circumvent per-element gather overhead and resolving the classic tradeoff between permutation flexibility and runtime throughput.
Limitations & Future Work¶
- Author-Acknowledged Limitations: Channel RMS is employed as a lightweight heuristic proxy rather than a direct loss-minimizing metric tied to discrete FP4 rounding error or layer output sensitivity. Furthermore, the channel permutation is determined solely by activation statistics, after which weights passively follow; a joint activation-weight optimization framework could potentially yield tighter overall error bounds.
- Additional Considerations: The static channel permutation relies on the cross-token consistency and channel locality of outliers. In domains with dynamic, non-stationary outlier emergence (e.g., extremely long contexts or cross-modal vision-language processing), a static permutation might experience localized degradation.
- Future Directions: Exploring lightweight dynamic reordering techniques, extending the outlier-companion principle to low-bit KV Cache compression, and co-designing compiler-level fused kernels for next-generation microscaling architectures.
Related Work & Insights¶
- vs ARCQuant: ARCQuant compensates for activation outliers using augmented residual channels, which necessitates auxiliary computation paths and extra memory bandwidth during inference. OCGQuant operates entirely within the native 16-channel block structure of NVFP4, restructuring block composition to eliminate collateral error with zero additional online memory or compute overhead.
- vs RPTQ / Atom: RPTQ and Atom were conceived for uniform INT quantization, relying on min-max range clustering or hybrid-precision paths. Adapting RPTQ directly to NVFP4 degrades performance on most models (e.g., Llama3.1-8B PPL drops from 6.95 to 7.63), demonstrating that uniform INT grouping assumptions fail on non-uniform microscaling FP4 grids. OCGQuant specifically derives its grouping bounds for the FP4-E2M1 format.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ [Pioneering block-composition perspective for native microscaling NVFP4; elegant outlier-companion pairing]
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive evaluations across Llama3 and Qwen3 spanning 1.7B to 32B, with true hardware benchmarks on RTX 5090]
- Writing Quality: ⭐⭐⭐⭐⭐ [Mathematically rigorous probabilistic derivations paired with crystal-clear systems and kernel architecture descriptions]
- Value: ⭐⭐⭐⭐⭐ [Directly addresses low-bit serving bottlenecks on modern GPU architectures with production-ready efficiency and accuracy]