Adversarial Attack and Disturbance Detection by Hadamard-Coded Output Representations for Object Detection and Semantic Segmentation¶
Conference: ECCV 2026
Paper: Official paper page · Paper PDF
Code: https://github.com/ifnspaml/HadamardPerturbationDetection
Area: AI Safety
Keywords: Hadamard coding, output consistency, perturbation detection, semantic segmentation, object detection
TL;DR¶
HadamardNet replaces conventional class outputs with redundant codewords and uses probability-simplex projection to produce both class probabilities and an inconsistency residual, enabling single-pass perturbation detection without guaranteeing that predictions on disturbed inputs are correct.
Background & Motivation¶
Semantic segmentation and object detection systems need more than accurate predictions: they also need evidence about when an input should not be trusted. Maximum posterior probability and predictive entropy are convenient monitoring signals, but they measure confidence rather than compliance with an independently meaningful output constraint. A model can remain confident on an abnormal input, undermining detectors that expect uncertainty to increase. Comparing predictions after input transformations supplies additional evidence at the cost of repeated forward passes, while a separately trained detector can become specialized to the perturbations represented in its training data.
Hadamard output coding offers a different source of evidence: a class is represented by multiple binary properties rather than a single competing output coordinate. Earlier work explored its benefits for classification robustness, but this does not establish that it preserves competitive dense segmentation performance or already supplies a useful disturbance detector. The paper identifies an overlooked question in decoding: can the predicted soft codeword be explained by any valid class-probability distribution? If not, the correction needed to make that explanation possible could expose contradictions between individual binary predictions.
The proposed framework connects output coding, constrained probabilistic decoding, and anomaly scoring in one inference path. Pixels and objects become different instances of the same position-wise classification problem, avoiding a separate language model or repeated image-processing branch. This is a defensive reliability-monitoring study, so anomaly detection and the accuracy of the underlying perception task must be assessed separately. Core idea: treat the minimum correction from redundant predictions to a valid probability distribution as consistency evidence, then judge that evidence relative to its normal behavior on clean data.
Method¶
Overall Architecture¶
The input is an image, and the perception network predicts a soft Hadamard codeword for each pixel or object instead of directly producing softmax class probabilities. “Redundant Output Coding” changes the classification representation; “Probability-Constrained Decoding” produces legal class probabilities and a correction residual; “Conditional Consistency Scoring” aggregates local evidence into an image-level anomaly score. The outputs are the ordinary segmentation or detection prediction together with an additional reliability signal suitable for flagging suspicious inputs. For object detection, the bounding-box regression branch remains standard: the contribution concerns classification and its decoding, not a replacement of the entire detector.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input image"] --> B["Redundant Output Coding"]
B --> C["Probability-Constrained<br/>Decoding"]
C --> D["Conditional Consistency<br/>Scoring"]
C --> E["Class probabilities<br/>and task predictions"]
D --> F["Image-level anomaly flag"]
Key Designs¶
1. Redundant Output Coding: make internal constraints observable
For a task with \(S\) classes, the authors select columns of a Hadamard matrix as class codewords. The code length \(L\) is a power of two with \(L\geq S\), and distinct codewords have equal Hamming distance; the training representation converts bipolar entries into binary entries. The classification head therefore predicts an \(L\)-dimensional vector and applies sigmoid to obtain soft bit predictions. These coordinates are predictions of coding bits, not competing class probabilities whose sum is necessarily one. That distinction matters because enforcing a probability simplex at the outset would remove the same opportunity to observe violations of its constraints.
The representation can be attached to segmentation pixels or classification positions in an object detector. Its redundancy comes from expressing each class through several related bits, not from running an ensemble of perception models. A valid soft codeword can be understood as a probability-weighted mixture of class codewords, whereas independent bit predictions need not lie in the set of such mixtures. The framework uses this discrepancy as detection evidence rather than merely treating the Hadamard matrix as fixed classifier weights. It consequently needs a dedicated decoder: the largest coding coordinate has no direct class interpretation, and an arbitrary normalization would not necessarily retain a meaningful correction signal.
2. Probability-Constrained Decoding: recover probabilities without discarding the correction
The decoder first exploits Hadamard orthogonality to transform the soft codeword into an unconstrained class-space estimate. This estimate may contain negative entries or fail to sum to one, so it cannot simply be interpreted as a valid posterior distribution. The authors require the final probabilities to be nonnegative and sum to one, and seek the smallest squared correction consistent with those requirements. They reduce this problem to projection onto the probability simplex, obtaining the nearest valid probability vector in that optimization. This is a geometrically constrained recovery procedure, not merely temperature calibration of confidence scores.
The decoder also retains the class-space correction between the projected distribution and the unconstrained estimate. A large correction indicates that the predicted bits are difficult to reconcile with a legal class-probability explanation. A small correction does not establish that the prediction is correct or that the image is in distribution; it only indicates limited disagreement under this internal test. The paper distinguishes codeword-space errors from class-space errors and places the proof relating their minimization problems in the supplement. Accordingly, the class-space residual should not be presented as an exhaustive measurement of all possible perception failures.
Several mathematical expressions in the local PDF extraction have lost operators, norms, or subscripts, and the derivations in Supplements B–E are not included in the cache. This note therefore does not reconstruct the complete algebra of Equations (11)–(16) or supply an unverified implementation of the projection. The main text does establish the probability constraints, minimum-squared-correction objective, simplex projection, and dual outputs of probabilities and residuals. Those statements support the mechanism described here, but not an independent line-by-line verification of the full proof.
3. Conditional Consistency Scoring: distinguish ordinary difficulty from abnormal residual behavior
The simplest image-level score averages the \(L_1\) norm of the correction vector over all pixels or objects. It needs no additional regression model, but difficult regions in clean images can also produce substantial inconsistency. A fixed threshold on residual magnitude can therefore flag normal images merely because they contain many low-confidence predictions. The authors introduce two conditional alternatives using the maximum class probability and class-distribution entropy as descriptors of local confidence. Entropy becomes contextual information rather than the sole anomaly signal.
The regression alternative learns the expected residual norm at a given confidence state using clean data. At inference, it compares observed and predicted residual magnitudes, aggregates local discrepancies into an image-level regression error, and measures the absolute deviation from the mean error on clean training images. The relevant evidence is an unusual departure from the normal relationship, rather than the assumption that a larger raw residual is always more suspicious. The extracted expression for the image-level regression error is also incomplete, so its normalization details are not reconstructed here. The regression-network architecture is deferred to Supplement G; layer counts and hidden dimensions cannot be verified from the cached main paper.
The quantile alternative accounts for variation in normal residuals at the same confidence state. It predicts the conditional 85% quantile and defines the image score as the fraction of pixels or objects whose residual norm exceeds their predicted quantile threshold. This distinguishes positions with naturally variable errors from positions that are normally stable instead of comparing every residual on one absolute scale. The 85% value specifies a conditional distributional quantile: it is neither the detector's accuracy nor a guarantee that exactly 15% of positions exceed the threshold in every clean image. Dense segmentation supplies many pixels for estimating a fraction, whereas images with few detected objects provide a more discrete and potentially unstable estimate. The authors use this sample-count difference to explain why quantile scoring works less reliably for object detection.
Loss & Training¶
The perception model is trained first using an encoded-space loss between predicted soft codewords and the codeword assigned to each ground-truth class. Section 5 specifies mean squared error for the final configuration; numerical comparisons of other loss combinations are deferred to the unavailable Supplement K. Object detection retains the standard bounding-box regression loss and combines it with the encoded classification loss; the classification-weight ablation is in Supplement Q. The framework therefore requires a model trained with the new representation, rather than simply renaming a post-processing operation on an existing one-hot softmax model.
In the second stage, the trained perception model supplies probability and residual observations from the same clean training set to fit the lightweight detector. Mean regression uses mean squared error, while quantile regression uses an 85% quantile loss; the main paper reports 20 epochs with AdamW for this stage. Neither training stage requires perturbed training images, distinguishing the approach from a detector explicitly fitted to examples of a particular perturbation. Inference uses one perception forward pass plus decoding and lightweight scoring; “single-pass” does not mean the projection and regression operations have literally zero cost. This note discusses the defensive mechanism and evaluation findings, not procedures for generating perturbations.
Key Experimental Results¶
Main Results¶
The following selection uses aggregate results from main-paper Tables 1 and 4 rather than reproducing every perturbation-specific entry. Cityscapes uses SegFormer MiT-B0 trained on the training split and evaluated on validation; VOC uses Faster R-CNN trained on VOC 2007+2012 trainval and evaluated on VOC 2007 test. Global AuROC pools detection scores across perturbation types and strengths before computing area under the ROC curve; higher is better, and it is not a simple average of individual AuROCs. Pooling tests a shared score scale across configurations, but AuROC still sweeps thresholds rather than reporting detection at an already selected deployment threshold. Intervals below are the paper's 95% confidence intervals over three seeds; lower FLOPs indicate less computation.
| Dataset / source | Method | Global AuROC (%, higher is better) | FLOPs (G) |
|---|---|---|---|
| Cityscapes, Table 1 | Entropy, single-pass | 71.6 ± 0.2 | 122 |
| Cityscapes, Table 1 | Feature squeezing, three-pass | 80.7 ± 0.6 | 366 |
| Cityscapes, Table 1 | Hadamard + direct residual | 77.1 ± 0.7 | 123 |
| Cityscapes, Table 1 | Hadamard + quantile score | 80.2 ± 0.9 | 124 |
| VOC, Table 4 | Maximum posterior, single-pass | 61.2 ± 0.1 | 215 |
| VOC, Table 4 | Feature squeezing, three-pass | 69.8 ± 2.6 | 645 |
| VOC, Table 4 | Hadamard + regression score | 74.4 ± 0.4 | 215 |
On Cityscapes, quantile scoring improves over the Table 1 entropy baseline by 8.6 percentage points, but does not exceed three-pass feature squeezing; its advantage is comparable aggregate detection with substantially less computation. On VOC, regression scoring improves over maximum posterior by 13.2 points and over the three-pass method by 4.6 points. These are within-task comparisons, not evidence that the higher Cityscapes AuROC makes that task easier or its models intrinsically safer.
Task accuracy must be checked separately: Table 2 reports clean Cityscapes SegFormer mIoU of 76.4 ± 0.2 for one-hot and 76.5 ± 0.3 for Hadamard, but the corresponding BDD100K values decrease from 59.5 ± 0.3 to 57.6 ± 1.5. Table 5 reports clean VOC Faster R-CNN AP of 41.0 ± 0.4 versus 40.4 ± 0.5; AP follows the COCO definition averaging IoU thresholds from 0.50 to 0.95, not AP50. A blanket claim of improved detection with no task-performance cost would therefore be inaccurate.
Ablation Study¶
Main-paper Table 3 separates the benefits of an inconsistency signal and of Hadamard coding using DeepLabv3+ on Cityscapes validation. The intermediate configuration uses sigmoid one-hot outputs, allowing unconstrained class estimates to be inconsistent, followed by the same simplex projection and direct residual scoring.
| Representation and detector | Global AuROC (%, higher is better) | Factor tested |
|---|---|---|
| One-hot + entropy | 72.5 ± 0.1 | Conventional confidence baseline |
| One-hot + projection residual | 76.9 ± 0.3 | Allowing and measuring probability-constraint violations |
| Hadamard + projection residual | 80.4 ± 1.1 | Changing coding structure under the same scoring principle |
The first change contributes 4.4 percentage points and the second another 3.5 points, showing that the improvement cannot be attributed only to a longer output code. This supports contributions from both residual monitoring and coding structure, but does not establish an optimal code length for every class count.
Key Findings¶
- Quantile scoring is not a universal winner: it works well with SegFormer on Cityscapes, whereas Faster R-CNN on VOC benefits more from regression scoring, consistent with the different numbers of positions available for aggregation.
- Natural disturbances remain a weakness: Table 4 gives regression-score AuROCs of 50.2 ± 0.5 for Gaussian noise and 51.2 ± 2.1 for salt-and-pepper noise, close to random ranking despite the aggregate value of 74.4.
- Tables 1 and 2 report inconsistent aggregate entropy baselines for SegFormer; the 8.6-point comparison above strictly uses Table 1 rather than mixing entries across tables.
- Tables 4 and 5 also disagree on the Faster R-CNN quantile aggregate, so it is not used as a numerical headline; individual Table 5 entries likewise do not support an absolute claim that regression scoring wins for every architecture and dataset.
Highlights & Insights¶
- Output redundancy becomes an interpretable monitoring signal. The contribution is not merely separating class codewords, but checking whether several predictions admit one coherent probabilistic explanation.
- Simplex projection serves both task prediction and anomaly monitoring. The recovered probabilities provide ordinary outputs, while the correction preserves evidence not usually exposed by standard normalization.
- Conditional scoring distinguishes normal difficulty from a departure from normal relationships. Its defensive relevance is reducing confusion caused by confidence alone, not demonstrating that confidence is useless.
Limitations & Future Work¶
- The authors explicitly discuss instability of quantile scoring when there are few objects, and report some clean-task performance reductions; changing the representation is not entirely free.
- Reader assessment: detection is not correction, and the anomaly score does not certify segmentation or detection correctness or specify a complete rejection and fallback policy.
- Reader assessment: the tested perturbation suite does not establish reliability under all distribution shifts or adaptive adversaries; deployment requires separate validation of thresholds, false alarms, and real-world drift.
- Evidence boundary: the cache contains the main paper and references, not the supplements covering architecture details, proofs, some loss ablations, and additional datasets; those results have not been independently verified here.
Related Work & Insights¶
- Versus maximum posterior / entropy detection: these read final confidence, while HadamardNet also measures the correction required to legalize its output and can condition that correction on confidence.
- Versus feature squeezing: transformation-based checks compare multiple predictions, whereas this framework examines coding consistency within one output; the computational paths differ, and single-pass consistency is not always more accurate.
- Versus earlier Hadamard classification and segmentation work: the emphasis is constrained probabilistic decoding, residual exploitation, and extension to object detection, not the invention of error-correcting output coding itself.
Rating¶
- Novelty: 4/5. The combination of constrained decoding and redundancy-based monitoring is clear, while the coding family has a substantial prior history.
- Experimental Thoroughness: 3/5. Multiple architectures and a mechanism ablation support the main claim, but noise weaknesses, inconsistent table entries, and unavailable supplements limit its strength.
- Writing Quality: 3/5. The conceptual narrative is clear, though some broad conclusions exceed individual table entries and local equation extraction complicates verification.
- Value: 4/5. The method offers a low-overhead reliability signal for vision systems, with value concentrated in detection rather than certified robustness.