SPARC: Scalable Path-Specific Counterfactual Fairness via Causal Conditional Independence

Conference: ECCV2026
Paper: ECCV Paper
Code: https://github.com/CASE-Lab-UMD/SPARC
Area: Causal Inference / Medical Imaging
Keywords: path-specific counterfactual fairness, conditional mutual information, adversarial masking, data utility, medical imaging fairness

TL;DR

SPARC turns path-specific counterfactual fairness for high-dimensional inputs into a label-conditioned sensitive-attribute independence objective, using mutual-information pretraining to retain diagnostic information and conditional adversarial masks to suppress unfair pathways, improving fairness on three medical datasets while retaining a predictive-performance cost relative to unconstrained models.

Background & Motivation

Sensitive information in medical imaging is not confined to patient metadata: models can infer attributes such as gender from image morphology and correlated patterns. Removing explicit sensitive fields therefore does not prevent a model from exploiting these cues for diagnosis. Conversely, demanding representations completely free of sensitive information can overcorrect, because some demographic attributes are associated with genuine disease risk. For example, when an attribute affects an image through disease status, removing that pathway may also destroy diagnostic information. Fair learning must distinguish shortcuts based on sensitive cues from recognition of genuine disease manifestations, rather than merely equalizing output rates across groups.

Path-Specific Counterfactual Fairness (PSCF) expresses this distinction through a causal graph: retain influences designated legitimate and remove only pathways designated unfair. Traditional implementations estimate potential-outcome probabilities and consequently depend on high-dimensional conditional distributions; reliable estimation becomes difficult for chest radiographs rather than a few tabular variables. Standard adversarial debiasing and Group-DRO can reduce group disparities, but their objectives do not explicitly distinguish the different causal pathways from sensitive attributes to predictions. The paper therefore seeks a sufficient condition for PSCF that can be optimized on images, rather than another disease-classification backbone.

The authors remove additional dependence between the sensitive attribute and the input after fixing the true label, then approximate this condition through discriminative adversarial training. This still requires accepting the paper's causal structure and designation of fair pathways; it does not automatically discover which influences are ethically legitimate from observational data. Core Idea: replace an intractable path-specific counterfactual effect with the conditional-independence objective that a processed image provides no additional sensitive information given the disease label, while explicitly protecting diagnostic utility.

Method

Overall Architecture

Training data contain images \(X\), true task labels \(Y\), and sensitive attributes \(S\); the model produces diagnostic predictions \(\hat{Y}\). A generator \(g\) produces an additive mask from each image, yielding \(\tilde{X}=X\oplus g(X)\); \(\oplus\) denotes iterative addition with upper and lower constraints, not XOR or a simple binary region cutout. An embedding network \(f_\theta\) encodes the processed image into \(E\), which a downstream head \(f_h\) uses for prediction. The framework comprises path-constraint reduction, mutual-information pretraining, and conditional adversarial masking in that order: the first defines the objective, and the latter two instantiate training. Pretraining protects the information relationship between inputs and representations; the predictor is then fixed while fairness masks are learned with limited damage to task performance. True labels and sensitive attributes supervise training, whereas deployed prediction needs neither the discriminator nor true diagnostic labels.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Data["Images, labels,<br/>sensitive attributes and causal graph"] --> Causal["Path-constraint reduction"]
    Causal --> Utility["Mutual-information pretraining"]
    Utility -->|Fix encoder and prediction head| Mask["Conditional adversarial masking"]
    Data -->|Training labels and sensitive supervision| Mask
    Mask --> Deploy["Deployment: generate masks,<br/>encode and predict"]

Key Designs

1. Path-constraint reduction: block sensitive influences that bypass disease labels

The paper designates \(S\to X\to\hat{Y}\) as unfair and \(S\to Y\to X\to\hat{Y}\) as the indirect pathway to preserve. Here, the direct effect is a path-specific effect relative to disease label \(Y\); it does not require a separate \(S\to\hat{Y}\) edge in the causal graph. If changing an attribute changes disease status and consequently the image, that information may remain useful for prediction; changing predictions through demographic cues while disease status is fixed is the influence to remove. Theorem 1 on page 6 states that, under the adopted causal setup, conditional independence between the input and sensitive attribute given the label suffices for a zero direct effect. Replacing the theorem's input with the processed image received by the model gives the following interpretation of the training target:

\[ I(S;\tilde{X}\mid Y)=0\quad\Longrightarrow\quad \operatorname{DE}(S)=0. \]

This constrains additional sensitive information within each label stratum, rather than requiring \(S\) and \(Y\) themselves to be independent. Consequently, even under this ideal condition, sensitive groups can have different overall positive prediction rates because their disease distributions differ. The theorem supplies a sufficient condition; it does not imply that every fair predictor must remove all label-conditioned sensitive information from its input. The condition can also be stronger than constraining only the final prediction: a classifier ignoring sensitive information does not mean the image no longer contains it. The cache omits the proof in Appendix A, and the main text explicitly simplifies exogenous influences, so this note does not generalize the claim to arbitrary structures with unobserved confounding.

2. Mutual-information pretraining: preserve useful information from processed images

If sensitive-attribute unpredictability were the only objective, a generator could destroy diagnostic image content and obtain an easy but degenerate debiasing result. SPARC first introduces data-utility pretraining so that the encoder retains rich input information even when receiving masked images. It uses a Jensen-Shannon (JSD) mutual-information estimator to distinguish genuine image-embedding pairs from negative pairs constructed by shuffling embeddings. Positive pairs share the same input, whereas negative pairs combine an image with a different sample's representation, providing a trainable dependence signal. The encoder and estimator jointly maximize this JSD objective without explicitly modeling the full image density.

The authors interpret this as protecting task information along the legitimate indirect pathway, allowing the representation to support different disease tasks through downstream heads. However, high input-representation mutual information does not itself establish that all causally legitimate diagnostic factors have been identified; it is primarily a data-utility surrogate. Subsequent mask training therefore still needs a task-preservation term, rather than relying on pretraining alone to guarantee unchanged accuracy. Page 10 also describes a utility evaluation starting from pneumonia-classification pretraining and fine-tuning the downstream head for eight other diagnostic tasks. The text cache does not reliably expose the individual task values in Figure 3, so they are not reconstructed as a precise results table.

3. Conditional adversarial masking: suppress extra sensitive information when the label is known

After pretraining, the conditional discriminator \(d_Y\) receives the processed image and true label and attempts to predict the sensitive attribute. It alternates updates with the generator: the discriminator learns to recognize the attribute more accurately, while the generator modifies images to make that recognition harder. Crucially, the discriminator knows \(Y\), so the adversarial objective targets sensitive information leaked by the image beyond the disease label. Without that conditioning, the generator could erase legitimate disease-related information in its effort to hide the attribute. This is the principal distinction from sensitive-information removal objectives that do not separate pathways.

The generator objective also includes an entropy regularizer on sensitive predictions to discourage merely flipping their labels, and a utility term protecting the original model's task accuracy. Weight \(\alpha\) controls entropy regularization, \(\beta\) controls the diagnostic-performance versus debiasing trade-off, and mask strength \(\eta\) limits input modifications. Conditional independence does not require the discriminator to guess randomly in every setting: when \(Y\) and \(S\) are correlated, the label alone can predict the attribute. The more precise goal is that the image adds no sensitive-attribute predictive information beyond the label, rather than mechanically targeting a fixed chance accuracy. Fooling a finite-capacity discriminator also does not establish that all sensitive information has disappeared, so practical training only approximates the ideal condition.

A Worked Example

Consider two chest radiographs that are both positive for pneumonia but differ in demographic appearance cues. The pathway designation requires retaining image information that genuinely supports pneumonia diagnosis while suppressing attribute cues that affect predictions independently of pneumonia status. During training, the generator first produces a bounded additive mask for each image, and the modified image passes through the encoder and diagnostic head. The conditional discriminator receives both the pneumonia label and the modified image, testing whether the latter supplies extra evidence for sensitive-attribute recognition. If such evidence remains available, the generator continues adjusting the mask; if an adjustment damages pneumonia prediction, the task-preservation term constrains that update. The representation capacity encouraged by mutual-information pretraining complements this task constraint rather than duplicating the fairness loss. At deployment, only the new radiograph passes through the generator, encoder, and diagnostic head; its true pneumonia label is not supplied to the prediction pipeline. This example explains the mechanism rather than reporting an additional paired-patient experiment, and it does not imply that a heatmap alone identifies any anatomical region as inherently biased.

Loss & Training

The text extraction of equations (9) and (12) through (14) loses some operators and term names, preventing reliable reconstruction of the authors' exact training loss. This note therefore preserves the optimization roles supported by adjacent prose instead of supplying an apparently complete but unverified formula. JSD pretraining jointly optimizes the encoder and mutual-information estimator; Figure 2 states that the deployed encoder and prediction head are fixed after pretraining. Fairness optimization alternates generator and conditional-discriminator updates, setting \(\alpha\) and \(\beta\) to zero during discriminator updates. Settings stated for subsequent experiments include \(\eta=0.2\) and \(\alpha=1\), but this does not justify treating all ablation tables as identical configurations. The implementation avoids high-dimensional density estimation through discriminative learning, without explicitly generating two counterfactual worlds and calculating potential-outcome probabilities for every image. Theorem 2 on page 6 further states that input-level conditional independence suffices for prediction-level Equalized Odds. Output fairness metrics are therefore useful monitors, but small output disparities do not conversely prove input independence or a zero direct effect.

Key Experimental Results

Main Results

Table 1 on page 11 compares a shared ResNet50 architecture on MIMIC-CXR, CheXpert, and TCGA-LUAD; the selection below retains Vanilla, Group-DRO, and SPARC. The main experiments use gender as the sensitive attribute, while Table 3 separately evaluates binary age with a threshold of 60 years. ACC and AUC are percentages; EOp is the absolute difference in true positive rates between groups, and DP is the absolute difference in their positive prediction rates. The paper's EOd is not the commonly used maximum TPR/FPR gap, but an information-theoretic quantity:

\[ \operatorname{EOd}=I(S;\hat{Y}\mid Y). \]

The \(10^{-2}\) and \(10^{-3}\) headers preserve the source scaling, and lower is better for all fairness metrics; the source's “±” values are retained, but their statistical meaning and number of runs are not specified in the cached main text.

Dataset	Method	ACC (%) ↑	AUC (%) ↑	EOp (\(10^{-2}\)) ↓	EOd (\(10^{-3}\)) ↓	DP (\(10^{-2}\)) ↓
MIMIC-CXR	Vanilla	85.46 ±0.49	63.85 ±0.28	5.10 ±0.13	10.59 ±0.23	6.31 ±0.14
MIMIC-CXR	Group-DRO	83.20 ±0.39	61.45 ±0.27	2.90 ±0.08	4.40 ±0.13	3.62 ±0.09
MIMIC-CXR	SPARC	84.26 ±0.38	62.10 ±0.30	2.02 ±0.09	2.25 ±0.10	3.53 ±0.08
CheXpert	Vanilla	85.87 ±0.47	65.93 ±0.31	5.05 ±0.10	11.18 ±0.26	8.86 ±0.16
CheXpert	Group-DRO	83.55 ±0.36	62.80 ±0.29	1.92 ±0.07	3.07 ±0.15	2.36 ±0.08
CheXpert	SPARC	85.80 ±0.42	64.99 ±0.32	0.98 ±0.05	1.20 ±0.08	1.16 ±0.05
TCGA-LUAD	Vanilla	98.47 ±0.17	98.58 ±0.13	3.57 ±0.09	7.21 ±0.15	11.94 ±0.21
TCGA-LUAD	Group-DRO	96.65 ±0.24	97.05 ±0.18	1.78 ±0.08	3.12 ±0.10	7.30 ±0.12
TCGA-LUAD	SPARC	97.85 ±0.20	98.20 ±0.15	1.39 ±0.07	1.46 ±0.08	6.89 ±0.10

Across these datasets, SPARC improves ACC, AUC, and all three fairness metrics relative to Group-DRO, but its ACC and AUC remain below Vanilla. For example, MIMIC-CXR EOd falls from \(10.59\times10^{-3}\) to \(2.25\times10^{-3}\) while ACC falls from 85.46% to 84.26%. The appropriate conclusion is a more favorable performance-fairness trade-off, not a cost-free improvement on every axis.

Ablation Study

The MIMIC-CXR mask-strength ablation in Table 4 on page 14 directly tests whether stronger perturbations necessarily improve fairness; metric units match the main table.

Mask strength \(\eta\)	ACC (%) ↑	AUC (%) ↑	EOp (\(10^{-2}\)) ↓	EOd (\(10^{-3}\)) ↓	DP (\(10^{-2}\)) ↓
0	85.46	63.85	5.10	10.59	6.31
0.1	84.75	62.90	3.48	3.98	3.75
0.2	84.26	62.10	2.02	2.25	3.53
0.3	81.00	60.27	2.25	2.33	3.72
0.4	80.70	59.95	2.67	2.59	4.50

Beyond \(\eta=0.2\), increasing the perturbation further degrades predictive performance while worsening all three fairness metrics, showing that input destruction is not a substitute for effective debiasing. For CheXpert at \(\eta=0.2\), Table 4 reports ACC 84.59% and EOd \(3.68\times10^{-3}\), unlike the main table's 85.80% and \(1.20\times10^{-3}\); the main text does not explain the configuration difference sufficiently, so these results are not merged.

Key Findings

• In Table 2 on page 12, ViT raises CheXpert AUC to 67.53%, but its EOd of \(2.51\times10^{-3}\) is worse than ResNet50's \(1.20\times10^{-3}\); a stronger backbone does not automatically imply better fairness.
• In Table 6 on page 14, MIMIC-CXR with \(\beta=0\) achieves EOd \(0.98\times10^{-3}\) but AUC only 54.65%, demonstrating why utility must be reported alongside fairness to exclude degenerate predictions.
• GradCAM separation supplies qualitative evidence for the authors' debiasing interpretation, but non-overlapping heatmaps do not independently prove elimination of direct effects; the cited synthetic experiment with computable direct effects is in the unavailable Appendix D.2.

Highlights & Insights

• Scalability comes primarily from replacing the objective: rather than estimating a complete high-dimensional counterfactual distribution, the method trains a label-conditioned sensitive-attribute discriminator. This explains its contribution better than emphasizing the generator architecture alone.
• Separating attribute conditioning from complete attribute-information removal allows legitimate task-related correlations to remain. Giving the discriminator access to labels is a design choice with causal significance.
• Separating data utility from fairness optimization constrains the degenerate solution of destroying the image. The strength ablation also shows that this constraint cannot be replaced by arbitrarily large perturbations.

Limitations & Future Work

• The authors adopt a simplified causal graph and identify more complex structures and richer attribute definitions as future directions. Unobserved factors and biased label-generation processes may affect the applicability of the sufficient condition.
• Treating influences through \(Y\) as legitimate is a modeling judgment, not a medical-ethics theorem; if diagnostic labels reflect unequal access to care or annotation bias, label conditioning can preserve illegitimate mechanisms.
• Finite discriminator capacity, finite samples, and incomplete optimization separate empirical adversarial success from exact conditional independence; observational metrics cannot close that guarantee gap.
• The cache contains only the main text and references, omitting Appendices A, C, and D, so full proofs, data splits, tuning protocols, synthetic SCM validation, and multi-attribute extension details cannot be verified.
• The TCGA-LUAD dataset description lists genomic and clinical data, whereas experiments use ResNet50 and image heatmaps; the exact image input, task, and split are insufficiently described in the available cache.
• The prose accompanying Table 5 says performance gradually decreases as \(\alpha\) increases, but its values are not monotonic; together with the CheXpert cross-table discrepancy, these remain reproduction questions rather than inconsistencies silently corrected by this note.

Related Work & Insights

• Compared with Counterfactual Fairness: the objective of Kusner et al. removes the overall counterfactual influence of sensitive attributes; SPARC adopts PSCF's pathway distinction and preserves influences mediated by the task label.
• Compared with PSCF by Chiappa and Gillam: the paper inherits the separation of fair and unfair paths, with its main contribution being the replacement of high-dimensional potential-outcome estimation by conditional independence and a trainable implementation.
• Compared with AD and FAAP: all use adversarial ideas, but SPARC conditions its discriminator on true labels and interprets masks through a path-specific objective; its advantage depends on the causal structure being appropriate, not on adversarial training alone.
• Research direction: independent sensitive-information probes, a label-only discriminator baseline, and tests under different label biases could separate discriminator failure, improved conditional independence, and reduced genuine path effects; these are reader proposals, not reported experiments.

Rating

• Novelty: 4/5. Connecting high-dimensional PSCF optimization with conditional independence has clear methodological value, while the adversarial and mutual-information components reuse established ideas.
• Experimental Thoroughness: 3/5. Three datasets, backbone comparisons, and parameter ablations provide coverage, but proof and synthetic-validation appendices are unavailable, and cross-table configurations and reproduction details need checking.
• Writing Quality: 3/5. The central narrative is clear, but some summaries exceed the table trends, and conditional independence, observable metrics, and causal guarantees require sharper separation.
• Value: 4/5. The method offers an actionable causal-fairness training approach for high-dimensional medical inputs, provided labels, pathway assumptions, and utility costs are carefully examined.