MGI: Member vs Generated Inference¶

Conference: ECCV 2026
arXiv: 2606.23872
Code: None
Area: Diffusion Models / AI Safety
Keywords: Membership Inference Attack, Generative Content Attribution, Privacy Auditing, Data Provenance, Autoencoder Consistency

TL;DR¶

This paper proposes a new task, Member vs Generated Inference (MGI): given a sample and a generative model, determine whether the sample is a training member of the model or an output generated by the model itself. The authors design a three-stage cascade method, DCB, which filters generated samples using autoencoder reconstruction/quantization errors, performs membership inference using the latent space generator, and finally traces the source by comparing conditional probabilities across generators. DCB significantly outperforms existing MIA and attribution methods on both Image Autoregressive (IAR) and diffusion models.

Background & Motivation¶

Modern image generative models (diffusion models, image autoregressive models) generate samples that are visually indistinguishable from real training data, and these models may unintentionally memorize and reproduce training samples. This blurs the boundary between "training members" and "model-generated outputs." Existing methods are divided into two categories: membership inference attacks (MIA, e.g., PIAR, ICAS, CLiD), which aim to determine whether a sample is in the training set but cannot distinguish generated samples, as generated samples exhibit likelihood scores in the latent space as high as real members, causing MIA to systematically misclassify generated samples as members; and image attribution methods (e.g., PRADA), which aim to identify if a sample was generated by a specific model but rely on the same likelihood signals, often misclassifying real training members as generated samples. The Key Challenge is that both types of methods focus solely on the likelihood/probability scores from the latent space generator while ignoring the distinguishing signals inherent in another critical component of the generative model: the autoencoder (AE). Generated samples undergo the complete "encoding-decoding" pipeline, leaving measurable traces under the autoencoder (lower reconstruction and quantization errors), whereas natural images do not.

This paper further extends MGI to the model derivative setting: images generated by model M1 might be scraped and used to train the next-generation model M2, forming "data circuits" that lead to model collapse. In this scenario, it is necessary to distinguish not only natural members vs. generated samples, but also three categories of generative data: "samples generated by M1 and used as training members for M2", "samples generated by M1 but not used to train M2", and "samples generated by M2 itself". Core Idea: By executing a three-stage cascade consisting of filtering generated samples via autoencoder self-consistency + performing membership inference via the latent space generator + tracing the source via cross-generator probability differences, the complete pipeline of the generative model is exploited to fundamentally resolve MGI.

Method¶

Overall Architecture¶

The core insight of the DCB method is that the complete pipeline of a generative model comprises an autoencoder $A = D \circ E$ (encoder $E$ + decoder $D$) and a latent-space generator $G$. Existing MIA and attribution methods only exploit the likelihood signals from $G$, whereas DCB utilizes both the reconstruction/quantization errors from $A$ and the conditional probabilities from $G$ to solve MGI through a three-stage cascade.

Input: Query image $x$, target generative model $M = \langle E, D, G \rangle$, and optional second model $M_2$ (for the model derivative setting). Output: The category of $x$, i.e., natural training member NM, natural non-member NN, or generated sample G (or $G_M$/$G_N$/$G'$).

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Image x"] --> B["Autoencoder Self-Consistency<br/>Compute Reconstruction + Quantization Error"]
    B -->|"Low L_A: Generated Sample"| C["Classified as G/G' (Generated)"]
    B -->|"High L_A: Natural Image"| D["Latent Space Generator<br/>MIA Score (e.g., ICAS)"]
    D -->|"High Score"| E["Classified as NM (Training Member)"]
    D -->|"Low Score"| F["Classified as NN (Non-Member)"]
    C -->|"Model Derivative Setting"| H["Cross-Generator Consistency<br/>Compare Conditional Probabilities of G1 and G2"]
    H -->|"Belongs to G1"| I["Classified as GM or GN"]
    H -->|"Belongs to G2"| J["Classified as G' (Generated by M2)"]

The pipeline consists of three stages. Stage 1 uses the autoencoder score $L_A$ to classify samples into "natural" and "generated" categories. Stage 2 applies a standard MIA to natural samples to distinguish training members from non-members. Stage 3 is triggered only in the model derivative setting, using cross-generator probability differences to further attribute the generated samples to specific models.

Key Designs¶

1. Autoencoder Self-Consistency: Separating Natural and Generated Images via Reconstruction and Quantization Errors

Generated images go through the complete "encode-quantize-decode" pipeline of the model's autoencoder and are naturally aligned with the manifold of that autoencoder. Consequently, both reconstruction and quantization errors are lower than those of natural images that have never passed through this pipeline. DCB defines the reconstruction error as $L_{Rec}(x) = \text{MSE}(x, A(x))$, which is the mean squared error between the original image and the autoencoder reconstruction. To stabilize comparison across different images, a double reconstruction ratio is introduced: $ρ_{Rec}(x) = L_{Rec}(x) / MSE(A(x), A(A(x)))$ , where the denominator serves as an image-specific baseline. If an image aligns well with the autoencoder manifold, the second reconstruction incurs almost no additional loss, stabilizing the ratio.

For image autoregressive models using VQ-VAE, a quantization error is additionally defined as $L_Q(x) = MSE(E(x), Q^{-1} o Q o E(x))$ , measuring the loss when mapping continuous latent representations to the discrete codebook and back. Generated images incur less loss during the quantization step because they were originally sampled and decoded from this codebook. The final autoencoder attribution score is set as: $L_A(x) = ρ_{Rec}(x) · L_Q(x)$ for IAR/VQ-VAE, and $L_A(x) = ρ_{Rec}(x)$ for DM/VAE. This score yields significantly lower values for generated samples, thereby achieving high-confidence natural/generated separation in Stage 1.

For IAR, an optional post-processing fine-tuning of the encoder can be performed: keeping the decoder $D$ fixed, the encoder $\hat{E}$ is fine-tuned on a disjoint set of latent features such that $\hat{E} \circ D(z) \approx z$, which enhances the stability of $L_A$.

2. Cross-Generator Consistency: Generative Source Attribution via 2D Probability Vectors and KDE

In the model derivative setting, since all generated images are decoded by the same decoder, the autoencoder score $L_A$ is identical for the three generated categories ($G_M$, $G_N$, $G'$), making them indistinguishable. DCB introduces cross-generator features: for two candidate models $M_1$ and $M_2$, a 2D feature vector is constructed as $φ(x, c) = (log P_{G1}(E(x)|c), log P_{G2}(E(x)|c))$ , representing the conditional log-probabilities of the image under the two latent-space generators. The intuition is that images generated by $G_1$ have relatively higher conditional probability under $G_1$, and vice versa.

Specifically, DCB independently generates reference sets $R_G$ and $R_{G'}$ using $M_1$ and $M_2$, respectively, and estimates class-conditional densities $\hat{p}_G(\phi)$ and $\hat{p}_{G'}(\phi)$ over the feature vectors of these references via Kernel Density Estimation (KDE). For a query image, its $\phi$ vector is evaluated under both density estimations, and the more likely source is selected. This stage enables DCB to perform fine-grained attribution among $G_M$/$G_N$/$G'$, which is impossible for pure MIA or pure autoencoder methods.

3. Three-Stage Cascade Protocol: Why the Order Cannot Be Reversed

The three stages of DCB are executed in the sequence of "filter generated samples first, classify membership second, and perform source tracing last." This order is strictly logical: Stage 1 must be performed first because if generated samples are allowed to enter Stage 2 (standard MIA), they will be systematically misclassified as training members as their latent-space likelihood is as high as actual members. This is the fundamental reason why existing MIAs fail, as proved in this paper. Stage 2 runs MIA only on the subset of natural samples filtered by Stage 1. At this point, the core assumption of "members vs. non-members" is restored, enabling standard methods like ICAS to work effectively. Stage 3 is triggered only in the model derivative setting and only after a sample is labeled as generated in Stage 1; it strictly focuses on "which model generated the sample", independent of membership status. This cascading order enables the three stages to complement each other without interference.

A Complete Example¶

Taking a VAR-d30 model derivative scenario as an example. Suppose $M_1$ (VAR) generates 5,000 images $G_M$ to fine-tune $M_2$ (sharing the same VAR architecture), and $M_2$ subsequently generates new images $G'$. Additionally, there are 1,000 images $G_N$ generated by $M_1$ but not used for training, along with natural training members NM and validation non-members NN from the original ImageNet dataset.

Stage 1: $L_A = ρ_{Rec} · L_Q$ is calculated for each image. The three types of generated images, $G_M$, $G_N$, and $G'$, exhibit significantly lower $L_A$ than NM and NN (since they all underwent encoding and decoding by the same VQ-VAE). DCB sets a threshold to flag low-$L_A$ samples as "generated" to enter Stage 3, while high-$L_A$ natural samples proceed to Stage 2.
Stage 2: ICAS scores are computed for natural samples. NM (training members) obtain significantly higher scores than NN (validation non-members), distinguishing them with TPR@1%FPR = 87.0%—restoring the performance of standard MIA in the absence of generated sample interference.
Stage 3: For samples labeled as "generated" in Stage 1, the feature vector $\phi = (\log P_{G1}, \log P_{G2})$ is computed. $G_M$ and $G_N$ (both generated by $M_1$) demonstrate higher conditional probability under $G_1$ than $G_2$, clustering together in the KDE density comparison; $G'$ (generated by $M_2$) shows a higher conditional probability under $G_2$, clustering in another group. Consequently, the TPR@1%FPR for distinguishing $G_M$/$G_N$ vs. $G'$ is 99.0%. Meanwhile, separating $G_M$ vs. $G_N$ degenerates to a standard MIA problem (since $G_M$ is a training member of $M_2$ and thus has a higher MIA score), achieving a TPR@1%FPR of 100.0%. This yields an overall average TPR@1%FPR of 95.8%.

Loss & Training¶

DCB is an inference-time method and does not involve training losses. However, the model derivative setting requires fine-tuning $M_2$: for IAR (VAR, RAR), fine-tuning is performed on 5,000 $M_1$-generated images for 5 epochs; for DM (SD1.4, SD2.1), it is fine-tuned for 20 epochs with a fixed learning rate of 1e-5 using the AdamW optimizer, fine-tuning only the latent space generator (transformer/UNet) while freezing the autoencoder weights. Stage 3 KDE uses a Gaussian kernel, with bandwidth and density thresholds detailed in Appendix J (DCB is insensitive within $\alpha=0.03\sim0.07$ and $\sigma=0.10\sim0.50$ ranges). The optional post-processing encoder fine-tuning in Stage 1 is trained on a disjoint set of latent features with the objective $L_{Inv} = \text{MSE}(\hat{E} \circ D(z), z)$.

Key Experimental Results¶

Main Results¶

Direct Training Setting (IAR): The following table reports the TPR@1%FPR metric, where DCB consistently outperforms the baselines across all models.

Method	RAR (Avg)	VAR (Avg)	LlamaGen (Avg)	Overall Avg
PIAR	54.0	53.9	8.3	38.8
ICAS	57.4	64.8	35.6	52.6
PRADA	81.3	40.6	28.4	50.1
DCB	90.8	99.2	72.4	87.4

Key details: For the NM/G (member vs. generated) comparison, DCB achieves 99.9 on RAR, 99.3 on VAR, and 100.0 on LlamaGen, whereas all baselines perform significantly worse on NM/G than on their respective NM/NN (member vs. non-member) tasks. This directly validates the failure of MIAs in the MGI task. LlamaGen's NM/NN is only 17.2 (likewise for the ICAS baseline), indicating that membership inference itself is highly challenging for this model, yet DCB still maintains 100.0 on NM/G and NN/G.

Direct Training Setting (DM): DCB also exhibits a substantial lead on diffusion models.

Method	SD1.4 (Avg)	SD2.1 (Avg)	Overall Avg
CLiD	41.5	37.9	39.7
ICAS	41.2	37.9	39.5
PRADA	0.5	0.4	0.5
DCB	78.5	77.2	77.8

PRADA fails almost completely on DM (Avg of only 0.5) because it relies strictly on the latent-space generator’s probability ratio, while likelihood estimation in diffusion models is inherently harder. DCB achieves 99.9+ (SD1.4) and 100.0 (SD2.1) on NM/G, but performs on par with CLiD/ICAS on NM/NN (35.7/31.5) since Stage 2 falls back on ICAS scores, limiting its potential to the baseline's MIA capability.

Ablation Study¶

Comparison with Strong MIAs (Model Derivative Setting): Even when LiRA and RMIA are allowed to train 5 shadow models (granting them an extra advantage), DCB still maintains an overall lead.

Model	Method	Natural vs Generated (Avg)	Among Generated (Avg)	Overall Avg
VAR	LiRA	38.0	55.2	40.0
VAR	RMIA	76.4	88.5	76.4
VAR	DCB	99.4	91.3	95.8
RAR	LiRA	40.3	49.0	40.6
RAR	RMIA	93.3	82.6	82.7
RAR	DCB	99.9	98.5	96.7

Even though RMIA has the advantage of shadow models to estimate the generated member/non-member distribution (giving it an inherent edge on $G_M$/$G_N$), DCB's reliance on pipeline-wide signals remains more accurate. LiRA is significantly weaker than DCB on "among generated" categories because a single-dimensional score distribution is far less discriminative than DCB's multi-dimensional features combined with KDE.

Robustness to Memorized Samples: Tested on 169 memorized samples of RAR-XXL with SSCD similarity > 0.7:

Metric	Delta(PIAR)	ICAS	PRADA	DCB
AUC	61.8	61.4	57.9	97.5
TPR@5%FPR	3.0	0.0	0.0	93.5

Since memorized samples reconstruct the training images almost pixel-by-pixel, normal MIA and attribution methods fail almost entirely (with TPR@5%FPR close to 0) because they assign nearly identical likelihood scores to the original and reconstructed images. DCB's autoencoder stage successfully captures the faint generative traces introduced by the encoder-decoder pipeline in reconstructed images, yielding an AUC improvement of over 35 points.

Key Findings¶

Stage 1 is the most critical module: Removing autoencoder filtering (i.e., using only ICAS for MGI) drops the NM/G TPR@1%FPR from 99.9 to 0.0 (RAR), confirming that the high-likelihood scores of generated samples are the root cause of MIA failure.
DM's NM/NN discriminability serves as the ceiling for DCB: Since DCB falls back on ICAS/CLiD in Stage 2 to handle membership inference, the highest TPR@1%FPR for NM/NN on SD1.4/SD2.1 is around 35%. This reflects the intrinsic difficulty of the membership inference task on diffusion models rather than a limitation of DCB.
Cross-architecture settings are simpler than same-architecture ones: When $M_2$ and $M_1$ have different architectures, their autoencoders are naturally different, making the separation in Stage 1 much easier (with $G_M$/$G'$ AUC $\ge 99.0$ in Appendix Table A10). Same-architecture is a harder setup because all generated images share the exact same autoencoder.
Insensitivity to KDE hyperparameters: Within the ranges of $\alpha \in [0.03, 0.07]$ and $\sigma \in [0.10, 0.50]$, the TPR@1%FPR remains stable at 97.2-99.5 for VAR-d30, and 96.9-98.4 for SD2.1.
Manageable computational overhead: Relative to a single MIA inference run, DCB only adds 16%-46% (under $M_1$ scenario) and 66%-71% (under $M_1+M_2$ scenario) overhead, requiring only forward passes and being fully parallelizable across images.

Highlights & Insights¶

Elevating the autoencoder from a "transparent pipeline" to a signal source: Previous MIA methods almost entirely overlooked the autoencoder, treating it as a lossless mapping from pixels to latent space. The core insight of DCB is that generated images are more "self-consistent" under the autoencoder—a signaling trace hiding in plain sight that is both elegant and powerful.
Ingenious logical order of the three-stage cascade: Filtering generated samples $\rightarrow$ classifying membership $\rightarrow$ tracing sources. Each stage addresses the blind spots of the previous one, rather than being a naive combination of modules. Stage 1 eliminates generated samples that violate MIA assumptions, Stage 2 restores classic MIA efficacy, and Stage 3 fills the gaps among generated subcategories. This "restore assumptions before solving" approach can be generalized to other inference tasks.
Surprising robustness in memorized sample scenarios: DCB successfully distinguishes original and reconstructed images even in near-exact pixel-by-pixel memorization samples (AUC 97.5 vs. baseline 61.8). This proves that the traces introduced by the autoencoder, while imperceptible to the naked eye, are highly measurable in latent space. This offers new directions for deepfake detection and data provenance—relying on the "fingerprints" left by the generation pipeline rather than the content itself.
Simplicity of 2D density comparison via KDE: Stage 3 avoids training complex classifiers by simply sampling reference samples from both models, executing KDE, and comparing likelihoods. This "zero-training" design makes DCB highly adaptable to new models.

Limitations & Future Work¶

NM/NN membership inference on DM is bounded by baseline MIA capabilities: DCB relies on existing MIA methods (ICAS/CLiD) in Stage 2. Consequently, when these baselines perform poorly on DM (with NM/NN TPR@1%FPR ~35%), the upper limit of DCB's overall performance is also constrained. Directly exploring better Stage 2 alternatives for DMs is a key direction.
Requires internal access to the model (white-box/gray-box): DCB requires the reconstruction outputs from the autoencoder and the log-probabilities from the latent space generator, putting it in a gray-box setting (consistent with the access assumptions of PIAR/CLiD/ICAS). DCB cannot operate in a strict black-box setting (where only API access is available).
Model derivative setting assumes known $M_1$ and $M_2$: In practice, we might only have access to $M_2$ without the full weights of $M_1$, or there may be multiple candidate models for $M_1$. The paper does not address the degraded scenario where only $M_2$ is available.
Evaluated only on class-conditional generation: Experiments are conducted on ImageNet class-conditional generation (IAR) and MS-COCO fine-tuning (DM). Open-domain text-to-image generation on large models (e.g., SDXL, Flux) is not covered, where the conditioning space is more complex, and the distributions of $L_A$ and $\phi$ may differ.
Automated threshold setting for Stage 1: Currently, the threshold requires estimating the KDE density threshold from reference generation sets. The paper does not provide a purely single-sample decision scheme independent of reference sets.

vs. PIAR / ICAS / CLiD (MIA methods): These methods rely strictly on the latent-space generator's conditional probability difference $\Delta$ for membership inference, under the assumption that "member scores > all non-member scores." DCB proves that this assumption collapses when generated samples are mixed into the non-members, as their $\Delta$ is as high as actual members. DCB restores this assumption by excluding generated samples in Stage 1.
vs. PRADA (Image Attribution): PRADA also employs probability ratios but in the opposite direction, assuming generated samples obtain the highest scores. DCB demonstrates that this assumption is also invalid because the scores of training members and generated samples highly overlap. DCB improves on this by using autoencoder errors (rather than generator probability) for generation detection, yielding complementary signals.
vs. LiRA / RMIA (Strong MIAs): Even with shadow models, their 1D score distributions are insufficient to separate generated from member samples in the MGI task, because the shadow models themselves are trained on similar data, leading them to assign high scores to generated samples as well. This conversely shows the core issue of the MGI problem is not a lack of statistical power, but a single source of signaling—DCB's key contribution lies in introducing a new signal source (the autoencoder).
vs. AEDR (Double Reconstruction Attribution): AEDR also uses double reconstruction for image attribution but only distinguishes "generated vs. non-generated." DCB extends this into the fully unified MGI framework: after separating generated/non-generated samples, it proceeds to perform membership inference and generative source tracing.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ First to formalize the MGI task, pointing out the fundamental blind spot of previous MIA and attribution methods (disregarding the autoencoder signal). The three-stage cascade design is simple yet complete, reshaping the problem instead of providing an incremental patch.
Experimental Thoroughness: ⭐⭐⭐⭐ Covers 3 IARs + 2 DMs + 2 SOTA DMs across direct training, model derivative, and memorized sample settings with strong MIA shadow model comparisons. The appendix includes analysis on AUC, TPR@5%FPR, cross-architecture, robustness, hyperparameters, and computational overhead. The limitation where NM/NN discrimination on DM is bounded by the baseline MIA is a fundamental challenge of the task rather than insufficient experimentation.
Writing Quality: ⭐⭐⭐⭐⭐ Clear problem definition (with an excellent diagram of notation in Figure 1). The logical progression from the failure analysis of MIA/attribution methods to the three-stage design of DCB is highly self-consistent, explaining thoroughly "why is it needed", "what does it solve", and "why can it not be skipped" for each stage.
Value: ⭐⭐⭐⭐⭐ MGI is a highly practical new inference task for detecting synthetic data contamination in training sets, blocking data loops to prevent model collapse, and auditing the data sources of training models. The concept of autoencoder self-consistency in DCB could easily be extended to other modalities (audio, video, text).