Skip to content

Locality-Aware Continual Unlearning for Diffusion Models

Conference: ECCV2026
Paper: ECCV 2026 official page
Code: https://github.com/SonyResearch/LACU
Area: Image Generation
Keywords: concept unlearning, continual unlearning, diffusion model, knowledge distillation, generative replay

TL;DR

The paper traces the collapse of one-shot concept unlearning methods after only 3-5 sequential deletions to coarse mapping targets, unprotected semantic neighbours, and text-space proxies, and proposes LACU: a single model-aware score-prediction distance drives both a per-prompt nearest safe mapping target and a replay set restricted to the forget concept's nearest neighbours in the model's own representation, which together with teacher-student distillation and ℓ2 parameter regularization holds unlearning accuracy at 0.90 over 10 sequential steps on SD v1.5 while lifting related retention from the 0.34-0.62 baseline range to 0.72.

Background & Motivation

Text-to-image diffusion models have already been deployed at scale in systems such as Sora, Imagen 3.0, and DALL·E 3, but deletion requests do not stop once a model ships: privacy regulations such as the GDPR, copyright claims, and tightening safety policies all turn "remove one more concept" into a recurring event. Continual unlearning (CUL) — deleting concepts one after another while preserving the rest of the model's capability — is therefore the operational baseline rather than a special case. Yet the dominant concept-unlearning methods are all single-step designs: ESD steers predictions away from the target concept via classifier-free guidance, Concept Ablation pulls forget prompts toward a null anchor by minimizing KL divergence, and UCE, MACE, and SALUN restrict updates through attention layers, lightweight adapters, or saliency-based weights. Applied repeatedly to the same model, they collapse within 3-5 steps.

Why do they collapse? The paper breaks the cause into three layers that compound. The first is coarse mapping: these methods redirect all forget prompts to the same fixed anchor (an empty string or one generic concept), but the same forget concept can appear in prompts whose scene context differs substantially — "a sliced apple on a wooden cutting board" and "an Apple logo engraved on a laptop lid" clearly need different replacements. Forcing one surrogate target makes the model displace its score field far more than necessary, and these oversized displacements accumulate into global degradation. The second layer is more fundamental: the collateral damage of unlearning — the ripple effect — is not uniform. It concentrates on concepts semantically adjacent to the forget concept, because neighbouring concepts share similar internal representations and noise predictions. Existing retention mechanisms (random replay, broad distillation, parameter regularization) spread their signal uniformly over the whole concept space, over-investing in already-safe distant concepts and under-investing in the most fragile neighbours; after several steps the neighbourhood breaks first, which shows up as a sharp drop in related retention (RRacc) while general retention (GRacc) still looks respectable. The third layer is proxy mismatch: the few methods that do attempt careful selection rely on CLIP-style text-space similarity, yet being close in text space does not mean being close in the model's denoising behaviour. Worse, every unlearning update reshapes the model's internal landscape, so proximity relations that held before an update may no longer hold after it — and such mismatches are amplified step after step in CUL.

The angle this paper takes is therefore: if "what counts as close" should be answered by the model itself, stop selecting in text space and select in the score-prediction space that unlearning updates directly modify, letting the same metric decide both "where to forget toward" and "what to protect". Core idea: define locality with the diffusion model's own score-prediction distance \(d_{\text{score}}\) over shared noisy states, recomputed at every continual step with the current frozen teacher — pick the minimum-displacement safe mapping target for each forget prompt (Locality-Aware Target Selection), and replay-distill only the concepts nearest to the forget concept in the model's representation (Locality-Aware Replay), minimizing each update while concentrating protection where it is most needed.

Method

Overall Architecture

The paper casts every continual unlearning step as a multi-objective teacher-student process. At step \(i\) the forget concept is \(c_f^{(i)}\), the frozen model from the previous step \(\hat{\epsilon}_{\hat{\theta}_{i-1}}\) acts as teacher, and the student \(\epsilon_{\theta_i}\) is initialized from the teacher, updated only within this step, and immediately frozen into the next step's teacher. The student must satisfy three things at once: its response to forget prompts is redirected to a safe, nearby mapping target (\(\mathcal{L}_{\text{unlearn}}\)); its denoising behaviour on prompts of the forget concept's neighbours stays identical to the teacher's (\(\mathcal{L}_{\text{retain}}\)); and its parameters do not drift far (\(\mathcal{L}_{\text{reg}}\)). The data flow is two parallel paths driven by the same \(d_{\text{score}}\): the unlearn path generates candidate replacements for each forget prompt and picks the nearest one, while the preservation path enumerates related concepts and keeps the \(N_r\) nearest as the local neighbourhood. The two paths yield their own distillation losses, which are optimized jointly with the remaining objectives, and the updated student becomes the teacher for the next step, cycling K times.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Step-i forget concept<br/>+ scene-level forget prompt"] --> B["Model-aware locality metric<br/>score-prediction distance d_score"]
    B --> C["Locality-Aware Target Selection<br/>nearest safe target per prompt"]
    B --> D["Locality-Aware Replay<br/>nearest neighbours in model space"]
    C --> E["Teacher-student distillation + parameter regularization<br/>small localized updates"]
    D --> E
    E --> F["Freeze student as next teacher"]
    F -->|i < K, next step| A

Key Designs

1. A model-aware locality metric: let the model's own denoising behaviour answer how similar two prompts are

Given two prompts \(p_a\) and \(p_b\), a frozen model \(\hat{\epsilon}_{\hat{\theta}}\), and \(K\) shared noisy-latent/timestep pairs \((z_{t_k}^{(k)}, t_k)\), the score-prediction distance is the mean squared ℓ2 distance between the two conditioned noise predictions:

\[d_{\text{score}}(p_a, p_b; \hat{\theta}) = \frac{1}{K}\sum_{k=1}^{K}\left\|\epsilon_{\hat{\theta}}\big(z_{t_k}^{(k)}, t_k, p_a\big) - \epsilon_{\hat{\theta}}\big(z_{t_k}^{(k)}, t_k, p_b\big)\right\|_2^2\]

It measures how differently the model denoises the same noisy image under two text conditions: a small value means the model treats the two prompts as the same thing, so redirecting \(p_a\) to \(p_b\) needs only a small perturbation of the learned score field. Strictly speaking this is a mean squared ℓ2 distance; the paper says "distance" only for brevity, since only relative ranking is used. The noisy latents \(z_{t_k}\) are obtained by forward-diffusing randomly sampled clean latents rather than by DDIM inversion, which would require a costly multi-step inference pass per prompt. This is legitimate because only ranking matters: all candidates share the same \((z_{t_k}, t_k)\) pairs, so content-independent bias cancels out in the comparison, and the authors verify in the supplementary material that the resulting selections agree with DDIM-latent ranking in the majority of cases.

As for why CLIP similarity is not used: CLIP compares prompts in a text-embedding space that is disjoint from the diffusion model's conditioning and denoising pathways, so two prompts can be very close in CLIP space yet induce very different noise predictions (and vice versa). Since the unlearning update modifies the model's weights, the metric steering target and neighbour selection should be built on the model's own noise predictions — and \(d_{\text{score}}\) is measured on exactly the same noisy states and the same weights that the update will change. The paper runs the identical LACU pipeline with CLIP ranking substituted (Table 2): at 10 steps, RRacc/GRacc improves from 0.61/0.78 to 0.74/0.87, confirming what "model-aware" buys in practice.

2. Locality-Aware Target Selection: give every forget prompt its own nearest safe target instead of mapping all of them to one anchor

This design targets the coarse-mapping failure. It works in two stages: an LLM first proposes \(M=10\) candidate replacements per forget prompt \(p_f\), with the template explicitly instructing it to replace only the forget concept while preserving the rest of the scene; selection then takes the candidate with the smallest distance to \(p_f\) under exactly the same metric as Design 1:

\[p_m^*(p_f) = \arg\min_{p \in \mathcal{P}(p_f)} d_{\text{score}}\big(p_f, p; \hat{\theta}_{i-1}\big)\]

The chosen target is the one whose score predictions are closest to the forget prompt's, so the required score-field displacement for this step is minimized. Selections are genuinely prompt-level rather than concept-level: when forgetting Apple, "an Apple logo engraved on a silver laptop lid on a desk" is mapped to "a tech company logo engraved on a silver laptop lid on a desk", whereas "a sliced apple on a wooden cutting board in kitchen light" is mapped to "a sliced red fruit on a wooden cutting board in kitchen light" — the same concept name, two unrelated replacements, precisely what a fixed anchor cannot express.

An easily overlooked but crucial detail is that this step must be recomputed at every continual step with the current frozen teacher: each update changes the model's denoising landscape, so a target chosen from an earlier model may no longer be aligned. Dynamic re-selection also removes a concrete failure mode of fixed-anchor methods — if a concept previously used as a mapping target later itself becomes a forget concept, a fixed mapping degenerates into an ineffective self-mapping, whereas re-scoring finds a new nearest safe target; the paper reports no degradation in Uacc in this case, only slightly slower convergence. This is also what separates LACU from adaptive-target methods such as AGE: AGE selects its target through minimax optimization over a CLIP-filtered class vocabulary, so the targets remain discrete labels lying outside the current diffusion score field, and its update does not explicitly protect neighbouring retain concepts — in Table 1, AGE reaches Uacc 0.99 but is left with RRacc/GRacc of only 0.15/0.23 after 10 steps.

3. Locality-Aware Replay: spend the replay budget on the forget concept's semantic neighbourhood

This design targets the unprotected-neighbourhood failure. Since the ripple effect concentrates on semantic neighbours, global replay is necessarily misallocated. The procedure is: an LLM enumerates a candidate pool of related concepts \(S(c_f)=\{c_1,\dots,c_L\}\) (the LLM supplies breadth); the same metric as Design 1 then ranks each candidate, using the concept name directly as the conditioning prompt and computing its score-prediction distance to the forget concept; the \(N_r=10\) nearest form the local neighbourhood \(\mathcal{N}(c_f)\); replay prompts are finally generated for those neighbours, so the LLM proposes concept names and the model decides which ones get in. The benefit is that the replay signal is no longer a uniformly spread regularizer but a local functional regularizer acting on the most fragile region: it constrains not the overall magnitude of parameter movement but the agreement between student and teacher along the denoising trajectories of these neighbouring prompts, thereby cancelling the ripple effect precisely without disturbing concepts that are already safe. The paper runs the same pipeline with CLIP-based neighbour ranking and drops from 0.74/0.87 to 0.61/0.78 at 10 steps — an error in the neighbour-filtering stage translates directly into misdirected protection.

4. Teacher-student distillation with parameter regularization: bound each edit to a small localized update

Design 2 decides where to go and Design 3 decides what to protect; this one decides how far and how fast. The unlearning loss is an asymmetrically conditioned distillation objective: on the same noisy states the teacher is conditioned on the mapping target \(p_m^*(p_f)\) while the student is conditioned on the original forget prompt \(p_f\), so the student learns to imitate the teacher's response to the safe target when it sees the forget prompt:

\[\mathcal{L}_{\text{unlearn}} = \left\|\epsilon_{\hat{\theta}_{i-1}}\big(z_t^u, t, p_m^*(p_f)\big) - \epsilon_{\theta_i}\big(z_t^u, t, p_f\big)\right\|_2^2\]

This differs fundamentally from adversarial-style recipes such as ESD, which steer predictions away from the target with classifier-free guidance: the redirection lands on a concrete, nearby, model-endorsed safe response rather than on a reversed direction, so it does not push the model toward the instability that gradient ascent is prone to (recent work analysing ascent failures makes exactly this point). The retention loss is symmetric instead, requiring student and teacher to produce identical noise predictions on neighbouring replay prompts, which pins down behaviour in the local neighbourhood:

\[\mathcal{L}_{\text{retain}} = \left\|\epsilon_{\hat{\theta}_{i-1}}\big(z_s^r, s, p_r\big) - \epsilon_{\theta_i}\big(z_s^r, s, p_r\big)\right\|_2^2\]

These two losses alone are not enough, because repeated updates let parameters drift continuously, so the paper adds a lightweight ℓ2 anchor that keeps each step's weights near the teacher and suppresses cumulative drift. Ablations show the two mechanisms cover different failures: regularization alone suppresses drift but cannot restore neighbour retention (RRacc 0.39 at 10 steps), replay alone clearly rescues the neighbourhood (RRacc 0.61) but lacks a long-horizon drift constraint, and only using both reaches 0.74/0.87.

A Worked Example

Take the "Apple" step and walk through the full data flow. Entering step \(i\), the forget concept is Apple and the teacher is the model frozen at the previous step. The unlearn path first generates 10 candidates for each of the 100 Apple scene prompts, 1000 candidates in total; each prompt then selects exactly one mapping target by \(d_{\text{score}}\), shrinking the candidate set from 10 to 1 — for instance "a sliced apple on a wooden cutting board in kitchen light" ends up mapped to "a sliced red fruit on a wooden cutting board in kitchen light", while an equally "safe" but context-destroying candidate such as "a red fruit on a table" is eliminated because its displacement is larger. On the preservation path, the LLM's related-concept pool \(S(\text{Apple})\) contains Pear, Red fruit, Orchard, and others; after scoring the concept names with the same metric, neighbours like Pear and Red fruit enter the top-10 while Orchard, whose denoising behaviour the model considers far away, is discarded (pool \(L\) → 10, ✓ kept / ✗ dropped). Replay prompts are then generated for these 10 neighbours and mixed with random global prompts for broader coverage. Training alternates sampled forget prompts and replay prompts for 250-350 AdamW optimizer steps with timesteps sampled uniformly from 0-600, and the updated student is frozen into the next step's teacher before moving on to the next concept.

Loss & Training

The training objective is a linear combination of three losses, with the parameter regularizer directly bounding the weight distance between student and teacher:

\[\mathcal{L}_{\text{reg}} = \left\|\theta_i - \hat{\theta}_{i-1}\right\|_2^2,\qquad \mathcal{L}_{\text{total}} = \lambda_{\text{unlearn}}\mathcal{L}_{\text{unlearn}} + \lambda_{\text{retain}}\mathcal{L}_{\text{retain}} + \lambda_{\text{reg}}\mathcal{L}_{\text{reg}}\]

⚠️ Equation 1 of the original paper is garbled in the cached text; the expression above reconstructs it as a linear combination following the surrounding prose. The actual \(\lambda\) values should be taken from the original paper (and its supplementary material).

Other training and data details: the base model is Stable Diffusion v1.5; each continual step trains for 250-350 AdamW optimizer steps, the full 10-concept sequence runs on 3 NVIDIA H100 GPUs, and a single unlearning update takes about 60 minutes; noise timesteps \(t\) (unlearn path) and \(s\) (replay path) are sampled uniformly from 0 to 600; each forget concept comes with \(N_f=100\) forget prompts, each with \(M=10\) candidate mapping prompts, and \(N_r=10\) replay neighbours; the unlearn-path latents are DDIM-sampled from the forget prompt while the replay-path latents are DDIM-sampled from the replay prompt. Notably, the LLM is used only to populate candidate pools — the LACU objective itself is agnostic to how those candidates are produced. A no-LLM variant (template-based forget/mapping prompts plus a same-category pool of related concepts) achieves 10-step Uacc/RRacc/GRacc of 0.85/0.81/0.87, indicating that model-aware scoring and local replay, not the LLM, drive the stability.

Key Experimental Results

Main Results

All methods start from the same Stable Diffusion v1.5 checkpoint (base model: Accuracy 89%, CLIP score 32.6) and unlearn the same 10 concepts in sequence: Pikachu, Brad Pitt, Golf Ball, Van Gogh Style, Apple, Spiderman, Lionel Messi, Cartoon Style, Banana, and Mickey Mouse. To eliminate order dependence, every method is run over 5 random permutations and averaged. Evaluation uses Qwen2.5-VL-7B-Instruct as a VLM judge answering binary questions (evaluation only — target and neighbour selection are decided by score-prediction distance), extending the UnlearnCanvas and EraseBench protocols.

Table 1: main comparison after 10 continual unlearning steps (Uacc: unlearning accuracy; Uclip: text-to-image alignment on the benign parts of forget prompts; RRacc/GRacc: related and general retention accuracy).

Method Uacc ↑ Uclip ↑ RRacc ↑ GRacc ↑
SD v1.5 (reference) 0.89 32.6 — —
ESD-u 0.93 21.3 0.13 0.15
ESD-x 0.96 19.0 0.09 0.08
UCE 0.96 19.5 0.08 0.05
MACE 0.97 19.4 0.03 0.02
Meta 1.00 18.2 0.02 0.01
EraseFlow 0.97 19.4 0.03 0.02
ANT 1.00 20.5 0.00 0.00
CA 0.90 22.8 0.34 0.34
AGE 0.99 19.1 0.15 0.23
DUGE 0.74 30.7 0.62 0.73
SMem 0.73 29.3 0.58 0.67
Grad Proj 0.83 28.1 0.61 0.71
LACU (Ours) 0.90 29.1 0.72 0.87

Table 2: retention decay across sequential steps (also averaged over 5 random orderings).

Method 1-step RRacc 5-step RRacc 10-step RRacc 1-step GRacc 5-step GRacc 10-step GRacc
ESD-u 0.60 0.30 0.13 0.75 0.35 0.15
ESD-x 0.55 0.29 0.09 0.77 0.28 0.08
CA 0.73 0.58 0.34 0.84 0.64 0.34
AGE 0.49 0.36 0.15 0.82 0.55 0.23
DUGE 0.70 0.73 0.62 0.85 0.80 0.73
SMem 0.70 0.72 0.58 0.84 0.78 0.67
LACU (Ours) 0.84 0.80 0.72 0.89 0.87 0.87

Ablation Study

Table 3: four target/neighbour selection strategies inside the identical LACU pipeline (after 10 steps).

Config Uacc Uclip RRacc GRacc
naive fixed anchor 0.86 22.9 0.41 0.55
direct LLM replacement 0.95 28.1 0.60 0.76
CLIP ranking 0.90 28.2 0.61 0.78
\(d_{\text{score}}\) ranking (Ours) 0.92 29.2 0.74 0.87

Table 4: component ablation of the training objectives (after 10 steps; all variants use \(d_{\text{score}}\)-based selection).

Config Uacc Uclip RRacc GRacc Note
\(\mathcal{L}_{\text{unlearn}}\) only 0.94 27.2 0.32 0.67 forgets well, neighbour retention collapses
\(\mathcal{L}_{\text{unlearn}}+\mathcal{L}_{\text{retain}}\) 0.91 28.6 0.61 0.85 replay pulls the neighbourhood back
\(\mathcal{L}_{\text{unlearn}}+\mathcal{L}_{\text{reg}}\) 0.92 28.0 0.39 0.78 regularization only limits drift
LACU (full) 0.92 29.2 0.74 0.87 the two are complementary

Key Findings

  • One-shot methods face a clear stability-efficacy trade-off under CUL: aggressive methods such as ESD, ANT, and EraseFlow drive Uacc close to 1.0 (ANT reaches exactly 1.00) while destroying retention, leaving RRacc/GRacc at 0.00-0.15 after 10 steps; conservative methods such as DUGE and SMem preserve retention but concede Uacc down to 0.73-0.74. Only CA and LACU hold Uacc at 0.90, and CA's RRacc/GRacc are just 0.34/0.34 versus LACU's 0.72/0.87 — direct evidence that this paper holds both axes at once.
  • The shape of the degradation curve says more than any single number: ESD-x's RRacc falls from 0.55 at 1 step to 0.09 at 10 steps (roughly a 6× decay) and CA's from 0.73 to 0.34, whereas LACU only declines from 0.84 to 0.72 with GRacc stable at 0.87-0.89; LACU's Uacc across 1/5/10 steps is 0.99/0.89/0.90, also essentially flat over time. The paper reads this flatter, higher trend as evidence that locality-aware design suppresses cumulative distortion.
  • Choosing a better target alone is far from sufficient. AGE uses adaptive targets to push Uacc to 0.99 yet ends at RRacc/GRacc of 0.15/0.23 after 10 steps — under CUL, target choice only addresses displacement magnitude, not the unprotected neighbourhood, which requires local replay.
  • Local replay is the largest single contributor. Going from \(\mathcal{L}_{\text{unlearn}}\) alone to adding \(\mathcal{L}_{\text{retain}}\) lifts RRacc from 0.32 to 0.61 (+0.29) and GRacc from 0.67 to 0.85; adding parameter regularization alone only reaches RRacc 0.39 (+0.07), because an ℓ2 anchor constrains how far parameters move without actively protecting any concept. Combining both gives 0.74/0.87.
  • Model-aware scoring beats text-space proxies. In the identical pipeline, \(d_{\text{score}}\) ranking (0.74/0.87) beats CLIP ranking (0.61/0.78), which beats direct LLM replacement (0.60/0.76), which beats fixed anchors (0.41/0.55); the gap between CLIP and \(d_{\text{score}}\) widens with the number of steps, consistent with the explanation that text space is decoupled from the model's denoising dynamics.
  • Order robustness: all main results average 5 random deletion orders, so the conclusions do not depend on a particular sequence. Dynamic target re-selection additionally avoids the self-mapping degeneracy of fixed-anchor methods: when a concept previously used as a mapping target later becomes a forget target, LACU re-scores at that step and still finds a new safe target, with no degradation in Uacc and only slightly slower convergence.
  • The no-LLM variant still reaches 0.85/0.81/0.87 (Uacc/RRacc/GRacc), showing that the gains come from model-aware scoring and local replay themselves; the LLM contributes candidate breadth and can be replaced by template-based prompts and a same-category concept pool.
  • Evaluation robustness: the paper reports that swapping in alternative VLM judges changes key rows by at most ±2% and preserves the relative ranking of LACU and the baselines; all evaluation prompts are unseen during training.
  • ⚠️ Self-consistency note: the LACU row in Table 1 (0.90/29.1/0.72/0.87 after 10 steps) differs slightly from the same configuration in Tables 2 and 3 (0.92/29.2/0.74/0.87), presumably due to separate runs and rounding/averaging; cite the table the number comes from.

Highlights & Insights

  • Defining "distance" in the space the model itself will modify: choosing a mapping target and choosing replay neighbours are two separate engineering problems, and the paper settles both with one \(d_{\text{score}}\). Because the metric acts on the weights and noisy states that the unlearning update will actually change, it measures "what this update will disturb" rather than "how similar the two sentences sound" — the most elegant reframing in the paper.
  • Replacing DDIM inversion with random latents: since only relative ranking is needed, sharing the same \((z_{t_k}, t_k)\) pairs across all candidates cancels content-independent bias, making the expensive multi-step inversion unnecessary. This "preserve the ordering, skip the calibration" cost-cutting move transfers to any sampling-based task that needs inter-prompt similarity.
  • Asymmetric conditioning turns "forgetting" into "redirection": the teacher sees the safe target while the student sees the forget prompt, so the model learns to denoise the sensitive prompt into its safe neighbour instead of being pushed away by gradient ascent. This sidesteps ascent degeneracy and removes the extra negative-guidance hyper-parameters.
  • Local replay is a general recipe for "spend the regularizer where it is fragile": replacing a global replay distribution with a neighbour set ranked by the model's own representation is effectively an importance weighting on the regularizer chosen by the model itself. The idea transfers directly to continual unlearning in LLMs, sequential knowledge editing, and any setting where a model is edited repeatedly and neighbouring capabilities must survive.
  • Ripple effects call for locality, not globality: the paper stresses that a stable GRacc does not mean the model is intact — RRacc breaks first. That observation is itself an evaluation-level insight, since looking only at general retention badly overestimates the health of continual unlearning methods.

Limitations & Future Work

  • Admitted by the authors: the implementation relies on an LLM to generate candidate prompts and related concepts, so the quality and coverage of the candidate pool bounds the effectiveness of the downstream model-aware selection; local replay carries an inherent precision trade-off when the forget concept and the retained neighbours share heavily overlapping visual features (mitigated by regularizing only the nearest neighbour trajectories rather than the full retain distribution, which keeps object and identity unlearning strong while style concepts remain harder because their features are shared more broadly); the teacher-student framework requires maintaining a frozen teacher copy, increasing memory and computation relative to single-model approaches; and evaluation covers only 10 sequential deletions, whereas real deployment may require hundreds or thousands of removals over a model's lifetime.
  • Directions worth adding: treating 250-350 optimizer steps × 10 steps (about 60 minutes per step on 3×H100) as the cost baseline, one could implement the "frozen teacher + small student" recipe with LoRA-style adapters, saving memory while naturally limiting parameter drift; the neighbourhood size \(N_r\) and the scoring sample counts \(K_f, K_r\) look like direct precision/cost knobs, but the paper gives no sensitivity curves (⚠️ the cached text does not state these hyper-parameter values or any sensitivity analysis; refer to the supplementary material).
  • Evaluation caveats: Uacc/RRacc/GRacc all depend on a VLM judge answering binary questions, and although the paper runs a judge-sensitivity check, the judge's prompt design, the number of sampled images, and random seeds may still shift absolute values; moreover, each paper defines its own set of "related concepts", so RRacc values are not directly comparable across works.
  • A natural extension: dynamic target re-selection is presented only as a side benefit (avoiding self-mapping degeneracy), but it points at a more interesting problem — the mapping target of one deletion can be overwritten by a later request, i.e. deletion requests depend on each other. Treating the whole deletion sequence as a planning problem instead of step-wise greedy re-scoring could plausibly go further than per-step recomputation.
  • vs ESD / CA / UCE / MACE (one-shot concept unlearning): they all pull forget prompts toward a single fixed anchor (an empty string, a generic concept, or steering away via classifier-free guidance) and work well in the single-step setting, but forcing diverse forget prompts onto one surrogate makes the model displace its score field far more than necessary, and repeated application accumulates into global degradation (RRacc ≤ 0.34 after 10 steps). LACU switches to prompt-level targets chosen inside the model's score field and additionally protects the neighbourhood.
  • vs AGE (adaptive target selection): AGE also argues forget concepts should be redirected to nearby non-synonym targets, but its targets come from a CLIP-filtered discrete class vocabulary chosen by minimax optimization and lie outside the diffusion score field; in Table 1, used on its own it yields Uacc 0.99 with RRacc/GRacc of only 0.15/0.23. LACU selects inside the model's own score-prediction space and ties target selection to neighbour protection through the same metric.
  • vs DUGE / SMem / Grad Proj (continual unlearning backbones): these already use replay, distillation, parameter regularization, or gradient projection and are far more stable than one-shot methods over long sequences (DUGE reaches GRacc 0.73 at 10 steps), but their protection happens either in parameter space or over a global prompt pool, so RRacc stays at 0.58-0.62. LACU keeps the same distillation-plus-regularization skeleton and only swaps the replay distribution for model-aware local neighbours, lifting RRacc to 0.72 and GRacc to 0.87.
  • vs continual-learning-style unlearning such as Selective Amnesia: these works also treat unlearning as a sequential task, and this paper cites them as sources on sequential failure modes without including them in the numeric comparison; by design their protection remains global or parameter-space based, which is a different route from the local functional regularizer here.
  • Takeaways: if \(d_{\text{score}}\) were replaced by another "model introspection" signal (attention-map discrepancy, hidden-representation discrepancy), could the same ranking quality be preserved at lower compute cost? And could target selection itself be made differentiable so that the target participates in optimization? Both directions extend straight out of this framework.

Rating

  • Novelty: ⭐⭐⭐⭐ Unifying locality under the model's own score-prediction distance and using it for both target selection and replay is a clean, transferable insight, though each ingredient (replay distillation, parameter regularization, teacher-student) is not new.
  • Experimental Thoroughness: ⭐⭐⭐⭐ 11 baselines × 10 steps × 5 random orderings, plus two ablations (selection strategy, components) and a no-LLM variant; however the main table covers only SD v1.5, hyper-parameter sensitivity is missing, and 10 steps remain far from the hundreds-to-thousands of deletions real deployment implies.
  • Writing Quality: ⭐⭐⭐⭐ The three-layer explanation of why one-shot methods fail under CUL maps one-to-one onto the method components with a clear logic chain; a few equations (Eq. 1) are garbled.
  • Value: ⭐⭐⭐⭐ It moves the evaluation focus of continual unlearning from "can the concept be forgotten" to "do neighbours survive sequential deletion", and the RRacc/GRacc contrast is instructive for evaluation across the whole concept-unlearning direction.