✏️ Knowledge Editing¶

🧪 ICML2026 · 8 paper notes

📌 Same area in other venues: 📷 CVPR2026 (2) · 🔬 ICLR2026 (15) · 💬 ACL2026 (10) · 🤖 AAAI2026 (4) · 🧠 NeurIPS2025 (6) · 🧪 ICML2025 (2)

🔥 Top topics: LLM ×4

AnyEdit++: Adaptive Long-Form Knowledge Editing via Bayesian Surprise: AnyEdit++ utilizes token-level Bayesian Surprise to identify semantic transition points in long-form text, replacing the fixed-window segmentation of AnyEdit with structure-aware Bayes-Chunk. It achieves stable improvements in BLEU and BERT Score across long-form knowledge editing tasks such as mathematics, code, news, and poetry.
CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing: LLM editing is formulated as a constrained optimization problem: "minimize edit loss s.t. capability loss remains invariant". This is equivalently transformed via Bregman divergence into a low-curvature subspace projection of the Gauss-Newton Hessian (GNH). By employing K-FAC and a Kronecker eigenbasis technique that avoids explicit construction of the projection matrix, 3,000 edits are completed in 6 minutes on an A40. The average performance drop of LLaMA-3-8B across MMLU/IFEval/ARC-C/TruthfulQA/GSM8K is suppressed to \(< 1\%\), significantly outperforming AlphaEdit, MEMIT, and fine-tuning.
Do Text Edits Generalize to Visual Generation? Benchmarking Cross-Modal Knowledge Editing in UMMs: This paper proposes UniKE—the first "cross-modal knowledge editing" benchmark for Unified Multimodal Models (UMMs) (2,971 editing subjects, 5,535 VQA-verifiable instances). It systematically reveals a modality gap where the "text-side editing success rate is ~92%, yet image generation VQA is only ~18.5%." By using a "reasoning-augmented parameter editing" protocol, it increases VQA accuracy by up to 18.6 percentage points and identifies the root cause as the LLM-to-DiT projection bottleneck using cosine drift metrics on the conditioning path.
From Backward Spreading to Forward Replay: Revisiting Target Construction in LLM Parameter Editing: This paper systematically analyzes why backward spreading in locate-then-edit works and where it falls short. It proposes forward replay: treating the hidden state of the first decisive layer as an optimization variable and performing a standard forward pass to obtain targets for subsequent layers. This achieves consistent performance gains over MEMIT/RECT/PRUNE/AlphaEdit without additional computational overhead.
KORE: Enhancing Knowledge Injection for Large Multimodal Models via Knowledge-Oriented Controls: KORE injects new knowledge into LMMs through two-stage "knowledge-oriented controls": automatically expanding single facts into structured multi-turn conversations and instruction tasks (to enhance generalization), while initializing LoRA adapters using the null space of the covariance matrix of prior knowledge (to minimize interference with existing capabilities). It achieves both strong adaptation and strong retention on LLaVA-v1.5 / Qwen2.5-VL.
Reverse-Engineering Model Editing on Language Models: The paper reveals that parameter update matrices of locate-then-edit knowledge editing methods (ROME/MEMIT/AlphaEdit) leak "edited subject" fingerprints through their row spaces. It proposes a two-stage attack, KSTER (recovering subjects via SVD, then prompts via relative entropy drop), and a defense called Subspace Camouflage based on "semantic decoy" injection.
Revisiting Parameter-Based Knowledge Editing in Large Language Models: Theoretical Limits and Empirical Evidence: Ours starts from the "dimension collapse" hypothesis, proving that parameter-level knowledge editing is amplified along directions with low singular values and accumulates linearly with sequential editing. This systematically degrades core LLM capabilities across multiple models, datasets, and evaluation dimensions. Ours further indicates that a simple retrieval-based baseline, SCR, outperforms existing parameter editing methods in all settings.
The Labyrinth and the Thread: Rethinking Regularizations in Sequential Knowledge Editing for Large Language Models: This paper proves from an optimization perspective that the stability of sequential editing (SE) stems from "cumulative updates being equivalent to the solution of one-time editing (OTE)." Fancy mechanisms like AlphaEdit's null-space projection or post-processing regularizations in PRUNE/RECT are not the critical factors—as long as OTE-SE alignment is ensured, 2000 steps of sequential editing can be stably completed across four mainstream LLMs even after removing these regularizations.