✏️ Knowledge Editing

🔬 ICLR2026 · 15 paper notes

📌 Same area in other venues: 📷 CVPR2026 (2) · 💬 ACL2026 (10) · 🧪 ICML2026 (8) · 🤖 AAAI2026 (4) · 🧠 NeurIPS2025 (6)

ACE: Attribution-Controlled Knowledge Editing for Multi-hop Factual Recall

ACE identifies a neglected mechanism via neuron-level attribution where "implicit subjects act as query neurons in multi-hop reasoning, activating value neurons layer-by-layer." Accordingly, it refines editing from "layer-level heuristics" to "query-value pathways," outperforming the SOTA PMET by 9.44% on GPT-J and 37.46% on Qwen3-8B in multi-hop factual recall.

Bilinear Representation Mitigates Reversal Curse and Enables Consistent Model Editing

By training Transformers from scratch on synthetic relational knowledge graphs, it is discovered that appropriate regularization leads to the emergence of a bilinear relational structure in hidden layers. This structure not only overcomes the reversal curse but also enables logically consistent propagation of updates to related facts after editing a single fact.

Disentangling Knowledge Representations for Large Language Model Editing

Addressing the neglected problem where knowledge editing collateralizes "same-subject but different-relation/object" fine-grained irrelevant knowledge, this paper proposes DiKE: it first uses a reusable disentanglement module to split subject representations into "target-related" and "irrelevant" parts, then performs editing only on the related part while explicitly constraining the irrelevant part to remain unchanged, deriving a closed-form rank-one parameter update formula that maintains mainstream editing performance while preserving fine-grained irrelevant knowledge.

EAMET: Robust Massive Model Editing via Embedding Alignment Optimization

This paper reveals that the root cause of failure in massive model editing is the structural inconsistency between key embeddings and residual embeddings (embedding misalignment). It proposes EAMET, which progressively saves optimized residual embeddings and aligns their neighborhood structure to the key embedding space using a dual loss of KL divergence and MSE. Experimental results across 6 LLMs and 3 datasets show that EAMET outperforms MEMIT by an average of 14% (CounterFact) and 8% (ZsRE) when editing 10k facts simultaneously, while maintaining robustness in scenarios involving long prefixes and multiple facts per subject.

Energy-Regularized Sequential Model Editing on Hyperspheres

Performance degradation in sequential model editing is understood from the perspective of hyperspherical uniformity (Hyperspherical Energy, HE). The SPHERE method is proposed: by projecting editing perturbations onto the orthogonal complement of the primary hypersphere directions of pre-trained weights, stable large-scale sequential editing is achieved, outperforming the strongest baseline by an average of 16.41% on LLaMA3-8B.

Fine-tuning Done Right in Model Editing

This paper reveals that the root cause of the underestimated performance of fine-tuning in model editing is an incorrect training pipeline (Depth-First sample-by-sample optimization). By correcting this to a standard Breadth-First mini-batch training and combining it with localized parameter tuning to form LocFT-BF, the authors achieve the first support for 100,000 sequential edits and a 72B model scale.

GOT-Edit: Geometry-Aware Generic Object Tracking via Online Model Editing

By using null-space constrained online model editing, this work integrates 3D geometric information provided by VGGT into a 2D generic object tracker. This enhances geometric awareness while maintaining semantic discriminative power, significantly improving tracking performance in scenarios with occlusions and background clutter.

KnowledgeSmith: Uncovering Knowledge Updating in LLMs with Model Editing and Unlearning

This paper proposes KnowledgeSmith, which unifies "knowledge editing" and "machine unlearning" into a single constrained optimization problem. By using knowledge graphs (KG) to automatically generate large-scale evaluation benchmarks across different hierarchies (root/intermediate/leaf) and data scales, the study systematically reveals counter-intuitive phenomena in LLM knowledge updates, such as propagation asymmetry, consistency-capacity trade-offs, and subject dependency.

MobiEdit: Resource-efficient Knowledge Editing for Personalized On-device LLMs

MobiEdit replaces the resource-heavy backpropagation in the classic locate-and-edit knowledge editing (ROME) with "quantization + forward zeroth-order gradient estimation," coupled with two system optimizations: early stopping and prefix activation reuse. This allows real-time knowledge editing for 3B LLMs to run on commercial smartphone NPUs for the first time, reducing memory by 7.1×, energy consumption by 15.8×, and latency by 3.4×.

MoEEdit: Efficient and Routing-Stable Knowledge Editing for Mixture-of-Experts LLMs

MoEEdit is the first "routing-stable" parameter-modifying knowledge editing framework for MoE LLMs. It employs "per-expert null space projection" to ensure that edits do not perturb input manifolds for downstream routers, combined with a stochastic Block Coordinate Descent (BCD) solver to decouple computational costs from the total number of experts to the expert hidden dimension. This achieves high editing success rates, strong generalization, and routing stability on sparse architectures simultaneously.

PICS: Pairwise Image Compositing with Spatial Interactions

PICS is proposed as a parallel pairwise image compositing method. Through mask-guided MoE and an adaptive \(\alpha\)-blending strategy within an Interaction Transformer, it simultaneously composites two objects in a single inference pass while explicitly modeling spatial interactions such as occlusion and contact, significantly outperforming existing sequential methods.

Scaling Knowledge Editing in LLMs to 100,000 Facts with Neural KV Database

This paper reinterprets existing Locate-and-Edit knowledge editing methods as "querying a KV database" and proposes NeuralDB. By replacing the traditional linear perturbation \(\Delta\) with a non-linear gated retrieval module, the fact capacity is extended from several hundred to 100,000, while maintaining the model's general capabilities.

SUIT: Knowledge Editing with Subspace-Aware Key-Value Mappings

SUIT constrains the "manually calculated" key vector \(k\) and residual vector \(\delta\) in locate-then-edit knowledge editing into low-dimensional subspaces truly relevant to the target edit. This significantly reduces damage to irrelevant knowledge without sacrificing editing success—doubling Specificity on LLaMA3 / GPT-J / Qwen2.5 compared to the strong baseline AlphaEdit.

TangleScore: Tangle-Guided Purge and Imprint for Unstructured Knowledge Editing

This paper proposes TangleScore, an intrinsic difficulty metric determined solely by the "model + knowledge sample" independent of specific editing algorithms, to measure how "hard" a piece of knowledge is to modify. Based on this, it designs PIPE (a two-stage editing framework that purges old knowledge before imprinting new knowledge), improving generalization performance by an average of 6.49% across four LLMs of varying scales and two unstructured editing benchmarks.

When Large Multimodal Models Confront Evolving Knowledge: Challenges and Explorations

This paper proposes the EVOKE benchmark to systematically evaluate the knowledge injection capabilities of Large Multimodal Models (LMMs) for evolving knowledge. It reveals two primary challenges: the poor performance of existing methods and catastrophic forgetting caused by fine-tuning. Furthermore, it explores two response paths: knowledge augmentation and continual learning.