Skip to content

Targeted Token-Level Unlearning for Repairing Text-to-SQL Models

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/oakor/TTU-SQL
Area: LLM Reasoning
Keywords: Text-to-SQL, Machine Unlearning, Abstract Syntax Tree, Negative Preference Optimization, Parameter-Level Repair

TL;DR

This paper proposes TTU-SQL, a targeted token-level machine unlearning framework that localizes the first structural divergence point between predicted and gold SQL using normalized ASTs, filters three forgettable error types, and applies localized Part-NPO with recovery SFT to eliminate recurring error generation tendencies without accessing the original training data.

Background & Motivation

Large Language Models have achieved impressive breakthroughs on structured semantic parsing benchmarks such as Text-to-SQL. Supervised fine-tuning on large-scale synthetic and human-annotated corpora equips open-source language models with strong structural query synthesis capabilities. However, a critical yet frequently overlooked drawback is that standard likelihood maximization is a double-edged sword: models inadvertently memorize and reinforce recurring, reproducible erroneous generation patterns alongside correct SQL structures. Because these systematic failures stem from deterministic local decision biases embedded in model parameters rather than random decoding noise, continuing standard supervised fine-tuning simply upweights the overall sequence likelihood, frequently exacerbating the entrenched erroneous continuations.

Existing paradigms for addressing Text-to-SQL errors face substantial limitations when dealing with such internal generative biases. Inference-time approaches, including schema-linking pruning, multi-candidate selection, and post-hoc correction modules, only apply heuristic patches externally to inputs or generated strings; they leave internal parameter deficiencies untouched and incur notable inference latency overhead. On the other hand, preference alignment paradigms such as Direct Preference Optimization (DPO) tightly couple the objective of elevating gold sequence probabilities with penalizing erroneous predictions. When applied on modest probe sets whose distribution departs from target evaluation suites, this coupled objective tends to induce severe overfitting to probe-specific surface realizations. Furthermore, conventional machine unlearning frameworks (such as Full-NPO) indiscriminately penalize whole sequences via gradient ascent, wiping out valid query prefixes and destroying fundamental SQL syntax.

Empirical investigation reveals that erroneous SQL predictions frequently contain extensive correct structural prefixes before branching off at an isolated, identifiable decision point. Furthermore, suppressing this dominant wrong continuation often enables the latent correct continuation to emerge naturally. This insight motivates reframing machine unlearning from instance-level privacy data deletion into post-hoc parameter-level surgical repair for structured prediction. Core idea: Expose systematic generation errors on an auxiliary probe dataset, pinpoint the earliest structural divergence token via normalized AST comparison, selectively suppress the erroneous local continuation using Part-NPO while retaining valid prefixes, and restore general fluent generation through recovery SFT.

Method

Overall Architecture

TTU-SQL operates via a modular four-stage pipeline: "Probe Adaptation & Diagnosis — AST Divergence Localization & Error Extraction — Part-NPO Unlearning — Recovery SFT". Given a task-adapted base model \(M_\theta\) whose original training corpus is unknown or unavailable, the system first conducts a 1-epoch lightweight probe adaptation on an auxiliary probe dataset \(D_P\) (50k single-turn examples sampled from OmniSQL) to bring the model under the probe distribution and expose reproducible error patterns, yielding reference model \(M_{\theta_1}\). The model then generates queries on \(D_P\). Erroneous predictions are parsed alongside gold SQL into normalized Abstract Syntax Trees to eliminate alias surface discrepancies and isolate the first structural divergence token index \(d\). Examining the model's logit distribution and token embeddings at position \(d\) extracts three high-precision forgettable error classes into \(D_{forget}\), while correctly predicted samples form \(D_{retain}\). Finally, Part-NPO applies negative preference optimization exclusively to token \(d\), followed by multi-epoch recovery SFT to synthesize the repaired model \(M_{\theta_3}\).

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400, 'subGraphTitleMargin': {'top': 8, 'bottom': 16}}}}%%
flowchart TD
    A["Probe Dataset D_P & Base Model M_θ<br/>Natural language question & schema context"] --> B["Probe Adaptation & Diagnosis<br/>Lightweight 1-epoch SFT exposes recurring error tendencies"]
    B --> C["AST Semantic Divergence Localization<br/>sqlglot parsing + alias normalization pinpoints first divergence token d"]
    C --> D["Forgettable Error Type Extraction<br/>Categorize Class 1/2/3 samples based on AST structure & logit behavior"]
    D --> E["Part-NPO Token-Level Unlearning<br/>Preserve correct prefix, apply localized NPO loss only to token d"]
    E --> F["Recovery SFT Stage<br/>Multi-epoch fine-tuning on D_P restores syntax fluency & likelihood"]
    F --> G["Repaired Model M_θ3<br/>Eliminates systematic biases & improves execution accuracy"]

Key Designs

1. AST Semantic Divergence Localization: Filtering surface-form artifacts to isolate the exact structural decision fracture

Direct string-level comparison fails in SQL generation because semantically identical queries frequently diverge in surface expressions such as table aliases (e.g., T1.name vs. driver.name), commutative predicate ordering, or projection whitespace. To isolate genuine semantic branching points, TTU-SQL implements tree-level AST divergence localization. Both predicted query \(\hat{y}\) and gold standard \(y^*\) are parsed into syntax trees using sqlglot, followed by an alias normalization pass that canonicalizes all table aliases and qualifiers. Traversal algorithms then perform top-down node-by-node comparisons across projection clauses, aggregations, filtering predicates, grouping clauses, and joins. The first mismatched semantic node is identified as the root divergence and mapped back to the exact token index \(d\) in the predicted sequence. This ensures that the unlearning objective acts strictly on the single decision point that alters query semantics rather than penalizing superficial formatting choices.

2. Forgettable Error Type Extraction: Filtering actionable decision biases with high repair potential

Unlearning every failed prediction introduces severe noise because many errors stem from non-local failures such as complex cross-table query planning or external schema-value grounding ambiguity that cannot be resolved via token-level parameter tweaks. TTU-SQL inspects the model's logit distribution and token embeddings at divergence token \(d\) to harvest three complementary, highly actionable error classes: (1) Structural Divergence (Class 1): The divergent token introduces an unnecessary structural clause operator in the AST, such as an extraneous GROUP BY, WHERE, or JOIN; (2) Gold Not in Top-5 (Class 2): The gold continuation token \(y^*_d\) falls outside the model's top-5 logit-ranked candidates, signifying an entrenched local preference gap where standard SFT struggles to overcome competing candidate logits; (3) Gold in Top-5 but Low Similarity (Class 3): The gold token ranks within the top-5 candidates, but its token embedding exhibits cosine similarity below \(\tau = 0.8\) against the predicted token \(\hat{y}_d\). This identifies sharp semantic misdirection where the model retains active memory of the correct concept but succumbs to an incompatible competitor. Taking the union of these three classes forms the high-precision forget set \(D_{forget}\).

3. Part-NPO Token-Level Unlearning: Decoupling error suppression from sequence destruction to preserve valid prefixes

Full-sequence Negative Preference Optimization penalizes all tokens across \(\hat{y}\), thereby corrupting the correct prefix preceding the divergence point. Part-NPO restricts the unlearning loss exclusively to the localized divergence set \(D = \{d\}\): $\(\mathcal{L}_{\text{Part-NPO}} = -2\beta \sum_{t \in D} \log \sigma \left( -\beta \log \frac{\pi_\theta(\hat{y}_t \mid \hat{y}_{<t}, x)}{\pi_{\text{ref}}(\hat{y}_t \mid \hat{y}_{<t}, x)} \right)\)$ where \(\pi_{\text{ref}}\) is frozen reference model \(M_{\theta_1}\) and \(\beta = 0.1\) is the inverse temperature coefficient. Simultaneously, a standard likelihood retain loss is maintained on \(D_{retain}\) with weight \(\alpha = 1.0\): \(\mathcal{L}_{total} = \mathcal{L}_{\text{Part-NPO}}(D_{forget}) + \alpha \mathcal{L}(D_{retain})\). In logit space, this localized gradient penalty sharply depresses the logit value of the erroneous competitor \(z_w\), reducing the logit gap \(z_g - z_w\) from a massive negative barrier down near the decision boundary. This allows subsequent recovery SFT to activate the correct candidate \(y^*_d\) within minimal optimization steps.

Loss & Training

All training stages leverage parameter-efficient LoRA (\(r = 32\), \(\alpha = 64\)) applied to query and value projection matrices under BF16 mixed-precision and cosine learning rate schedules on a single A100 80GB GPU. Training unfolds in four strictly decoupled steps: (1) Stage 1 (Probe Adaptation): 1 epoch on \(D_P\) at learning rate \(2 \times 10^{-5}\) yields \(M_{\theta_1}\); (2) Stage 2 (Offline AST Extraction): CPU-based offline parsing and filtering constructs \(D_{forget}\) and \(D_{retain}\); (3) Stage 3 (Part-NPO Unlearning): 1 epoch on \(D_{forget}\) and \(D_{retain}\) at learning rate \(1 \times 10^{-5}\) (\(\beta = 0.1\), retain weight 1.0) yields intermediate model \(M_{\theta_2}\); (4) Stage 4 (Recovery SFT): 7 epochs of standard cross-entropy fine-tuning on \(D_P\) at learning rate \(2 \times 10^{-5}\) rebuilds syntax fluency, producing final model \(M_{\theta_3}\).

Key Experimental Results

Main Results

Evaluations are performed on Spider Dev (cross-domain benchmark) and BIRD Dev (complex real-world database benchmark), reporting Execution Accuracy (EX %). Main parameter-update baselines operate under strictly identical localized data budgets, retain sets, LoRA configurations, and optimizer step counts using DeepSeek-R1-Distill-Llama-8B and DeepSeek-R1-Distill-Qwen-7B backbones.

Method / Strategy Llama-8B (Spider) Llama-8B (BIRD) Qwen-7B (Spider) Qwen-7B (BIRD)
Base Model 61.41 32.79 58.51 31.94
Direct SFT (Equal budget fine-tuning) 66.15 33.57 62.48 33.24
Localized DPO (Gold vs. predicted) 67.70 34.49 64.31 34.09
SFT on Error Subset + Retain 66.44 33.38 62.57 33.12
Weighted SFT at Divergence 64.80 33.38 61.41 32.79
TTU-SQL (Ours) 68.57 35.53 65.57 34.87

Composition with training-free inference-time enhancement methods (macro-averaged across the 4 settings): - Rethinking Schema Linking: Base (48.62%) → Direct SFT (50.74%) → TTU-SQL (52.91%) - DIVER (Dynamic value linking): Base (48.61%) → Direct SFT (50.36%) → TTU-SQL (52.01%) - SEA-SQL (Adaptive output refinement): Base (54.62%) → Direct SFT (55.21%) → TTU-SQL (55.93%) - SHARE (Hierarchical SLM correction): Base (52.75%) → Direct SFT (53.48%) → TTU-SQL (54.60%) - MBR-SQL (Execution-guided MBR, N=10): Base (46.50%) → Direct SFT (49.32%) → TTU-SQL (51.37%)

Ablation Study

Ablation investigations conducted on Spider Dev with DeepSeek-R1-Distill-Qwen-7B evaluate key architectural components against the 58.51% Base Model:

Configuration / Variant EX (%) Samples / Details
Full TTU-SQL (Part-NPO divergence token + SFT) 65.57 18,689 samples, all three classes combined
NPO on all errors (unfiltered) 63.82 Broad error set introduces unlocalizable noise
w/o Recovery SFT 44.20 Unlearning alone destabilizes output fluency
Class 1 only (Structural Divergence) 64.12 3,811 samples, targets operator additions
Class 2 only (Gold Not in Top-5) 64.50 6,551 samples, strongest standalone class
Class 3 only (Top-5 Low Similarity) 63.44 12,027 samples, addresses competing tokens
Class 1 + 2 64.22 9,545 samples
Class 1 + 3 63.64 12,955 samples
Class 2 + 3 64.89 18,578 samples
Full-NPO (Full sequence suppression) 64.22 Destroys valid prefix; invalid SQL rate is 4.9%
Part-NPO (Divergence suffix) 64.70 Suppresses potentially valid continuation tokens

Key Findings

  • Token-level pinpointing surpasses sequence-level suppression: Restricting unlearning from full sequence (64.22%) to suffix (64.70%) and finally to the single divergence token (65.57%) yields consistent accuracy gains and drives post-recovery invalid SQL rates down from 4.9% to 3.5%, proving that preserving correct prefixes is essential for structured syntax stability.
  • The suppress-then-recover trajectory: Analysis of epoch-wise optimization shows that while Part-NPO temporarily lowers execution accuracy to 44.20% by destabilizing entrenched bad habits, subsequent recovery SFT rapidly overtakes Direct SFT by recovery epoch 4, peaking at 65.76% (vs. Direct SFT's ceiling of 62.96%). The performance leap stems from relearning upon an unlearned parameter foundation rather than extra SFT steps.
  • Error concentration in complex queries: Over 86% of extracted forgettable errors reside in Hard and Extra-Hard queries (Class 1: 70.0% Extra-Hard; Class 2: 69.7% Extra-Hard; Class 3: 69.2% Extra-Hard), and over 60% of divergence positions occur within the first 10% of sequence tokens. Early operator missteps trigger cascade failures in complex queries, confirming the necessity of early divergence localization.

Highlights & Insights

  • Conceptual paradigm expansion of machine unlearning: Successfully transitions machine unlearning from instance-level data removal and privacy protection into surgical parameter repair for structured reasoning biases.
  • AST-grounded semantic error isolation: Replaces brittle surface string heuristics with canonicalized AST divergence analysis, ensuring that the unlearning signal is injected precisely at the structural root of the semantic breakdown.
  • Decoupled unlearning and recovery: Decoupling the suppression of erroneous branches from gold likelihood maximization avoids the severe probe-set distribution overfitting inherent to coupled preference optimization schemes like DPO.

Limitations & Future Work

  • Dependence on labeled probe data: The current framework relies on a labeled probe corpus (e.g., OmniSQL) with gold query annotations to construct AST trees and extract divergence tokens; extending the paradigm to unlabeled settings via execution feedback or weak supervision remains an open challenge.
  • Boundaries of token-level repair: Token-level unlearning is primarily effective for localized operator and column selection errors; it cannot readily resolve macro-level failures requiring global multi-step compositional query planning or deep schema-value grounding (explaining the narrower gains on BIRD compared to Spider).
  • Hyperparameter and dialect portability: Selection thresholds (\(k=5\), \(\tau=0.8\)) require recalibration across different model tokenizers and architectures; empirical validation has been centered on SQLite benchmarks, leaving dialects like DuckDB, PostgreSQL, and broader code generation tasks to future work.
  • vs. Output-Level Post-Correction (SEA-SQL / SHARE / Self-Debug): External correction mechanisms alter outputs via secondary models or multi-turn execution loops, incurring heavy latency without resolving internal model flaws; TTU-SQL fixes parameter biases directly and provides orthogonal, additive gains when composed with inference-time tools.
  • vs. Standard Machine Unlearning (TOFU / WMDP): Prior unlearning methods penalize entire sequences via gradient ascent, inducing syntax collapse in structured programming; TTU-SQL introduces fine-grained, AST-guided token-level Part-NPO to preserve structural integrity.
  • vs. Direct Preference Optimization (DPO): Applying pairwise DPO directly on probe pairs couples positive reinforcement with negative suppression, overfitting to probe-specific patterns; TTU-SQL decouples the two stages, yielding superior stability and generalizability across target benchmarks.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Beautifully repurposed machine unlearning into token-level AST parameter repair for structured semantic parsing.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous evaluation across multiple models and benchmarks, featuring comprehensive ablations, sensitivity sweeps, logit-space dynamics, and orthogonal inference composition tests.
  • Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally structured and clear, featuring robust theoretical analysis, informative diagrams, and well-substantiated empirical conclusions.
  • Value: ⭐⭐⭐⭐⭐ Provides an efficient, post-hoc parameter-level repair methodology that circumvents the need for massive retraining while outperforming standard continued fine-tuning.