Plasticity without Collapse: Plasticity-Inducing Control Optimizer for Cross-Lingual Continual Pre-Training¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/infonet-gist/infonet-ai-pico
Area: Multilingual and Machine Translation
Keywords: Cross-Lingual Continual Pre-Training, Stability-Plasticity Dilemma, Optimizer-Level Control, Spectral Sensitivity Monitoring, Coordinate-Wise Utility Gating
TL;DR¶
PICO rethinks continual pre-training as an optimizer-level control problem, combining coordinate-wise utility gating to protect key parameters, periodic spectral sensitivity monitoring to detect implicit degradation, and composed-gate stochastic perturbations to maintain adaptation plasticity, achieving positive forward transfer and the lowest worst-case collapse ratio across multilingual streams without replay buffers or model modifications.
Background & Motivation¶
Deploying large language models in non-stationary production environments requires continually adapting representations to emerging linguistic regularities and specialized domain corpora. Retraining models from scratch for each data shift is computationally prohibitive, making Continual Pre-Training (CPT) the de facto adaptation paradigm. However, sequential multi-stage training across compounded language and domain boundaries exposes an acute stability–plasticity dilemma: aggressive gradient updates tailored to incoming corpora overwrite foundational linguistic knowledge and induce catastrophic forgetting of earlier stages, whereas rigid retention constraints freeze network parameters and lead to loss of plasticity on future tasks. Existing CPT paradigms manage this tension via experience replay buffers, dynamic learning rate schedules, or parameter-efficient modules like LoRA. Nevertheless, these strategies incur substantial buffer storage or architectural modifications, while universally treating the underlying optimizer backend as a fixed, passive component.
Preliminary diagnoses conducted across cross-lingual streams highlight two fundamental failure patterns of standard optimization backends. First, existing optimizers split into distinct, compromised regimes: passive optimizers like SGD exhibit near-zero adaptation (inert regime), Sophia suffers regression under compromised curvature estimates, Adam achieves high immediate plasticity at the cost of severe catastrophic forgetting, and AdamW secures modest stability but retains only one-third of Adam's adaptation capacity. Crucially, empirical analysis uncovers a severe loss–generation mismatch: during sequential Korean-to-English medical training, the Korean cross-entropy loss monotonically decreases across stages (3.5873 to 3.4363 to 3.3381), which conventional monitoring would interpret as continuous improvement; yet the actual generation quality collapses, with BLEU dropping sharply from 0.1526 to 0.1056. Furthermore, standard stream-averaged metrics like backward transfer (BWT) and average forgetting (AvgF) routinely mask catastrophic collapse localized to a single stage behind multiple healthy stages. To faithfully capture this risk, the authors introduce the Worst Collapse Ratio (WCR), which tracks the maximum relative performance degradation observed across all previously learned domains relative to their in-stream post-training peaks.
These findings demonstrate that loss landscapes and gradient updates directly shape representational stability under cross-lingual shifts, suggesting that the stability–plasticity dilemma can be resolved directly at the update step. The core idea is to introduce PICO, an optimizer-level control mechanism that integrates coordinate-wise utility gating (attenuating updates on critical parameters), periodic matrix-level spectral monitoring (detecting geometric distortion beyond training loss), and composed-gate stochastic perturbations (reactivating exploration across low-utility coordinates) to preserve cross-lingual knowledge while sustaining forward plasticity.
Method¶
Overall Architecture¶
PICO adheres strictly to the standard PyTorch optimizer interface, serving as a plug-and-play drop-in replacement for AdamW in sequential CPT pipelines without requiring auxiliary loss functions or model parameter expansion. At each global optimizer step \(k\), given parameters \(\theta^{(k)}\) and current mini-batch gradient \(g^{(k)}\), PICO governs parameter updates through three tightly coupled mechanisms: the coordinate-wise utility gating module (GATEU) estimates the parameter–gradient product and outputs an attenuated, protected gradient; the periodic spectral monitoring module (SPECFLAG) inspects 2D weight matrices via warm-started power iterations every \(f\) steps, setting binary structural alert flags when leading singular value shifts or spectral concentration surges occur; and the composed perturbation path scales additive Gaussian noise according to instantaneous utility and structural flags, injecting exploration dynamics exclusively into uncompromised low-utility coordinates before applying decoupled weight decay.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input: Batch Gradient g and Current Parameters θ"] --> B["Coordinate-Wise Utility Gating<br/>Compute instantaneous utility and EMA to derive protected gradient"]
B --> C["Matrix-Level Spectral Monitoring<br/>Estimate spectral concentration via power iteration with threshold annealing"]
C --> D["Composed-Gate Stochastic Perturbation<br/>Modulate Gaussian noise via utility and spectral flags, apply weight decay"]
D --> E["Output: Updated Model Parameters θ_next"]
Key Designs¶
1. Coordinate-Wise Utility Gating: Destructive gradient overwriting of high-utility parameters -> Exponentially smoothed parameter-gradient statistics -> Knowledge retention without task-specific masks Standard backpropagation applies raw gradients uniformly across parameter dimensions, inadvertently destroying foundational cross-lingual representations. GATEU estimates the importance of individual parameter coordinates online by tracking the negative element-wise parameter–gradient product as an instantaneous utility proxy \(\text{instU}_\theta = -g \odot \theta\). An exponential moving average (EMA) with smoothing factor \(\beta_u\) smooths these observations over local tensor updates: $\(\text{avgU}_\theta^{(n_\theta)} = \beta_u \text{avgU}_\theta^{(n_\theta - 1)} + (1 - \beta_u) \text{instU}_\theta^{(n_\theta)}\)$ After applying stabilized bias correction, the tensor-local statistic is normalized by its infinity norm \(u_{\max, \theta} = \|\widehat{\text{avgU}}_\theta\|_\infty + \epsilon\) and mapped through a sigmoid function to construct an element-wise gate \(u \in [0, 1]\). The deterministic gradient is then attenuated according to estimated utility: $\(\tilde{g} = g \odot (1 - u)\)$ When a coordinate exhibits high utility (\(u \to 1\)), its deterministic update is suppressed toward zero, preventing destructive interference with prior cross-lingual knowledge while permitting low-utility coordinates to absorb new domain tokens.
2. Matrix-Level Spectral Monitoring: Training loss misses generation degradation and singular spectrum collapse -> Warm-started power iteration tracking spectral concentration -> Real-time structural alerts for parameter flow Because coordinate-wise statistics overlook holistic geometric deformations of weight matrices, and training cross-entropy fails to expose generation collapse, SPECFLAG periodically audits all 2D weight matrices \(W \in \mathbb{R}^{d_{\text{out}} \times d_{\text{in}}}\) at intervals of \(f\) steps. Using \(K=1\) step of warm-started power iteration, it computes the leading singular value \(\hat{\sigma}_1(W)\) and evaluates the stabilized spectral concentration ratio against the squared Frobenius norm \(\|W\|_F^2 = \sum_l \sigma_l(W)^2\): $\(\hat{\rho}_\epsilon(W) = \text{clip}_{[0, 1]}\left( \frac{\hat{\sigma}_1(W)^2}{\|W\|_F^2 + \epsilon} \right)\)$ For a matrix of rank \(r_W\), \(\hat{\rho}_\epsilon\) is bounded in \([1/r_W, 1]\), where equal singular spectra achieve the lower bound and rank-one collapse attains the upper bound. Running EMA mean and variance proxies track deviations for both \(\hat{\sigma}_1\) and \(\hat{\rho}_\epsilon\). If either standardized deviation score \(z = (x - \mu)/\sqrt{\tilde{\nu} + \epsilon}\) exceeds an annealed threshold \(\tau(c_W)\) (decayed from \(\tau_{\text{start}} = 6.0\) to \(\tau_{\text{end}} = 3.0\) after a \(c_0 = 10\) check burn-in), SPECFLAG caches a persistent binary flag \(s_W = 1\) and activates a transient optimizer-group pause \(\pi = 1\). This structural check flags latent representational collapse before it degrades downstream language generation.
3. Composed-Gate Stochastic Perturbation: Prolonged continual optimization leads to loss of plasticity -> Utility-gated and spectrum-regulated noise injection -> Revitalizing parameter exploration while preventing collapse Repeated gradient optimization over non-stationary data streams tends to trap deep networks in sharp, poorly adaptable minima, causing irreversible loss of plasticity. PICO remedies this by introducing a controlled stochastic perturbation path. To prevent noise from causing arbitrary degradation, the perturbation gate \(m^{(k)}\) integrates coordinate utility \(u^{(k)}\), the cached matrix spectral flag \(s^{(k)}\), and the group pause indicator \(\pi^{(k)}\): $\(m^{(k)} = (1 - u^{(k)})(1 - s^{(k)})(1 - \pi^{(k)})\)$ The base noise scale \(\sigma_0\) is scaled element-wise to \(\sigma_k = \sigma_0 m^{(k)}\), and Gaussian perturbations are sampled as \(\xi^{(k)} \sim \mathcal{N}(0, \text{diag}(\sigma_k^2))\). Parameter coordinates with low utility receive full stochastic exploration only when both structural indicators are inactive (\(s=0, \pi=0\)), encouraging parameter reorganization in redundant subspaces. Incorporating decoupled weight decay \(\lambda\), the final parameter update executes as: $\(\theta^{(k+1)} = (1 - \eta \lambda)\theta^{(k)} - \eta (\tilde{g}^{(k)} + \xi^{(k)})\)$ This design confines stochastic exploration to verified safe subspaces, rejuvenating adaptation plasticity without risking catastrophic parameter collapse.
Loss & Training¶
PICO is trained purely under standard causal language modeling autoregressive cross-entropy objectives without auxiliary losses or penalty functions. Standard hyperparameter settings across the primary stream include a base learning rate of \(\eta = 2 \times 10^{-5}\), decoupled weight decay \(\lambda = 0.01\), EMA coefficient \(\beta_u = 0.999\), base perturbation scale \(\sigma_0 = 0.001\), spectral check interval \(f = 1\) (and \(f = 10\) for scaling and wall-clock profiling), power iteration steps \(K = 1\), and numerical stability constant \(\epsilon = 10^{-12}\). Threshold annealing parameters are configured with warm-up check threshold \(c_0 = 10\), annealing length \(r = 30\), \(\tau_{\text{start}} = 6.0\), and \(\tau_{\text{end}} = 3.0\). On a single A100-40GB GPU training a 1B model, the amortized arithmetic overhead of spectral checks scales as \(\mathcal{O}((K+1)d_{\text{out}}d_{\text{in}} / f)\), resulting in a wall-clock update slowdown of +34.5% at \(f=1\) and +20.0% at \(f=10\) compared to AdamW, while persistent optimizer memory scales independently of the number of sequential stages.
Key Experimental Results¶
Main Results¶
The primary benchmark evaluates Llama-3.2-1B-Instruct across a four-stage sequential CPT curriculum: Korean Medical \(\to\) English Medical \(\to\) Korean Legal \(\to\) English Legal (sequence length 256). All models are evaluated post-CPT on held-out downstream tasks (Q&A and summarization) under a prompt-only zero-shot protocol, strictly excluding gold target tokens from the generation context.
The table below presents the final end-task performance across all evaluated domain–language–metric combinations (all metrics are higher-is-better, truncated to four decimal places):
| Domain | Language | Method Family / Model | ROUGE-L ↑ | Cosine Sim. ↑ | METEOR ↑ | BLEU ↑ | Token F1 ↑ |
|---|---|---|---|---|---|---|---|
| Medical | English | Initial Model (Reference) | 0.1498 | 0.4635 | 0.1966 | 0.0342 | 0.2223 |
| Medical | English | Sophia (2nd-order) | 0.0464 | 0.1290 | 0.0622 | 0.0010 | 0.0835 |
| Medical | English | SGD (1st-order) | 0.0834 | 0.2339 | 0.0743 | 0.0127 | 0.0929 |
| Medical | English | Adam (1st-order) | 0.0173 | 0.0400 | 0.0282 | 0.0002 | 0.0443 |
| Medical | English | EWC W.Adam (Regularization) | 0.0558 | 0.1390 | 0.0536 | 0.0039 | 0.0628 |
| Medical | English | Replay W.Adam (Replay) | 0.0311 | 0.1171 | 0.0326 | 0.0030 | 0.0474 |
| Medical | English | MER-Replay W.Adam (Replay) | 0.0857 | 0.2500 | 0.0820 | 0.0034 | 0.1290 |
| Medical | English | LoRA W.Adam (PEFT) | 0.0462 | 0.2213 | 0.0460 | 0.0077 | 0.0531 |
| Medical | English | Re-warm W.Adam (Schedule) | 0.0735 | 0.2206 | 0.0732 | 0.0127 | 0.0961 |
| Medical | English | PICO (Ours) | 0.1505 | 0.4387 | 0.1851 | 0.0344 | 0.2233 |
| Medical | Korean | Initial Model (Reference) | 0.1474 | 0.2285 | 0.1733 | 0.0608 | 0.2461 |
| Medical | Korean | Sophia (2nd-order) | 0.0092 | 0.0014 | 0.0287 | 0.0001 | 0.0246 |
| Medical | Korean | SGD (1st-order) | 0.1099 | 0.2196 | 0.1476 | 0.0520 | 0.1962 |
| Medical | Korean | Adam (1st-order) | 0.0098 | 0.0015 | 0.0229 | 0.0006 | 0.0318 |
| Medical | Korean | EWC W.Adam (Regularization) | 0.0173 | 0.0098 | 0.0242 | 0.0015 | 0.0265 |
| Medical | Korean | Replay W.Adam (Replay) | 0.0681 | 0.0841 | 0.0875 | 0.0188 | 0.1963 |
| Medical | Korean | MER-Replay W.Adam (Replay) | 0.1015 | 0.1269 | 0.0962 | 0.0300 | 0.2107 |
| Medical | Korean | LoRA W.Adam (PEFT) | 0.0370 | 0.0799 | 0.0508 | 0.0122 | 0.0610 |
| Medical | Korean | Re-warm W.Adam (Schedule) | 0.1172 | 0.2117 | 0.1576 | 0.0565 | 0.2176 |
| Medical | Korean | PICO (Ours) | 0.1378 | 0.2067 | 0.1589 | 0.0563 | 0.2236 |
| Legal | English | Initial Model (Reference) | 0.2012 | 0.6596 | 0.1621 | 0.0255 | 0.2684 |
| Legal | English | Sophia (2nd-order) | 0.0577 | 0.1794 | 0.0482 | 0.0037 | 0.0961 |
| Legal | English | SGD (1st-order) | 0.1254 | 0.4530 | 0.0942 | 0.0109 | 0.1663 |
| Legal | English | Adam (1st-order) | 0.0091 | 0.0196 | 0.0058 | 0.0000 | 0.0345 |
| Legal | English | EWC W.Adam (Regularization) | 0.0441 | 0.1194 | 0.0340 | 0.0000 | 0.0804 |
| Legal | English | Replay W.Adam (Replay) | 0.0216 | 0.4232 | 0.0144 | 0.0003 | 0.0453 |
| Legal | English | MER-Replay W.Adam (Replay) | 0.1509 | 0.5973 | 0.1040 | 0.0103 | 0.1809 |
| Legal | English | LoRA W.Adam (PEFT) | 0.0877 | 0.4428 | 0.0618 | 0.0064 | 0.1136 |
| Legal | English | Re-warm W.Adam (Schedule) | 0.1717 | 0.5598 | 0.1284 | 0.0201 | 0.1914 |
| Legal | English | PICO (Ours) | 0.1939 | 0.6305 | 0.1473 | 0.0236 | 0.2490 |
| Legal | Korean | Initial Model (Reference) | 0.0641 | 0.1061 | 0.0743 | 0.0228 | 0.2289 |
| Legal | Korean | Sophia (2nd-order) | 0.0087 | 0.0004 | 0.0166 | 0.0000 | 0.0243 |
| Legal | Korean | SGD (1st-order) | 0.0474 | 0.0999 | 0.0614 | 0.0186 | 0.1987 |
| Legal | Korean | Adam (1st-order) | 0.0168 | 0.0011 | 0.0171 | 0.0005 | 0.0368 |
| Legal | Korean | EWC W.Adam (Regularization) | 0.0204 | 0.0037 | 0.0153 | 0.0010 | 0.0315 |
| Legal | Korean | Replay W.Adam (Replay) | 0.0835 | 0.0517 | 0.0490 | 0.0090 | 0.2234 |
| Legal | Korean | MER-Replay W.Adam (Replay) | 0.0566 | 0.1105 | 0.0643 | 0.0170 | 0.2392 |
| Legal | Korean | LoRA W.Adam (PEFT) | 0.0180 | 0.0234 | 0.0209 | 0.0029 | 0.0547 |
| Legal | Korean | Re-warm W.Adam (Schedule) | 0.0629 | 0.1120 | 0.0719 | 0.0251 | 0.2368 |
| Legal | Korean | PICO (Ours) | 0.0655 | 0.0905 | 0.0630 | 0.0179 | 0.1969 |
Ablation Study¶
Component-level ablations conducted on the 1B backbone under identical four-stage CPT configurations (\(f=1, \sigma_0=0.001\), evaluated on ROUGE-L) isolate the exact utility of each algorithmic building block:
| Ablation Variant | FWT ↑ | BWT ↑ | AvgF ↓ | Average ↑ | WCR ↓ | Note |
|---|---|---|---|---|---|---|
| Full PICO | +0.0009 | -0.0035 | 0.0044 | 0.1369 | 0.0553 | Optimal balance of transfer & retention |
| w/o SPECFLAG | +0.0004 | -0.0015 | 0.0093 | 0.1315 | 0.1400 | WCR spikes by 2.5x without spectral check |
| w/o noise (\(\sigma_0 = 0\)) | -0.0179 | -0.0105 | 0.0116 | 0.0921 | 0.1624 | Transfer turns negative, average drops 32.7% |
| w/o GATEU, SPECFLAG, noise | -0.0286 | -0.0313 | 0.0313 | 0.0423 | 0.4709 | Severe collapse under uncontrolled updates |
Model scaling results across model sizes (with \(f=10, \sigma_0=0.001\) on ROUGE-L): - 1B Backbone: FWT = +0.0002, BWT = -0.0026, AvgF = 0.0026, Average = 0.1354, WCR = 0.0450 - 3B Backbone: FWT = +0.0012, BWT = -0.0188, AvgF = 0.0227, Average = 0.1299, WCR = 0.3292 - 8B Backbone: FWT = -0.0001, BWT = -0.0144, AvgF = 0.0182, Average = 0.1397, WCR = 0.1957
Key Findings¶
- Dominance in Low-Collapse and Transfer Regimes: PICO is the only trained method achieving positive ROUGE-L forward transfer (\(\text{FWT} = +0.0009\)) while securing the lowest Worst Collapse Ratio (\(\text{WCR} = 0.0553\)). In stark contrast, Adam exhibits catastrophic failure (\(\text{WCR} = 0.9184, \text{FWT} = -0.0979\)), losing over 91% of its post-training capability on its worst domain; even the best replay baseline (MER-Replay) suffers an elevated WCR of 0.2724.
- Strict Role Complementarity: Ablating SPECFLAG increases WCR from 0.0553 to 0.1400, proving that matrix-level spectral checks are vital for halting stage-localized catastrophic forgetting. Concurrently, removing stochastic noise causes FWT to flip negative (-0.0179) and degrades the final cross-domain average from 0.1369 to 0.0921, demonstrating that controlled perturbation is indispensable for sustaining model plasticity.
- Scale Sensitivity without Scale Invariance: Scaling from 1B to 3B and 8B reveals non-monotonic WCR trajectories (0.0450 \(\to\) 0.3292 \(\to\) 0.1957), showing that while PICO effectively maintains competitive cross-domain averages across scales, retention dynamics remain sensitive to model capacity and matrix dimensionalities.
Highlights & Insights¶
- Optimizer-as-a-Controller: Reframes CPT from data-level rehearsal or parameter-space freezing to an intrinsic optimization dynamics problem, achieving robust retention and positive transfer without modifying model architectures or storing sensitive training data.
- Spectral Concentration as an Objective Early Warning: Uncovers the loss–generation mismatch phenomenon, introducing the bounded Frobenius spectral concentration ratio \(\hat{\rho}_\epsilon\) via power iteration to provide an objective, loss-independent diagnostic for parameter manifold degeneration.
- Broad Transferability: The dual mechanism of coordinate utility gating and spectral sensitivity monitoring is model-agnostic, offering immediate extensibility to other continual learning settings, such as multi-round instruction tuning, visual-language continual adaptation, and online RL exploration.
Limitations & Future Work¶
- Author-Acknowledged Constraints: Evaluated curricula are restricted to English, Korean, and Spanish across medical and legal domains, stopping at an 8B scale; distributed sharding implementations (such as FSDP 1D views) require specialized handling to compute 2D matrix spectral statistics reliably; and maintaining fp32 master weights alongside utility EMAs incurs an additional 4.6 GiB of persistent optimizer state over AdamW.
- Potential Future Directions: Investigating hybrid architectures combining PICO's optimizer control with parameter-efficient fine-tuning (e.g., LoRA) or sparse experience replay to explore multi-level synergy; developing distributed communication-efficient spectral estimation kernels to scale monitoring seamlessly to 70B+ distributed models.
Related Work & Insights¶
- vs Adam / AdamW: Standard first-order adaptive optimizers aggressively overwrite historical features during continual distribution shifts, suffering catastrophic forgetting (\(\text{WCR} > 0.9\)); PICO incorporates coordinate gating and controlled exploration noise directly into the update step, mitigating collapse while preserving adaptive plasticity.
- vs EWC / Regularization Baselines: EWC relies on quadratic Fisher penalties that accumulate linearly with task count and fail to halt representation collapse under drastic cross-lingual shifts (\(\text{WCR} = 0.7287\)); PICO maintains a constant-size optimizer state and intervenes dynamically during gradient application.
- vs Experience Replay (Replay / MER-Replay): Replay strategies require storing historical raw text in local buffers, risking data leakage and storage overheads while remaining in negative FWT territory; PICO operates entirely buffer-free (replay-free), achieving superior forward transfer and dramatically lower collapse ratios.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Systematically tackles the cross-lingual CPT stability-plasticity dilemma through optimizer control, introducing spectral concentration monitoring and gated noise injection.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorously evaluated across 4 corpora, 9 baselines, 5 metrics under strict prompt-only zero-shot protocols, with thorough ablations and model scaling.
- Writing Quality: ⭐⭐⭐⭐⭐ Highly transparent prose detailing loss-generation discrepancies, empirical failure modes, exact algorithm recursions, and reproducibility caveats.
- Value: ⭐⭐⭐⭐⭐ Delivers an actionable, replay-free foundation for maintaining plasticity and preventing collapse in continual language model pre-training.