Neutralizing Token Aggregation via Information Augmentation for Efficient Test-Time Adaptation

Conference: ECCV 2026
Paper: ECCV 2026
Code: https://github.com/Bostoncake/NAVIA
Area: Model Compression
Keywords: Test-Time Adaptation, Token Aggregation, Mutual Information, Vision Transformer, Inference Acceleration

TL;DR

The paper formalizes plugging token aggregation into Test-Time Adaptation (TTA) to save compute as Efficient Test-Time Adaptation (ETTA), uses a mutual-information analysis to show that aggregation is a many-to-one mapping whose information loss cannot be recovered by the LayerNorm tuning that TTA usually relies on, and proposes NAVIA: two learnable information carriers — an input-level [CLS] embedding augmentation and a feature-level [CLS] bias augmentation — are attached to the [CLS] token before aggregation. With only a handful of extra trainable parameters (5.3K more than pure LayerNorm tuning on ViT-B/16), NAVIA beats the TTA state of the art (FOA) by up to 3.1% across four OOD benchmarks while achieving 14%–26% wall-clock speedup.

Background & Motivation

TTA lets a pretrained ViT adapt itself online on unlabeled samples from the test stream, without source data or labels — representative methods include Tent's entropy minimization, EATA's sample selection, SAR's sharpness-aware minimization, DeYO's disentangled-factor view, and the FOA family that works with only forward passes (or very few backward passes). These methods are effective under distribution shift, but almost all of them treat inference cost as free: a single backbone forward pass already dominates latency, SAR additionally computes second-order gradients across multiple inference cycles, DeYO forwards the same batch repeatedly under varying patch sequences, and one CoTTA iteration burns 111 GFLOPs. Once TTA has to run on edge devices or in real time, these multi-pass overheads decide whether the method is deployable at all — this is the ETTA problem the paper targets.

The most natural way to cut compute is token aggregation: ToMe progressively merges similar image tokens via lightweight bipartite matching, EViT keeps critical tokens using attention scores, and ToFu combines pruning with fusion. None of them changes ViT parameters or interferes with the optimization inside TTA, so they are nominally orthogonal to TTA and can be plugged in off the shelf. Yet when the authors attach ToMe to Tent / FOA / SAR / DeYO / CoTTA and measure, even at the mild 12.5% sparsity (r=2 tokens merged per layer) every TTA method loses more than 3% accuracy on average, and raising the compression rate to r=4 degrades it further. Two individually effective components therefore sabotage each other, and the efficiency–accuracy trade-off central to ETTA cannot be resolved by naive stacking.

The paper traces the failure to information theory. The aggregation matrix applies a many-to-one mapping to the token sequence; viewed as a Markov chain, the data processing inequality gives \(I(\mathbf{T}^{Out}_l; Y) \le I(\mathbf{T}^{In}_l; Y)\), with a strict inequality whenever aggregation is non-injective (Corollary 1). The [CLS] token after aggregation therefore carries strictly less information about the label Y than before it. Meanwhile the norm-tuning that dominates TTA only tunes LayerNorm, and LayerNorm is a deterministic map that cannot reconstruct discarded information, so \(I(\mathrm{LN}(\mathbf{T}^{Out}_l); Y) \le I(\mathbf{T}^{Out}_l; Y)\) (Corollary 2). These two results pin down the design constraint: compensation must happen before aggregation, on the input tokens; tuning normalization layers afterwards is futile. Core idea: place a learnable information carrier, built on the pretrained [CLS], before each layer's token aggregation — an input-level embedding bias δ for domain-level compensation and shallow-layer feature-level biases δ_l for the layerwise accumulation of loss — and use entropy minimization as the practical surrogate objective that raises the upper bound of learnable information.

Method

Overall Architecture

NAVIA builds on the standard TTA loop and changes only two places; the forward network and ToMe's aggregation logic are untouched. A test batch is patch-embedded with a prepended [CLS], and before the very first aggregation a learnable augmentation vector δ is added to the [CLS] embedding (input level). The batch then enters layerwise ViT encoding, where each layer first merges r tokens with ToMe using a [CLS]-dependent similarity, and the first \(L_{\mathrm{bgt}}\) blocks additionally add a per-layer bias δ_l to the [CLS] before it is aggregated (feature level). The forward pass ends by producing the [CLS] prediction from the aggregated token sequence (the prediction is taken before back-propagation, guaranteeing a single forward pass per batch), after which one backward pass on the "entropy minimization + alignment to cached source-domain layer statistics" loss updates only δ, δ_l and the LayerNorm parameters. In short, saving compute is delegated to token aggregation in the forward pass, and recovering information to learnable carriers updated in the backward pass; the two never compete for the same budget.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Test image batch (unlabeled)"] --> B["Input-level [CLS] embedding augmentation<br/>add learnable δ before aggregation"]
    B --> C["Shallow-layer [CLS] bias augmentation<br/>add δ_l in the first L_bgt blocks"]
    C --> D["Layerwise ViT encoding<br/>merge r tokens per layer with ToMe"]
    D --> E["Lightweight update anchored on the pretrained [CLS]<br/>entropy minimization + feature-statistics alignment"]
    E -->|single backward pass, only δ, δ_l and LayerNorm| B

Key Designs

1. Input-level [CLS] embedding augmentation: hang a learnable information carrier on the [CLS] before aggregation

This design addresses the blind spot exposed by Corollary 2: the aggregation matrix is non-injective, so the information in merged tokens is gone for good, and LayerNorm tuning is deterministic and cannot increase mutual information. Compensation therefore has to land before aggregation, on the input tokens. The authors deliberately avoid adding learnable prompts — in a ViT whose self-attention cost grows quadratically with sequence length, inserting extra tokens directly inflates the forward cost, contradicting the point of ETTA. Instead they reuse the existing [CLS]: prior work shows the [CLS] token encodes domain-level information useful for cross-domain learning, so it is given a bias δ learned on top of its pretrained embedding (this is the "information augmentation", a complement to norm-tuning).

The choice is backed by a theorem: for a [CLS] embedding bias δ learned through entropy minimization and an aggregation matrix \(P_0\),

\[I\!\left(\left[t_{\mathrm{CLS}}^{\mathrm{Em}}+\delta,\; P_0\mathbf{T}^{\mathrm{Em}}_{\mathrm{img}}\right]; Y\right) > I\!\left(\left[t_{\mathrm{CLS}}^{\mathrm{Em}},\; P_0\mathbf{T}^{\mathrm{Em}}_{\mathrm{img}}\right]; Y\right)\]

Placing δ before aggregation lets the information carrier enter the sequence and be compressed together with it, which raises the upper bound of learnable information; the paper treats entropy minimization as the practical surrogate for approaching that bound (Theorem 1, proof in the supplementary). In practice δ is updated directly with the gradient of the TTA loss on each test batch, \(\widetilde{\delta} = \delta - \eta_\delta \nabla_\delta \mathcal{L}(\theta; \mathbf{x})\), using SGD with the same learning rate as LayerNorm; δ is a lightweight vector updated online per batch rather than a large persistent parameter, so both its parameter and iteration overhead are negligible.

2. Shallow-layer feature-level [CLS] bias augmentation: compensate layerwise for information squeezed out along depth

The input-level δ alone is not enough. Token aggregation happens layer by layer, so the loss accumulates with depth, while δ is a single vector acting on the whole forward pass and becomes progressively weaker in deeper layers. The direct alternative — one δ_l per layer — would push trainable parameters to \(L \times d\) (for ViT-L/16, 24 layers × 1024), and with only a few hundred TTA iterations such a parameter count simply does not converge and destabilizes optimization. The paper therefore augments biases only in the first \(L_{\mathrm{bgt}}\) blocks, i.e. \(\mathbf{L}_{\mathrm{aug}} = \{0, 1, \dots, L_{\mathrm{bgt}}-1\}\), with each \(\delta_l\) (\(l \in \mathbf{L}_{\mathrm{aug}}\)) updated by the same gradient rule as the input-level term.

Three concrete arguments favor shallow layers. In terms of token features, the aggregation matrix of layer l, \(P_l = g(\mathbf{T}^{In}_l, t_{\mathrm{CLS}})\), itself depends on the [CLS], so modifying the [CLS] in shallow layers also changes which tokens later layers choose to merge — it intervenes in what gets compressed, not only in what gets added. In terms of information, layerwise aggregation is a lossy compression, so adding biases in shallow layers acts as a pre-emphasis on the information that is about to be squeezed, protecting discriminative signals in advance. In terms of architecture, ViT's residual connections let a shallow update to the [CLS] be reused by every subsequent block, whereas a late update has far fewer chances to take effect. \(L_{\mathrm{bgt}}\) is selected from \(\{4,5,6\}\) by minimizing the error of the loss in Eq. (5) on a small set of gathered target images; experiments show 4–6 layers is the stable range, while going beyond 6 saturates or slightly degrades because too many parameters cannot converge within the limited iteration budget.

3. Lightweight update anchored on the pretrained [CLS]: maximum information compensation for a minimal parameter budget

Having settled what to compensate and where, the remaining question is what carrier to use. The authors compare against two alternatives that also add trainable capacity: inserting new learnable tokens (1/2/3 per layer, 43.0K–52.2K parameters in total) and re-initializing the [CLS] before tuning it (the "Init. [CLS] Embed." row of Tab. 4). Learning tokens from scratch is hard with so few TTA iterations and underperforms the proposed scheme; re-initialization is the worst of all (65.9% average on ImageNet-C versus 69.8% for NAVIA), showing that the class-relevant information already encoded in the pretrained [CLS] must be preserved and that the augmentation has to be an incremental bias on existing semantics. A comparison against LoRA (r=1/2/3, 66.0K–121.3K parameters, at best 68.8%) leads to the same conclusion.

This defines NAVIA's efficiency boundary: on ViT-B/16 the total trainable parameter count is 43.7K, only 5.3K more than the 38.4K of standard LayerNorm tuning and smaller than any prompt-tuning or LoRA configuration, yet it is 0.8–1.9 points higher on ImageNet-C. Moreover, every modification happens on the input embedding and the shallow [CLS], so the forward graph, sequence length and FLOPs are unchanged (still 42.98 GFLOPs for ViT-L/16 at r=4). That is precisely what lets it coexist with a plug-and-play aggregator such as ToMe without adding forward passes. ⚠️ The paper does not state explicitly whether δ and δ_l persist across batches; following the main text's "for each image batch x ... we directly update", this note reads them as updated online at batch level — refer to the original paper.

A Worked Example

Take one batch (32 images, r=2) of ViT-L/16 on ImageNet-C Gaussian noise. Each 224×224 image is split into 14×14=196 patch tokens; with the prepended [CLS] the sequence length is 197. The input-level augmentation adds the learned δ to the [CLS], and the batch enters block 0: ToMe uses the [CLS] as reference for similarity matching and merges the 2 most similar image tokens, shortening the sequence to 195. Because block 0 belongs to the shallow set \(\mathbf{L}_{\mathrm{aug}}\), its [CLS] also receives δ_0 before being aggregated. Blocks 1 to 5 repeat the same "add bias → aggregate" step, and the remaining layers only aggregate. Over the 24 layers the forward pass removes the equivalent of about 48 tokens of computation, and the final [CLS] yields a prediction through the classification head; that prediction is taken first, then the loss combines entropy minimization with layer-statistics alignment, and one backward pass sends gradients back to δ, δ_0…δ_5 and the LayerNorms. The next batch repeats the whole loop — aggregation saves a little compute each time, information augmentation loses a little less information each time.

Loss & Training

The overall objective follows the established TTA recipe: entropy minimization reduces prediction uncertainty, and a feature-statistics discrepancy term alleviates feature shifting. Source statistics \(\{\mu_l^S\}, \{\sigma_l^S\}\) are computed once before deployment from 64 unlabeled source-domain images and cached (an offline pre-deployment step; no source data is needed at deployment), and the target batch's per-layer means and variances are aligned to them:

\[\mathcal{L}_{\mathrm{TTA}} = \sum_{c \in \mathcal{C}} -\hat{y}_c \log \hat{y}_c + \lambda \sum_{l=1}^{L}\left(\left\|\mu_l(\mathbf{T}_l)-\mu_l^{S}\right\| + \left\|\sigma_l(\mathbf{T}_l)-\sigma_l^{S}\right\|\right)\]

⚠️ This equation is corrupted in the cached text; the form above is reconstructed from the prose description (class set C, class probability \(\hat{y}_c\), trade-off λ, layerwise mean/variance alignment) — refer to Eq. (5) of the original paper. Hyper-parameters: learning rate 5e-3 (SGD for both LayerNorm and information augmentation), λ=30, batch size 32 for ViT-L/16 and 64 for ViT-B/16, with ToMe as the token aggregation method. Each test batch is forwarded exactly once and its prediction is read out before back-propagation; the backward pass updates only δ, δ_l and LayerNorm, never the pretrained weights.

Key Experimental Results

Main Results

Evaluation covers four OOD benchmarks (ImageNet-C at severity 5, ImageNet-R, ImageNet-V2, ImageNet-Sketch) on ViT-B/16 and ViT-L/16, at two compression levels following PYRA: moderate (r=2 for ViT-L, r=4 for ViT-B) and high (r=4 for ViT-L, r=8 for ViT-B). Token aggregation baselines are all attached to the best-performing FOABP.

Average accuracy and cost on ImageNet-C with ViT-L/16 (excerpt; all 15 corruption types in Tab. 1):

Method	ImageNet-C avg	GFLOPs	Wall-clock per batch (s)	Note
NoAdapt	61.1	55.60	0.50	no adaptation
FOABP	69.9	55.60	0.59	TTA SOTA, no aggregation
FOACMA†	68.2	1588.64	7.73	backward-free but extremely expensive
Tent	68.5	55.60	0.51	entropy minimization
SAR	60.8	111.20	1.08	collapses on some corruptions (Frost 22.3, Fog 13.5)
DeYO	44.9	111.20	0.76	collapse is worse (Noise 4.5, Weather 4.3)
ToMe2	68.9	50.15	0.51	naive combination: 1.0 below FOABP
ToMe4	65.8	42.98	0.44	4.1 drop at high compression
EViT4	54.4	46.55	0.49	15.5 drop at high compression
NAVIA2	71.9	50.15	0.51	+3.0 over ToMe2, +2.0 over FOABP
NAVIA4	70.7	42.98	0.44	+4.9 over ToMe4, +0.8 over FOABP

The same pattern holds on ViT-B/16: NAVIA4 reaches 70.7 on ImageNet-C (ToMe4 69.9, EViT4 69.1, Tofu4 69.7, TCA4 69.7), slightly above the un-aggregated FOABP (70.6) while cutting GFLOPs from 15.71 to 14.29 and wall-clock from 0.39s to 0.32s; at r=8, NAVIA8 still holds 69.8 (12.22 GFLOPs, 0.30s) against ToMe8's 68.9 and EViT8's 67.2.

On ImageNet-R/V2/Sketch (Tab. 2) with ViT-L/16, the averages are FOABP 67.1, ToMe2 65.8, Tofu2 65.7, TCA2 66.3, EViT2 60.2, while NAVIA2 reaches 68.7. On ImageNet-Sketch alone NAVIA2 scores 57.7 against FOABP's 54.6, a 3.1-point margin — the source of the "up to 3.1%" claim in the abstract. On ViT-B/16, NAVIA4/NAVIA8 obtain 66.0/65.6 and again lead the table.

Ablation Study

Contribution of the two augmentations (ViT-B/16, Tab. 3; [CLS]-E is the input-level embedding augmentation, [CLS]-S the feature-level bias augmentation):

Config	ImageNet-C	R	V2	Sketch	Avg
r=4, none	69.9	67.6	75.3	52.2	66.3
r=4, [CLS]-E only	70.3	68.2	75.2	52.6	66.6
r=4, [CLS]-S only	70.4	68.0	75.5	53.4	66.8
r=4, both	70.7	68.8	75.5	53.6	67.2
r=8, none	68.9	66.8	74.6	52.0	65.6
r=8, [CLS]-E only	69.3	67.5	75.0	52.9	66.2
r=8, [CLS]-S only	69.5	67.3	74.7	53.0	66.1
r=8, both	69.8	68.4	75.2	53.4	66.7

Design choices versus parameter budget (ViT-B/16, ImageNet-C grouped into noise/blur/weather/digital):

Scheme	Total trainable params	Noise	Blur	Weather	Digital	Avg
Shallow layers	—	63.7	65.0	75.5	73.4	69.8
Deep layers	—	63.5	64.6	74.4	72.8	69.2
Uniform	—	63.4	64.5	74.6	72.8	69.2
Re-initialized [CLS]	—	60.2	60.8	72.0	69.3	65.9
New learnable tokens	—	63.6	64.7	74.7	73.0	69.4
Tune pretrained [CLS] (ours)	43.7K	63.7	65.0	75.5	73.4	69.8
Prompt tuning +1 token/layer	43.0K	62.9	64.0	74.4	72.2	68.7
Prompt tuning +3 tokens/layer	52.2K	63.1	64.0	74.9	72.4	69.0
LoRA r=1	66.0K	63.0	64.0	71.1	72.2	67.9
LoRA r=3	121.3K	62.9	63.9	74.4	72.3	68.8

(Pure LayerNorm tuning uses 38.4K parameters as the baseline. ⚠️ The grouped numbers here differ slightly from the grouped averages of Tab. 1; they are transcribed as printed in the paper.)

Key Findings

• The two augmentations are complementary and both necessary: at r=4 the input-level augmentation alone gives +0.3 and the feature-level one +0.5, while using both gives +0.9 (66.3 → 67.2); the same holds at r=8 (65.6 → 66.7). This confirms the intended division of labor — the input level can only carry domain-level compensation, and layerwise accumulation must be handled by per-layer biases.
• Shallow-layer biases are clearly the better choice: shallow {0..5} averages 69.8, whereas deep {6..11} and uniform selection both reach only 69.2. The number of layers has a stable range too: 4–6 is optimal, and beyond 6 it saturates or degrades slightly because too many parameters cannot converge within the limited TTA iteration budget.
• The gain comes from the mechanism, not from parameter capacity: scaling capacity with prompt tuning (up to 52.2K) or LoRA (up to 121.3K) yields only 68.7–69.0, all below NAVIA's 69.8 with just 5.3K extra parameters; re-initializing the [CLS] collapses to 65.9, showing that the pretrained [CLS] semantics are an anchor that must be kept.
• The higher the compression rate, the more valuable the information compensation: on ViT-L/16, going from r=2 to r=4 drops ToMe from 68.9 to 65.8 while NAVIA only slips from 71.9 to 70.7 — more aggressive aggregation means heavier information loss, so the compensation matters more. EViT collapses at high compression on the large backbone (54.4 on ImageNet-C, only 46.8 on Sketch), empirically supporting Corollary 2.
• Faster convergence: with a fixed number of adaptation iterations on 10,000 images held out from ImageNet-R, NAVIA shows the fastest early accuracy rise and the highest final accuracy, indicating that the information augmentation does mitigate the early-stage loss caused by aggregation.
• Averages hide failures: SAR and DeYO look acceptable in the average column (60.8 / 44.9) yet collapse on entire corruption columns (Frost 22.3, Fog 13.5, Noise 4.5, Weather 4.3), i.e. these methods become completely unstable on some corruption types — a failure mode worth watching in ETTA settings.
• Memory is not the bottleneck: the peak-memory comparison (Fig. 3) shows NAVIA on par with the most efficient aggregation baselines (roughly the 8.6 GB scale for ViT-L/16 and 5.6 GB for ViT-B/16), so information augmentation adds no extra memory burden.

Highlights & Insights

• "Compensation must precede compression" is a provable design constraint rather than a tuning heuristic: the data processing inequality shows the post-aggregation [CLS] carries strictly less mutual information about the label (Corollary 1), and LayerNorm's determinism shows norm-tuning cannot restore it (Corollary 2), which prunes the entire design space down to "input tokens before aggregation". Reasoning that eliminates a whole class of invalid designs before building anything is far more convincing than stacking modules, and it transfers to any "compress-then-adapt" pipeline (quantization + TTA, token pruning + continual learning).
• Hanging the carrier on the [CLS] instead of adding prompts kills three birds with one stone: extra tokens make self-attention cost grow quadratically with sequence length and are hard to learn from scratch within the few TTA steps; the [CLS] already encodes class semantics (re-initializing it and dropping to 65.9 is the counter-proof) and participates in determining each layer's aggregation matrix \(P_l\), so modifying it intervenes in both "what gets compressed" and "what gets added".
• The residual structure is what makes shallow layers better than deep ones: a shallow update to the [CLS] is reused by every subsequent block through residual connections, whereas a late update has fewer chances to take effect. This argument transfers directly to any lightweight ViT fine-tuning work as a default prior for layer selection.
• Using capacity scaling as a falsification experiment: comparing against prompt tuning and LoRA at matched parameter budgets rules out "more parameters" as the explanation. This ablation design is worth copying — many lightweight-adaptation papers merely report "small parameter count, good accuracy" without testing capacity as the competing hypothesis.

Limitations & Future Work

• Evaluation is confined to ImageNet-derived classification benchmarks and two backbones (ViT-B/16, ViT-L/16); dense prediction tasks such as detection and segmentation are not covered, nor is validity on CLIP-style vision-language backbones or larger models verified.
• The feature-statistics term depends on source statistics cached before deployment (64 source-domain images). This is unavailable in a strictly source-free TTA setting; the authors stress that it is a one-time offline step, but it remains a stronger assumption than "no source data at all".
• The layer count \(L_{\mathrm{bgt}}\) must be chosen on a small set of target images by the loss (the paper's candidate set is {4,5,6}), which amounts to a probe of target data; only the stable range is shown in Fig. 5a, with no discussion of how much accuracy degrades when the wrong layer count is picked.
• ⚠️ Whether δ and δ_l persist across batches is not stated explicitly in the paper. In TTA, batch-level online quantities are notoriously sensitive to data order and batch size (the collapsed columns of SAR/DeYO in Tab. 1 are exactly such a symptom), and no robustness analysis of this kind is provided.
• NAVIA is fixed to ToMe as its aggregator; although EViT/ToFu/TCA are compared in the same table, the augmentation is never attached to those aggregators to verify generality. EViT's score-based selection prunes information in a different way from ToMe's merging, so it deserves a separate check.
• Memory is merely on par with the most efficient baselines rather than reduced, so the deployment benefit comes mainly from wall-clock time and GFLOPs.
• Possible improvements: replace the naive per-batch SGD for δ with a reliability-filtered update (e.g. borrowing EATA's sample selection) to keep noisy gradients out of the information carrier; or replace offline grid search over layer counts with an online per-layer information estimate.

Related Work & Insights

• vs pure TTA methods (Tent / SAR / DeYO / CoTTA / FOA): they optimize accuracy under distribution shift and barely control forward cost — SAR needs second-order gradients, DeYO forwards the same samples under different patch sequences several times, one CoTTA iteration costs 111 GFLOPs, and FOACMA reaches 1588 GFLOPs. NAVIA is over 17× faster than FOACMA on ViT-L/16 while being 2.5 points more accurate, and beats FOABP with 23% fewer GFLOPs, promoting efficiency to a first-class citizen alongside accuracy.
• vs token aggregation methods (ToMe / EViT / ToFu / TCA): they assume the downstream model's parameters are fixed and that training and inference agree, so similar tokens can simply be merged; in TTA the weights change every batch and the aggregation-induced information loss is amplified by the adaptation, so direct combination loses accuracy (4.1 points for ToMe4 and 15.5 for EViT4 on ViT-L/16). This paper compensates exactly the loss those methods do not own.
• vs structural compression / pruning / quantization / distillation: those methods rely on extensive retraining to recover the compression loss, but TTA has neither the training data nor the compute budget; token aggregation was chosen precisely because it is training-free, plug-and-play and leaves ViT parameters untouched, which is why the augmentation is designed as learnable biases instead of backbone surgery.
• vs prompt tuning (VPT / TPT) and LoRA: these dominate lightweight transfer learning, but under the very small iteration budget of ETTA, learning new tokens or low-rank increments from scratch under-converges (65.9–69.0) with larger parameter counts; the incremental "pretrained [CLS] + bias" augmentation buys a better cost–benefit ratio.
• Insight: a natural extension is to invert the framework — since shallow [CLS] biases protect information that is about to be compressed, could the norm or gradient magnitude of the augmentation vectors serve as an adaptive per-layer compression-rate signal, making aggregation strength track the degree of domain shift? That fits the spirit of ETTA better than a fixed r and connects naturally to the paper's own observation that the aggregation matrix depends on the [CLS].

Rating

• Novelty: ⭐⭐⭐⭐ First to formalize "TTA + token aggregation" as ETTA and to derive a provable design constraint (compensate before aggregation) from mutual information and the data processing inequality; both the problem definition and the theoretical motivation are substantive.
• Experimental Thoroughness: ⭐⭐⭐⭐ Four OOD benchmarks × two backbones × two compression rates, plus layer selection, token carrier, parameter budget, convergence speed, memory and t-SNE analyses — but classification only, and no discussion of batch-level adaptation stability.
• Writing Quality: ⭐⭐⭐⭐ The analysis → theorem → design chain is clear and the tables are information-dense; some LaTeX equations are corrupted in the source and the grouped numbers of Tabs. 4/5 differ slightly from the main table.
• Value: ⭐⭐⭐⭐ Offers a nearly cost-free, plug-and-play path to TTA under tight compute budgets (5.3K extra parameters, unchanged forward FLOPs), with direct practical value for edge deployment.

Related Papers

• [ECCV 2026] Audio-Visual Continual Test-Time Adaptation without Forgetting
• [ECCV 2026] MixTTA: Low-Rank Cross-Channel Mixing for Reliable Test-Time Adaptation
• [CVPR 2026] FOZO: Forward-Only Zeroth-Order Prompt Optimization for Test-Time Adaptation
• [CVPR 2026] Back to Source: Open-Set Continual Test-Time Adaptation via Domain Compensation
• [ECCV 2026] Distill Once, Adapt Life-Long: Exploring Dataset Distillation for Continual Test-Time Adaptation