LOREAL: Mitigating Low-Resolution Challenges in Vision-Language Models with Attribute-driven Prompt Self-Distillation¶

Conference: CVPR 2026
Paper: CVF Open Access
Code: Project Page
Area: Multimodal VLM
Keywords: Prompt Learning, Low-Resolution Robustness, Self-Distillation, Attribute Guidance, CLIP

TL;DR¶

To address the practical degradation of VLM performance when forced to process low-resolution inputs on edge devices, LOREAL uses LLMs to mine "resolution-robust semantic attributes" to enrich prompts. It introduces a co-distillation framework between a "standard-resolution student" and a "low-resolution student". By training only a small number of meta-nets, LOREAL significantly recovers the harmonic mean of mainstream methods like CoOp, MaPLe, MMA, and MMRL under low resolution (up to +19.95%).

Background & Motivation¶

Background: To adapt VLMs such as CLIP to downstream tasks, the mainstream approach is prompt learning (PL) within parameter-efficient fine-tuning, which freezes the entire pre-trained backbone and only inserts a few learnable prompt tokens into the text/vision encoders. Evolving from CoOp to MaPLe, MMA, and MMRL, this line of research features extremely small parameter sizes and strong compatibility.

Limitations of Prior Work: Almost all PL methods are designed and evaluated on meticulously curated standard datasets with a default fixed input resolution of 224×224. However, this contradicts real-world edge deployment scenarios: mobile or IoT devices are constrained by storage and GPU memory, hence they usually can only process low-resolution images and generate fewer visual tokens (where position embeddings are resized/interpolated). The authors define this overlooked setting as the Low-Resolution (LR) setting.

Key Challenge: The LR setting is highly practically justified—the collection, transmission, and storage overhead of low-resolution images is nearly an order of magnitude lower than that of standard resolution, and the complexity of vision encoders grows polynomially with the number of tokens, so reducing tokens saves considerable GPU memory. The authors' empirical measurements show that reducing the resolution $\phi$ saves up to 62% memory and speeds up inference by 64%. However, the cost is performance: when $\phi$ is cut to about half, the accuracy of all existing SOTA methods drops off a cliff because discriminative visual features are blurred out. "Resource saving" and "accuracy protection" are directly opposed in the LR setting.

Goal: To make the prompt learning of VLMs robust against resolution shifts without sacrificing the parameter efficiency of PL.

Key Insight: Low resolution blurs inter-class distinctions, but the authors observe that not all visual attributes are equally fragile. Coarse-grained attributes (like coat color, body shape, car windows, rooflines) remain perceptible even when resolution drops, whereas fine-grained attributes (like textures, eye colors, car logos) are rapidly lost. If the model can be guided to focus on those "resolution-robust" attributes, it has a promising chance of withstanding blur.

Core Idea: Utilize an LLM to automatically mine robust attributes that remain salient under resolution variations, structurally embed them into prompts, and let a dual-student framework (one standard-resolution student and one low-resolution student) mutually distill each other via shared meta-nets. This forces the model to align semantics across both resolutions—trading "attribute guidance + cross-resolution self-distillation" for LR robustness.

Method¶

Overall Architecture¶

LOREAL (LOw-REsolution Attribute-guided prompt Learning) is a prompt self-distillation framework. The entire pipeline consists of three steps: Offline robust attribute generation via LLM $\rightarrow$ Dynamic embedding of visual features into attribute prompts via meta-nets $\rightarrow$ Cross-resolution mutual distillation training of dual students. It does not retrain the backbone; the only learnable components are $K$ meta-nets, allowing it to be directly integrated as a plug-and-play enhancement for existing PL methods like CoOp, MaPLe, MMA, and MMRL.

During training, there are two student VLMs sharing the meta-nets: student $\alpha$ takes standard-resolution images $x$, and student $\beta$ takes low-resolution images $x'$, with their backbones fully frozen. The key lies in the "cross-bridging" of the meta-nets of the two students: the visual embeddings of $\alpha$ are used to fill the text prompts of $\beta$, and the visual embeddings of $\beta$ serve to fill the text prompts of $\alpha$. This forces visual information from different resolutions to generate mutually consistent attribute contexts. Double-layer distillation is stacked on top of this setup: Low-Level Distillation (LLD) aligns the generated attribute contents of both paths, and High-Level Distillation (HLD) aligns their output logits. During inference, only the low-resolution path is retained: the LR image is passed through the meta-nets to generate attribute concepts, which are filled into the prompt to compute cosine similarity with each category's text features to yield predictions.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Images<br/>Standard x / Low-res x'"] --> B["LLM Mines Resolution-Robust Attributes<br/>CoT: Describe LR appearance -> Extract invariant attributes"]
    B --> C["Attributed prompt<br/>'A photo of [CLS] with S1[A1]...SK[AK]'"]
    C --> D["Cross-modal meta-nets<br/>LoRA-style, writing visual embeddings into attribute content Sk"]
    D --> E["Dual-Student Cross-Bridging<br/>α vision fills β text / β vision fills α text"]
    E --> F["LLD Low-Level Distillation<br/>Aligns attribute contents (contrastive)"]
    E --> G["HLD High-Level Distillation<br/>Aligns logits (KL)"]
    F --> H["L = LCE + λ1·HLD + λ2/K·LLD"]
    G --> H
    H -->|Inference (LR path only)| I["LR Image -> meta-net -> attribute prompt -> Prediction"]

Key Designs¶

1. LLM-Mined Resolution-Robust Attributes: Directing prompts to focus on features discernible in blurred states

The fundamental challenge of LR is that discriminative features are blurred out. Therefore, instead of forcefully learning all attributes, LOREAL employs an LLM with an explicit Chain-of-Thought (CoT) to filter a set of "resolution-invariant" attributes. This is implemented in two steps: first, the LLM is prompted to describe what each category roughly looks like in a low-resolution photo (e.g., "Under low resolution, an Abyssinian presents as a sleek warm brown..."). Second, taking these descriptions as context, the LLM summarizes several generic attributes that remain perceptible as the resolution varies. This step naturally favors macro-level attributes (such as coat color/body shape for OxfordPets, car windows/rooflines for StanfordCars) while filtering out details that vanish once blurred, like textures, eye colors, or car logos. The authors emphasize that while both macro and fine-grained attributes are considered "general" at standard resolution, the key insight here is to specifically select the subset that remains salient in LR. After obtaining the raw attributes $\{A_k\}_{k=1}^K$, a contextualized prompt is constructed:

\[\text{A photo of a [CLS] with } S_1 [A_1]\, S_2 [A_2] \cdots S_K [A_K]\]

where $S_i \in \mathbb{R}^{M\times D_t}$ is the learnable token corresponding to each attribute ($M$ is the number of tokens per attribute). The order of attributes has minimal impact on performance, thus it is not deliberately arranged. This upgrades "which features to pay attention to" from blindly learnable prompts to those with semantic priors specifically targeting blur-resistant content.

2. Cross-Modal Meta-Nets: Dynamically activating prompts with visual features instead of learning static ones

Merely having static attribute slots is insufficient—different images of the same class should activate attribute intensities differently. LOREAL equips each attribute with an independent meta-net $\{M_k\}_{k=1}^K$ to dynamically transform the output visual embedding $f_v$ into corresponding attribute content:

\[S_k = M_k(f_v) = W_{\uparrow,k}\big(W_{\downarrow,k}(f_v)\big)\]

where $W_{\downarrow,k}\in\mathbb{R}^{D_v\times D_s}$ and $W_{\uparrow,k}\in\mathbb{R}^{D_s\times D_t}$ are LoRA-style down-to-up projection matrices, and $D_s$ is the bottleneck dimension (set to 32 by ablation). This is a lightweight LoRA-like structure with minimal parameters (only +104.5K trainable parameters for the entire model). Its mechanism is to "pour" visual semantics into the text-side attribute prompts, conditioning text features on the image, equivalent to image-guided instance-level contextualization on text prompts. This bridge is precisely what makes the subsequent cross-resolution distillation possible.

3. Dual-Student Cross-Bridged Self-Distillation: Constraining different resolutions to generate consistent attribute contexts

Simply fine-tuning attribute prompts is inadequate because the model only witnesses low-resolution images during inference. LOREAL introduces two students $\{E^\alpha_t, E^\alpha_v\}$ and $\{E^\beta_t, E^\beta_v\}$ whose backbones are fully frozen and only meta-nets are learnable, handling standard-resolution and low-resolution images respectively, while sharing the same set of meta-nets. The core design is the cross-bridging—the visual embeddings of one branch generate the text attributes of the other branch:

\[f^\alpha_v = E^\alpha_v(x),\quad f^\beta_t = E^\beta_t\big(p(S(f^\alpha_v))\big)$$ $$f^\beta_v = E^\beta_v(x'),\quad f^\alpha_t = E^\alpha_t\big(p(S(f^\beta_v))\big)\]

In other words, the standard-resolution visual features are forced to support the low-resolution student's text prompt, and vice versa. This self-distillation, where "one acts as one's own teacher, teaching each other bidirectionally," forces the shared meta-net to learn to generate interchangeable and aligned attribute contents from visual inputs of arbitrary resolutions, thereby progressively pulling the semantics of both resolutions together in the multimodal manifold space. Unlike classic KD (where the student is fully tuned) or PromptKD (which relies on caching teacher logits), LOREAL does not require an external large teacher; the two students are simply different resolution views of the same model.

4. LLD + HLD Double-Layer Distillation: Aligning cross-resolution semantics at both attribute and final prediction levels

The cross-bridging only establishes the channels for "mutually filling prompts." Truly aligning the two paths relies on two distillation losses. Low-Level Distillation (LLD) applies contrastive learning between the attribute contents $S(f^\alpha_v)$ and $S(f^\beta_v)$ generated by both paths, encouraging the same attribute to generate similar contents across different resolutions while distinguishing different attributes:

\[L_{LLD} = -\sum_{k=1}^{K}\log\frac{\exp\big(\mathrm{Sim}(S_k(f^\alpha_v), S_k(f^\beta_v))/\tau\big)}{\sum_{k'}\exp\big(\mathrm{Sim}(S_k(f^\alpha_v), S_{k'}(f^\beta_v))/\tau\big)}\]

High-Level Distillation (HLD) aligns the cross-resolution semantics via KL divergence between the class predictions $\hat y^\alpha$ and $\hat y^\beta$ of both paths:

\[L_{HLD} = \sum_{c=1}^{C}\big(\hat y^\alpha_c\log\hat y^\alpha_c - \hat y^\alpha_c\log\hat y^\beta_c\big)\]

The objective function fuses the task cross-entropy $L_{CE}=-\sum_c y_c\log\hat y^\beta_c$ (computed on the low-resolution student) and the two distillation terms: $L = L_{CE} + \lambda_1 L_{HLD} + \lambda_2\cdot\frac{1}{K}\cdot L_{LLD}$. Ablations exhibit that LLD contributes more than HLD since attribute-level alignment is easier to optimize than logit-level alignment—explaining why the optimal $\lambda_2$ is set higher than $\lambda_1$.

Loss & Training¶

The pre-trained CLIP-ViT-B/16 is adopted as the backbone and frozen throughout, with only the meta-nets being learnable. The SGD optimizer is used with a learning rate of 0.002, evaluated under 16-shot (16 samples per category). Hyperparameters: meta-net intermediate dimension $D_s=32$, number of tokens per attribute $M=2$, $\lambda_1=1$, $\lambda_2=2$, $\tau=4$ (note that the softmax temperature is written as $\tau=1$ in preliminaries, while the distillation temperature is $\tau=4$). GPT-4o is employed as the LLM to generate 5 attributes per class. When combining with methods that cannot decouple visual/textual embeddings (e.g., MaPLe/MMRL), their visual embeddings are first cached offline before distillation. Training epochs and resolution $\phi$ are set individually for each baseline.

Key Experimental Results¶

Three new benchmarks are evaluated under 16-shot using CLIP-ViT-B/16. LOREAL is plugged into CoOp/MaPLe/MMA/MMRL, with resolutions set to $\varphi\in\{96^2,144^2,192^2\}$.

Main Results¶

LR-B2N (Low-Resolution Base-to-New, 11 datasets, average Harmonic Mean (HM) under $\varphi=96^2$):

Method	Base	Novel	HM	+LOREAL HM	Gain
CoOp	38.40	33.74	35.54	47.48	+11.94
MaPLe	37.17	33.71	34.85	57.25	+22.40
MMA	41.55	39.91	40.50	63.14	+22.64
MMRL	41.64	36.92	38.90	61.71	+22.81

Consistent improvements are also observed in cross-dataset (LR-CE) and domain generalization (LR-DG): LR-CE on the source domain ImageNet improves by +21.0%, +6.58%, and +2.54% respectively for the three resolutions of $\phi$; LR-DG achieves an average of +12.71% improvement across four ImageNet variants under $\varphi=96^2$. The pattern is that the lower the $\phi$, the higher the gain—precisely corresponding to the most challenging and realistic edge deployment scenarios.

Efficiency (Table 4, $\varphi=96^2$): Integrating LOREAL introduces only +104.5K trainable parameters, adding +4~5ms of training time per sample, +1ms during inference, and +33MB of VRAM overhead. Yet, it recovers MaPLe's HM from 34.85 to 57.25 and MMRL's HM from 38.90 to 61.71, yielding substantial gains at virtually no cost.

Ablation Study (Table 6, Components; HM)¶

Config	Base	New	HM	Description
LR->St. only	75.45	63.42	68.91	Only use LR embeddings to fill standard student prompts
St.->LR only	74.78	62.46	68.07	Only use standard embeddings to fill LR student prompts, 0.84% lower than the former
Bidirectional + missing one distillation layer	75.70 / 76.10	64.72 / 66.35	69.78 / 70.89	Removing either LLD or HLD leads to notable performance drops
Full (Bidirectional + LLD + HLD)	77.30	67.37	71.73	Full model

Other hyperparameter ablations: The optimal $D_s$ is 32 (too large makes optimization difficult); the optimal distillation temperature $\tau$ is 4 (too small or too large leads to over-sharpness or over-flatness); increasing the number of attribute tokens $M$ leads to a slight performance drop (~2%), so it is set to 2; $\lambda_1/\lambda_2$ is set to 1/2 after grid search.

Key Findings¶

LLD is more critical than HLD: Removing LLD yields a larger drop than removing HLD because direct alignment at the attribute level is easier to optimize than at the logit level, which also accounts for $\lambda_2 > \lambda_1$.
Aligning LR inputs during training is more critical: LR $\rightarrow$ St. is slightly better than St. $\rightarrow$ LR by 0.84% HM, indicating that firmly establishing image-text alignment for LR inputs during training is more beneficial for LR inference generalization.
The lower the $\phi$, the higher the gain: Low resolution is precisely the interval where existing methods collapse most severely and where LOREAL's correction is most prominent, validating the value of the attribute-level robust prior.
Visualization (Figure 6) reveals that after adding LOREAL, the attention heatmaps stably focus on robust attribute regions despite resolution degradation, and the learned attribute tokens correspond to interpretable semantics such as "coat color", "short legs", or "silky".

Highlights & Insights¶

Redefining an overlooked real-world problem: It explicitly formalizes "edge low-resolution inference" as the LR setting, quantifying its GPU memory/speed benefits (up to 62% memory savings, 64% speedup) alongside the accuracy trade-offs, presenting a clean problem framing.
Clever attribute filtering insight: It is not the concept of "mining attributes" that is novel, but rather "mining only the resolution-robust attribute subset." By using the LLM's CoT to explicitly distinguish between macro-level (blur-resistant) and fine-grained (easily lost) attributes and injecting this prior into the prompts, the approach is highly targeted.
Self-distillation without an external teacher: By employing standard- and low-resolution views of the same model to mutually act as teachers, combined with the shared and cross-bridging meta-nets, cross-resolution alignment is cast as a self-supervised constraint. This is lighter and more self-contained than PromptKD, which relies on caching teacher logits.
Plug-and-play gain at near-zero cost: Introducing only +104.5K parameters brings pervasive gains to four mainstream PL methods. This "mountable" design holds strong engineering deployment value, as any prompt learning method can theoretically integrate it.

Limitations & Future Work¶

Attribute generation relies on offline calls to GPT-4o, and the quality of the attributes is heavily tied to category names. Its efficacy on open-vocabulary or vague-category tasks (where attributes are hard to describe accurately by LLM) remains unverified.
Experiments are conducted solely on CLIP-ViT-B/16 under a 16-shot setting. It is not reported whether the gains persist with larger backbones or full-set data. The resolutions tested only cover $96^2 \sim 192^2$, leaving a lack of quantitative results for more extreme ultra-low resolutions (e.g., $48^2$, which only appears in visualizations).
Training requires simultaneous forward passes of two students. Although only one path is kept during inference, the training VRAM/time overhead is still doubled compared to single-student PL, a point downplayed in the paper under the "+ms" metric.
Future directions: Transitioning attribute mining from "one-time offline" to adaptive online updates; exploring structural relationships among attributes (currently explicitly stated as order-independent); or applying uncertainty weighting to attributes that still risk being lost under LR.

vs CoOp / MaPLe / MMA / MMRL: These methods are standard-resolution prompt learning fixed at 224×224. This work does not replace them but serves as an add-on plugin specifically to patch the weakness in LR robustness.
vs PromptKD: PromptKD distills from an external large teacher using prompts, tailored for non-LR inference. LOREAL requires no external teacher, utilizes dual-resolution views of the same model for self-distillation, and aims to resist resolution drift.
vs Low-Resolution Recognition (ResFormer / MSPE / PixelDistillation): These works primarily adapt to LR by designing flexible position encodings or distilling small CNN models, focusing on the vision backbone. LOREAL is the first to tackle the LR challenge within the post-VLM prompt learning framework, taking the route of attribute guidance and cross-modal self-distillation.

Rating¶

Novelty: ⭐⭐⭐Formulates low-resolution edge deployment within VLM prompt learning for the first time; the combination of a robust attribute subset and dual-student self-distillation is novel.
Experimental Thoroughness: ⭐⭐⭐⭐ Covers 3 benchmarks $\times$ 4 baselines $\times$ 3 resolutions + comprehensive component/hyperparam ablations, although the backbone models and shot numbers are somewhat homogeneous.
Writing Quality: ⭐⭐⭐⭐ Motivating quantifications are solid and figures are clear; some notations (dual $\tau$ definitions, Table 6 row correspondences) require verification with the original text.
Value: ⭐⭐⭐⭐ Plug-and-play with near-zero cost, highly aligned with edge deployment, and offers strong practicality.