Rapidly Deploying On-Device Eye Tracking by Distilling Visual Foundation Models¶

Conference: ECCV 2026
Paper: ECCV 2026
Area: Model Compression
Keywords: knowledge distillation, eye tracking, visual foundation model, synthetic-to-real, on-device deployment

TL;DR¶

DistillGaze first self-distills DINOv3 into a domain-specialized 86M teacher using labeled synthetic eye images plus unlabeled real infrared images, then distills that teacher into a 256K on-device student, using no real gaze labels at all; on the Project Aria crowd-sourced benchmark it cuts median gaze error from 3.48° to 1.44° (-58.6%) and tail error E90U90 from 14.84° to 8.45° (-43.1%) with no increase in model size.

Background & Motivation¶

Eye tracking is a core sensing capability for AR/VR: foveated rendering, gaze-driven interaction, and attention-aware interfaces all depend on it. Yet getting a high-accuracy gaze estimator onto a new piece of hardware is extraordinarily expensive. A team must walk through hardware prototyping, calibration, data collection, and annotation before it has usable gaze ground truth, and almost none of that work transfers — any change in camera placement, camera pose, illumination geometry, sensor characteristics, or even the on-device image processing pipeline can invalidate the previously trained model. Every device generation therefore triggers a full repeat of data acquisition and retraining, and under crowd-sourced protocols a single device variant costs weeks to months of collection. Gaze supervision itself is also unreliable: even under controlled calibration protocols, subjects exhibit imperfect fixation and incomplete compliance, producing label noise that is essentially undetectable at scale.

Synthetic data is the most practical way out. Given a device's camera intrinsics/extrinsics and illumination geometry, photo-realistic near-eye images with pixel-perfect gaze labels can be rendered at scale in Blender, covering eye appearances and gaze directions that are impractical to capture with calibrated ground truth; because existing 3D eye assets and subject appearance models are reusable across generations, only the known optical parameters need updating, compressing turnaround from months to days. The catch is a residual synthetic-to-real gap in texture, noise characteristics, and optical artifacts. The obvious next step is to skip adaptation entirely and lean on visual foundation models (DINOv3, SAM3, Sapiens2), which transfer extremely well on natural-image benchmarks. The paper's experiments say no: with frozen DINOv3 ViT-B (86M) under linear probing, gaze error exceeds 5° — 336× more parameters than the comparison model, yet clearly worse than a 256K on-device model trained on synthetic data alone, a pattern that holds across the whole DINOv3 ViT and ConvNeXt family. A t-SNE visualization explains why: DINOv3 embeddings of near-eye images cluster tightly by subject identity, with gaze direction suppressed, yet the intra-cluster distribution shows smooth gradients correlated with gaze — the gaze signal is genuinely encoded, but organized as an identity representation that must be reshaped in-domain. That is the core tension: VFMs carry the signal but not in a usable form, and the compute needed for them to reach their best performance is far beyond an on-device budget.

The core idea is to split "extracting the signal from a VFM" and "deploying on device" into two sequential steps: first use synthetic labels plus unlabeled real images (self-distillation + pseudo-labels) to optimize the VFM into a domain teacher, then distill that teacher into a 256K on-device student, with no real gaze label anywhere in the pipeline. The absence of real gaze labels means a new device does not have to wait for a collection cycle, and unlabeled real images can be gathered passively during normal use before a calibrated eye tracking system even exists — which is the operative meaning of "rapidly deploying": not that training is faster, but that adapting to a new hardware configuration is no longer bottlenecked by the annotation cycle.

Method¶

Overall Architecture¶

DistillGaze is a two-stage framework. Its input is a synchronized pair of near-infrared eye images and its output is a per-eye gaze direction (yaw and pitch). Stage one (VFM optimization) starts from an off-the-shelf DINOv3 ViT-B and runs teacher-student self-distillation over labeled synthetic eye images plus unlabeled real eye images, reshaping a representation organized by identity into one organized by gaze regression and yielding an 86M optimized VFM teacher. Stage two (on-device distillation) transfers that teacher's knowledge into a 256K on-device student; only the student is deployed at inference time, while the teacher and the EMA student exist only during training. Both stages share one device-aware strong/weak augmentation pipeline that simulates the degradation of real on-device cameras and illumination.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Synthetic + unlabeled real eye images"] --> B["Device-aware strong/weak augmentation<br/>MTF / motion blur / glint"]
    B --> C["Mixed-supervision VFM in-domain optimization<br/>synthetic labels + real-image self-distillation"]
    C --> D["Optimized VFM teacher<br/>86M, distillation source only"]
    D --> E["Two-source distillation to the on-device student<br/>feature VIC-KD + predictive pseudo-labels"]
    G["EMA student<br/>momentum copy of the student"] --> E
    E --> F["256K on-device student<br/>FBNet backbone, real-time"]

Note that although both stages speak of a "teacher" and a "student," the roles are entirely different. In stage one both are DINOv3 backbones and the teacher is merely an EMA copy of the student, so the goal is to let DINOv3 grow gaze structure inside the eye-tracking domain. In stage two the teacher is that already-optimized 86M VFM, the student becomes an FBNet-backbone on-device model, and a 336× parameter gap opens up between them; the difficulty there is not same-architecture self-distillation but moving knowledge across capacity and architecture.

Key Designs¶

1. Device-aware strong/weak augmentation: turning unlabeled real images into usable supervision

Both stages rely on unlabeled real eye images to bridge the synthetic-to-real gap, but an unlabeled image carries no supervision by itself; the only way to extract information is the requirement that different views of the same image yield consistent gaze. The paper therefore designs two augmentation pipelines and deliberately assigns them by model role: the teacher and the EMA student see only weakly augmented views (gamma jitter plus random scaling), while the student sees strongly augmented views. Weak augmentation keeps the teacher-side supervision target stable and trustworthy; strong augmentation forces the student to reproduce the teacher's and EMA student's judgments under adverse conditions.

Beyond weak augmentation, the strong pipeline stacks two families of transforms, each triggered with probability 0.3, and these are not generic image augmentations but deliberately shaped around eye-tracking failure modes. Camera augmentations simulate sensor degradation through modulation transfer function (MTF) filtering, motion blur, and compression artifacts. Illumination augmentations target imaging problems specific to active infrared illumination: brightness and contrast adjustment, glint inpainting, random shadows, and coarse dropout. Corneal glints are the most characteristic artifact of head-mounted infrared eye tracking — the glint and the pupil highlight directly corrupt the pupil boundary, so a model trained only on clean renders will mistake these artifacts for appearance cues on a real device. The point of this design is that it makes passively collected unlabeled real images genuinely trainable: the student is asked to stay consistent with a clean, weakly augmented teacher after these degradations have been injected, which effectively turns the domain gap into data augmentation.

2. Mixed-supervision VFM in-domain optimization: letting a general representation grow gaze structure

In stage one, synthetic-only fine-tuning (DINOv3 SynFT in the paper) already pulls E50U50 from the linear-probe 5.47° down to 2.01°, but it still treats the VFM as an ordinary backbone and uses none of the unlabeled real data. Unsupervised domain adaptation (UDA) is the other natural candidate; the paper tries DARE-GRAM, a UDA method designed for regression, and it is actually worse than synthetic fine-tuning (4.84° vs 2.01° at E50U50), which the authors attribute to overfitting to the synthetic distribution that amplifies the synthetic-real mismatch. What works instead is wiring "synthetic supervision" and "self-distillation on unlabeled real images" into a single teacher-student framework: the teacher sees weakly augmented images and the student strongly augmented ones, both pass through a projector to produce embeddings, and an embedding-alignment loss pulls the two sides together; a synthetic supervised gaze loss is applied on labeled synthetic samples, while on unlabeled real samples the teacher's prediction serves as a pseudo-label for the student.

\[\mathcal{L}_{\text{OptVFM}} = \lambda_{\text{SynSup}}\sum_{X\in\mathcal{D}_{\text{syn}}}\mathcal{L}_{\text{SynSup}} + \sum_{X\in\mathcal{D}_{\text{syn}}\cup\mathcal{D}_{\text{real}}}\left(\mathcal{L}_{\text{SD}} + \mathcal{L}_{\text{Pseudo}}\right)\]

(⚠️ refer to the original paper: the cached PDF text is garbled here; the equation above is reconstructed from the surrounding prose.) The weight \(\lambda_{\text{SynSup}}\) is not fixed but annealed on a cosine schedule: early training leans heavily on synthetic supervision to force the VFM representation toward the gaze task, and then the emphasis shifts to self-distillation and pseudo-label supervision over all available data, letting the real-domain distribution take over the details. Teacher parameters are updated by EMA (\(\theta_t \leftarrow \alpha\theta_t + (1-\alpha)\theta_s\), where \(\theta\) covers backbone, projector, and gaze head) so the student never chases a target that is itself moving fast. The ablation shows this recipe pushes E50U50 from 2.01° to 1.33° (-33.8%) and E90U90 from 12.11° to 8.32° (-31.3%) — and this 86M teacher is never deployed; its entire value is being the knowledge source for stage two, which is why spending 336× the parameters on the teacher side is a rational trade.

3. Two-source distillation to the on-device student: feature-space alignment and predictive pseudo-labels together

Stage two has to cross a 336× capacity gap, and a single type of supervision is not enough. The paper sets up three models: the teacher is the optimized VFM from stage one (its backbone and gaze head are available), the student is a 256K on-device model pretrained on synthetic data (FBNet backbone, weights shared between the left and right eyes), and an additional EMA student holds exponential moving averages of the student's backbone, projector, and gaze head. The design is inspired by Community KD but differs in two key ways: it distills in both feature space and prediction space, and the EMA student is updated by momentum rather than gradient descent, borrowing from momentum-based self-supervised learning such as BYOL/DINO. In feature space the student is therefore aligned with two targets at once, one projection matching the teacher (\(z_{s\to t}\)) and one matching the EMA student (\(z_{s\to e}\)), each with its own projector.

Feature-space distillation uses a VIC-KD style composite loss — an invariance loss between the teacher and student projections, plus variance and covariance regularization that prevents the student from collapsing its representation into a constant just to match the teacher:

\[\mathcal{L}_{\text{KD}} = \lambda_{\text{inv}}\mathcal{L}_{\text{inv}} + \lambda_{\text{var}}v(\cdot) + \lambda_{\text{cov}}c(\cdot)\]

(⚠️ refer to the original paper: subscripts and regularizer arguments are garbled in the cached text; this form is reconstructed from the prose, where \(v(\cdot)\) and \(c(\cdot)\) denote the VICReg variance and covariance regularizers.) In prediction space, both sources supply pseudo-labels: the teacher contributes \(\mathcal{L}_{\text{Pseudo-}t}\) and the EMA student contributes the self-distillation gaze term \(\mathcal{L}_{\text{Pseudo-}e}\), so the student's gaze head is pulled by two stable targets simultaneously. The relative weight \(\lambda_t\) also follows a cosine schedule — early training favors distillation from the optimized VFM so the student first inherits structured representations, then the balance gradually shifts toward EMA self-distillation, refining details against a smoother momentum target. The arrangement is complementary by construction: the teacher is large but its targets are "hard," while the EMA student is isomorphic to the student and produces "smooth" targets, so each covers the other's weakness at a different stage of training. At inference only the student network is exported; the teacher and EMA student are discarded, so the on-device cost is identical to that of the synthetic-supervision baseline.

A Worked Example¶

Take one training iteration of stage two. A pair of near-infrared eye images (640×480 native, downsampled to 320×240 following [29]) goes down two paths. The weakly augmented view enters the 86M optimized VFM teacher, producing a projection \(z_t\) and a gaze prediction \(\hat y_t\). The strongly augmented view of the same pair enters the 256K student (FBNet backbone with weights shared across eyes, features concatenated and mapped to per-eye yaw/pitch by the gaze head), producing \(z_s\) and \(\hat y_s\). The EMA student, also fed the weakly augmented view, produces \(z_e\) and \(\hat y_e\). The student's gradient comes from four terms: an invariance loss against \(z_t\) (with variance and covariance regularization), an L2 loss against \(z_e\), pseudo-label regression onto \(\hat y_t\), and self-distillation regression onto \(\hat y_e\), with the two sources weighted by the cosine schedule \(\lambda_t\). The teacher is frozen; the EMA student's backbone, projector, and gaze head each roll forward from the student's corresponding parts by momentum. When the iteration ends, only the student's 256K weights survive into the deployment package.

Loss & Training¶

Gaze supervision uses a smooth L1 loss with outlier rejection (following [29]): quadratic for small errors, linear for moderate errors, and scaled by a factor \(k<1\) beyond a threshold \(\gamma\) for outliers, which provides robustness to label noise in crowd-sourced annotations. Notably, on the teacher side the paper drops centering-and-prototype soft-clustering losses of the DINO family in favor of plain MSE regression alignment, for two reasons: soft clustering induces discrete, categorical representations that conflict with the fine-grained continuous regression gaze estimation requires, and synthetic supervision already supplies enough regularization to prevent representation collapse without such a mechanism. The ablation confirms this choice.

The on-device model has 256K parameters, an FBNet backbone, and shared weights across the two eyes. Optimization uses AdamW with a cosine learning-rate schedule and 10% warm-up, batch size 256, learning rate adjusted between \(10^{-3}\) and \(10^{-5}\), and up to 50,000 iterations per experiment. For self-distillation the EMA teacher is updated every 100 iterations with momentum adjusted between 0.95 and 0.99, and the feature projection head is three fully connected layers with GELU activations. VFM experiments initialize from the ViT-B variant of DINOv3 (86M), and all experiments run on 4× NVIDIA A100 80GB GPUs. On the data side: the real dataset comes from Project Aria with 6,299 recordings from 2,222 crowd-sourced participants split into 1,825 training and 397 validation participants, and the ground-truth gaze labels of the training set are withheld entirely to simulate the unlabeled setting; two global-shutter monochrome cameras record at 20 fps for roughly 60 seconds per recording under diffused infrared illumination, temporally subsampled by a factor of 10 for training. The synthetic data comprises 165K frames from 998 subjects rendered in Blender.

Key Experimental Results¶

Evaluation follows the Error-User (EU) table: per-user percentiles E50/E75/E90 of the per-frame angular error are computed first, then aggregated across users at U50/U75/U90, so E50U50 is the median error of the median user and E90U90 characterizes the tail. Angular error is the angle between predicted and ground-truth 3D gaze vectors, averaged over the left and right eyes per frame. Sixty-four frames are uniformly sampled per recording, 9 reserved for test-time personalization and 55 used for metrics, and 95% confidence intervals come from a hierarchical bootstrap over users and frames (1,000 iterations).

Main Results¶

Teacher side (is optimizing the VFM worth it):

Method	Inference params	E50U50 ↓	E75U75 ↓	E90U90 ↓
On-device SynSup (synthetic supervised)	256 K	3.48	7.41	14.84
DINOv3 linear probe	86 M	5.47	10.64	18.74
DINOv3 SynFT (synthetic fine-tuning)	86 M	2.01	4.29	12.11
DARE-GRAM (UDA)	86 M	4.84	9.36	17.89
Optimized VFM (our stage one)	86 M	1.33	3.07	8.32

On-device students (⋆ = ours; bold = best among on-device students):

Method	Inference params	E50U50 ↓	E75U75 ↓	E90U90 ↓
On-device SynSup	256 K	3.48	7.41	14.84
DARE-GRAM	256 K	2.96	6.20	13.41
On-device self-distillation (no VFM teacher)	256 K	2.20	4.64	10.95
Optimized VFM ▲ (not deployed)	86 M	1.33	3.07	8.32
Pseudo labels only	256 K	1.59	3.49	8.19
SP	256 K	1.59	3.50	8.21
VIC-KD	256 K	1.46	3.32	8.40
Community KD	256 K	1.48	3.24	8.47
DistillGaze (ours)	256 K	1.44	3.29	8.45
Fully supervised (upper bound, reference)	256 K	0.82	1.94	5.91

Ablation Study¶

Config	Backbone / params	E50U50 ↓	E90U90 ↓	Note
Synthetic supervised baseline	256 K	3.48	14.84	synthetic labels only
+ DARE-GRAM	256 K	2.96	13.41	teacher-free UDA, limited gain
+ on-device self-distillation (no VFM teacher)	256 K	2.20	10.95	unlabeled real data helps on its own, -36.8%
+ pseudo labels only (with VFM teacher)	256 K	1.59	8.19	the single largest gain, from adding the teacher
DistillGaze (full)	256 K	1.44	8.45	feature distillation adds a little more at E50
Teacher with DINO-style soft-clustering loss	86 M teacher	1.50	8.86	plain MSE is better for regression

Key Findings¶

Optimizing the teacher is the big step, not the distillation trick. Going from synthetic supervision (3.48°) to pseudo-label distillation with a VFM teacher (1.59°) captures most of the gain; within the distillation stage itself, VIC-KD, Community KD, SP, and pseudo-labels-only differ by only 1.44°–1.59° at E50U50, indicating that when the teacher is strong enough, student performance is driven more by teacher quality than by the specific distillation objective. The loss-function ablation on the teacher side shows the same effect: plain MSE regression (1.33°/8.32°) clearly beats a DINO-style soft-clustering loss (1.50°/8.86°), because for continuous regression a discretizing representation target is itself a burden.
Unlabeled real images really do close the synthetic-to-real gap, but how matters. Without any VFM teacher, simply applying stage-one EMA self-distillation to the on-device model already reduces E50U50 from 3.48° to 2.20° (-36.8%), while the teacher-free UDA method DARE-GRAM reaches only 2.96° — a consistency constraint from self-distillation suits this task better than explicit domain alignment.
Tail metrics behave differently from the median. The paper observes explicitly that tail samples in crowd-sourced datasets are more susceptible to label noise (ambiguous gaze targets, inconsistent collection conditions), and consequently the pseudo-labels-only variant actually achieves the best E90U90 (8.19°), while versions with additional distillation losses lose slightly at the tail; Community KD is best at E75U75 (3.24°). The authors handle this honestly, claiming no blanket superiority and instead arguing that with only a strong teacher available, regressing directly onto teacher predictions is already a robust objective.
The cross-architecture ablation overturns an intuitive hypothesis. The authors expected ConvNeXt to suit eye tracking better than ViT, since gaze features concentrate near the pupil and ViT patch resolution could be a bottleneck; the measurements say the opposite, with ViT-B better at linear probing, synthetic fine-tuning, and self-distillation alike, though the advantage narrows as adaptation deepens. Their explanation is that peripheral regions within each patch (eyelid, iris boundary) provide enough contextual cues to offset the tokenization resolution loss.
Accuracy approaches the upper bound, but the tail still lags. The optimized VFM and the DistillGaze student produce nearly overlapping curves across user percentiles at E50 and E75 (evidence of complete knowledge transfer), degrading only modestly at E90; both come close to the fully supervised on-device model trained on real labels (0.82°/5.91°), but a roughly 6° tail gap remains real.

Highlights & Insights¶

Redefining "on-device deployment" as escaping the annotation cycle, not merely shrinking the model. The whole argument is that a new device's adaptation time is bottlenecked by crowd-sourced annotation (weeks to months), and synthetic labels plus passively collected unlabeled real images compress that from months to days — which gives the 256K figure a clear engineering narrative rather than compression for its own sake.
A t-SNE analysis that turns "why the VFM fails" into an actionable spatial diagnosis. Rather than vaguely asserting domain mismatch, it shows embeddings cluster by identity with gaze suppressed, yet exhibit smooth gaze-correlated intra-cluster gradients — a finding that directly determines that the fix is in-domain reshaping rather than a stronger backbone. This "first prove the signal exists, then prove it is wrongly organized" pattern transfers directly to other foundation-model adaptations onto proprietary modalities.
Strong augmentation models device failure modes, not generic image perturbations. MTF filtering, motion blur, compression artifacts, glint inpainting, and coarse dropout amount to enumerating the physical differences between synthetic rendering and real infrared imaging and injecting them as augmentation. Any sim-to-real on-device task (depth, hands, face) can follow the same recipe: enumerate your own sensor's failure modes instead of copying ImageNet augmentation policies.
The EMA student acts as a second teacher. The teacher is large with hard targets, while the EMA student is isomorphic to the student with smooth momentum targets; the two are complementary and switch dominance on a cosine schedule, avoiding being dragged by an equal-capacity copy early on and misled by hard targets late. This is a cheap component to lift into any large-teacher/on-device-student setting.
The "teacher exists only for distillation" trade is well computed. Spending 336× the parameters on a teacher that never ships buys a 58.6% reduction in median error with zero size increase at the edge — an excellent exchange rate wherever compute is cheap and on-device budget is expensive.

Limitations & Future Work¶

The limitation the authors acknowledge most directly is the remaining gap to the fully supervised upper bound, concentrated in the tail: difficult cases such as atypical eye appearance or extreme gaze angles remain hard without labeled supervision, and the tail from the optimized VFM (1.33°) to full supervision (0.82°) is not yet closed.
The directions they propose include stronger generative modeling to bridge the real-synthetic domain gap, stronger self-supervised objectives, and lightweight online adaptation at inference time.
All training and evaluation use a single device configuration (Project Aria), so "rapid adaptation to new hardware" is currently a persuasive argument rather than a claim validated by cross-device experiments; extension to multiple devices with varying camera geometries, illumination, and sensor characteristics is left as future work. This is the most visible tension between the paper's narrative and its experimental coverage.
On-device constraints are argued only via parameter count. The paper repeatedly states that 256K is "suitable for real-time on-device deployment," but the cached full text reports no measured on-device latency, memory footprint, or power, nor an on-device comparison against the synthetic-supervision baseline — and since the FBNet backbone is itself a hardware-aware NAS product, more convincing numbers could plausibly have been included. ⚠️ measurable on-device data may exist in the supplementary material; refer to the original paper.
From a methods standpoint, unlabeled real images are described as passively collectible, but in a shipping product they still require a user actually wearing the device; gathering enough unlabeled real images on a not-yet-launched device still depends on early user volume, which the paper does not discuss in terms of what it means for the "rapidly" timeline.
Concrete technical improvement paths: since the tail is dominated by label noise and hard examples, one could confidence-weight pseudo-labels during distillation instead of regressing on all of them uniformly, or apply difficulty-aware reweighting per subject; and since both \(\lambda_{\text{SynSup}}\) and \(\lambda_t\) currently follow fixed cosine schedules, validation-error-driven adaptive scheduling might remove one round of tuning.

vs DINOv3 linear probe / synthetic fine-tuning: Both apply an off-the-shelf VFM to eye tracking directly. Linear probing shows frozen features under a linear map are simply insufficient (5.47°, worse than a 256K small model); synthetic fine-tuning already reaches 2.01° but still treats the VFM as an ordinary backbone and uses no unlabeled real data. This paper instead places the VFM in a self-distillation framework for in-domain reshaping, at no extra inference cost since the teacher never deploys, reaching 1.33°.
vs DARE-GRAM: A state-of-the-art unsupervised domain adaptation method for regression that explicitly aligns second-order statistics of source and target representations. Measured here, it loses to plain self-distillation at both the 86M teacher scale (4.84°) and the 256K on-device scale (2.96°), which the authors attribute to overfitting the synthetic distribution and thereby amplifying the domain gap. The lesson: when the synthetic data already carries a strong prior, "aligning" serves worse than "letting the model stay consistent on real images."
vs Community KD: The direct foundation of this work, which performs bidirectional knowledge transfer by co-distilling multiple students under a pretrained teacher. The two modifications here are distilling in feature space and prediction space simultaneously and replacing co-trained peers with a momentum-updated EMA student (borrowed from BYOL/DINO). The ablation shows the two are close (1.44° vs 1.48°); the benefit of this paper's design shows up more in robustness when no strong teacher is available, verified with a ConvNeXt-S teacher in the supplementary material.
vs VIC-KD / SP: VIC-KD ports VICReg's variance-invariance-covariance regularization to distillation, while SP aligns similarity structure across samples. This paper reuses VIC-KD directly as its feature-space distillation loss; in the on-device comparison it is the best of the pure-distillation methods at E50U50 (1.46°) but still slightly behind the full version that also uses two-source pseudo-labels (1.44°).

Rating¶

Novelty: ⭐⭐⭐⭐ It chains foundation-model in-domain reshaping and on-device distillation into a complete "unlabeled real images are all you need to adapt to new hardware" pipeline with an actionable t-SNE diagnosis; individual components are mostly combinations and modifications of existing methods, but the combination and the problem framing are fresh.
Experimental Thoroughness: ⭐⭐⭐⭐ Large-scale evaluation over 2,222 crowd-sourced participants with 95% confidence intervals and ablations spanning teacher side, on-device side, loss functions, and backbone architectures, plus honest reporting that the method is not uniformly best at the tail; marked down because on-device latency/power are not measured and cross-device adaptation is argued rather than tested.
Writing Quality: ⭐⭐⭐⭐ The motivational chain (annotation-cycle pain → synthetic data → VFM failure → t-SNE diagnosis → two-stage solution) reads smoothly, and the fact that "teacher/student" means different things in the two stages is clearly explained; the weaknesses are formulas that are difficult to verify from the typeset text and a thin quantitative treatment of on-device constraints.
Value: ⭐⭐⭐⭐⭐ Directly useful for anyone deploying a foundation model onto a proprietary sensor modality at the edge: the teacher never ships, and the combination of synthetic labels with passively collected unlabeled real images compresses new-hardware adaptation from months to days with zero inference overhead.