Skip to content

Wid3R: Wide Field-of-View 3D Reconstruction via Camera Model Conditioning

Conference: ECCV 2026
Paper: ECCV 2026 / Project Page
Area: 3D Vision
Keywords: wide field-of-view 3D reconstruction, feed-forward reconstruction, spherical harmonics ray representation, camera model conditioning, fisheye and panoramic cameras

TL;DR

Wid3R replaces the point-map output of multi-view feed-forward reconstruction with a decoupled "spherical-harmonics ray direction + radial distance" representation and conditions a fully shared backbone on a learnable camera model token, so a single network predicts dense point maps and camera poses from distorted fisheye and 360° images without calibration or undistortion, raising AUC@30° from π3's 40.94 to 74.61 on Zip-NeRF (fisheye) and from a previous best of 2.60 to 79.93 on Stanford2D3D (360°).

Background & Motivation

Multi-view feed-forward geometry models have advanced quickly over the past two years: DUSt3R recasts reconstruction as "an image pair in, a point map out," Spann3R adds an external spatial memory for incremental reconstruction, VGGT recovers scale and robustness through large-scale data and a global world reference frame, and π3 removes the variance caused by reference-view selection with a permutation-equivariant architecture. All of them, however, share a premise that is baked into both their architecture and their training data — the input images come from pinhole cameras, or have already been calibrated, undistorted and rectified. This assumption is not merely part of the experimental setup: the output head regresses 3D points or depth per pixel, so the pixel-to-3D-point mapping is implicitly assumed to be a perspective projection, and the training sets are uniformly perspective imagery. The consequence is systematic. Feed a fisheye image into VGGT and the network can only treat it as a "strange pinhole image" — predictions near the image center are barely usable, and geometric error grows the closer a pixel lies to the field-of-view boundary. A 360° image is more extreme still: not a single one of its pixels falls inside a pinhole viewing cone.

Real-world cameras, by contrast, deviate from the pinhole assumption all the time. Robotics, surround-view perception for automated driving, AR/VR, and large-scale mapping systems routinely use fisheye and spherical cameras precisely to maximize coverage and minimize blind spots under a limited number of viewpoints. To reuse existing feed-forward models, the engineering workaround is to calibrate, undistort, and then treat the result as a perspective image — a pipeline that adds stages and, worse, discards information during resampling, since pixels near the original field-of-view boundary are compressed and the distortion-model estimate is itself error-prone there. Learning-based wide field-of-view alternatives have not closed the gap either. DAC projects different camera models into a common equirectangular space and UniK3D performs monocular geometry estimation in ray space with spherical harmonics, but both address only single-view depth or point maps: neither produces camera poses, and neither does joint multi-view reconstruction. The classical route to multi-view 360° reconstruction (EDM dense matching plus IM360 plus SfM and bundle adjustment) is accurate enough in practice but relies on all-pairs dense matching, costing minutes to half an hour per scene.

This paper's angle is to move distortion out of the representation entirely: if a pixel is described from the start as a 3D ray direction plus a distance along that ray, then the projection model only affects the angular half, and the scale half is clean and camera-model agnostic. Core idea: replace the point-map head with "spherical-harmonics rays (an Angular Module) plus radial distance and uncertainty (a Radial Module)," encode the camera model as a learnable token injected as an explicit condition into a fully shared network, and pair this with camera-type-mixed training data and pinhole-to-fisheye augmentation, so that one feed-forward model covers pinhole, fisheye and 360° cameras at inference time without calibration or undistortion.

Method

Overall Architecture

Wid3R takes N possibly distorted wide field-of-view images plus a camera model token for each (pinhole, fisheye, or 360°), and outputs a dense point map per image in its own camera coordinate frame, the camera poses, and a globally aligned point cloud under a single scale. The whole chain is one feed-forward pass: images go through a DINO backbone and a feature aggregation module (both of which, together with the pose header, are initialized from π3 pretrained weights); the aggregated per-pixel features are no longer regressed directly into a point map but split into two branches — the Angular Module predicts a set of spherical-harmonics coefficients conditioned on the camera token, from which a 3D ray direction is expanded for every pixel, while the Radial Module predicts a radial distance and its uncertainty along that ray. Element-wise multiplication of the two yields the local point map (\(\hat{\mathbf{P}}_i=\hat{\mathbf{R}}_i\odot\hat{\mathbf{D}}_i\)); the pose header supplies poses, and a single scale factor shared across all views finally assembles everything into one world coordinate frame.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["multi-view wide FoV images<br/>pinhole / fisheye / 360°"] --> B["DINO encoding + feature aggregation<br/>π3 pretrained initialization"]
    B --> C["camera model token<br/>select and inject camera prior"]
    C --> D["ray decoupled representation<br/>angular module → SH rays R<br/>radial module → distance D, uncertainty U"]
    D --> E["pose header → poses T<br/>point map P = R ⊙ D"]
    E --> F["single-scale alignment → global point cloud<br/>localization / reconstruction"]

Two things separate this from existing feed-forward models: how the output is parameterized (the point-map head becomes an angular/radial pair, decoupling direction from scale) and how the model is conditioned ("which camera is this" turns from something the network must infer implicitly into an explicit input alongside the images).

Key Designs

1. Camera model token: turning "which camera is this" from an implicit inference into an explicit input

Existing feed-forward models have no channel through which to learn the imaging geometry of a distorted image; all they can rely on are image statistics, yet near the field-of-view boundary the same pixel displacement corresponds to wildly different 3D directions under different projection models, so the network can only compromise. Wid3R writes a camera model token \(\mathbf{C}_i\) into the problem definition as an input alongside the image \(\mathbf{I}_i\), so the network explicitly knows which camera family it is looking at while predicting geometry.

Concretely, the token is a learnable embedding (one per pinhole/fisheye/360°). It does not participate in backbone feature extraction; instead it conditions the Angular Module's spherical-harmonics coefficient prediction. Every other network parameter is fully shared across the three camera types. The sharing is deliberate: 360° geometry-labeled samples account for less than 1% of all training images, and a dedicated parameter set for them would have far too little supervision to stand on. A shared backbone differing only by a token lets 360° "borrow" the statistics of the far more abundant pinhole and fisheye data. The ablation supports this (Table 7): on the same mixed-camera data, removing the token degrades Stanford2D3D mean Acc from 0.248 to 0.284 and N.C. from 0.706 to 0.680.

The token only requires the network to know the camera family; it requires no calibration parameters at all (no focal length, principal point, or distortion coefficients), which makes it far lighter than explicit camera models (Kannala–Brandt, Double Sphere, Mei) and sidesteps the large and unstable errors of calibrating a heavily parameterized model. The price is that it can only distinguish camera families seen during training: an unseen projection model cannot be expressed precisely by interpolating tokens.

2. Ray decoupled representation: spherical-harmonics ray directions plus radial distances replacing the point-map head

A point-map head regresses "pixel to 3D point" as one entity. That mapping is roughly linear under a pinhole model, but highly nonlinear under fisheye and 360°, and ill-conditioned near the field-of-view boundary — the closer to the pole, the faster the direction changes per pixel — so the network ends up spending capacity fitting the projection itself rather than the scene. A subtler problem lies in the supervision: the usual scale-alignment loss weights residuals by perpendicular depth, which is not a well-defined quantity under a wide field of view.

This paper instead uses a per-pixel ray representation whose directions come from the Angular Module. Rather than regressing direction vectors directly, the module predicts spherical-harmonics coefficients \(\mathbf{k}^{lm}_i(\mathbf{C}_i)\) conditioned on the camera token and expands them with a fixed spherical-harmonics basis \(Y^{lm}(\cdot,\cdot)\):

\[\hat{\mathbf{R}}_i(\theta,\phi)=\sum_{l=0}^{L}\sum_{m=-l}^{l}\mathbf{k}^{lm}_i(\mathbf{C}_i)\,Y^{lm}(\theta,\phi)\]

where \(l\) and \(m\) are the degree and order of the harmonics, and \(\theta,\phi\) are the polar and azimuthal angles in spherical coordinates. Because the basis is smooth and compact on the unit sphere, the pixel-to-direction mapping of any central camera can be approximated well with finitely many coefficients, so one head naturally covers pinhole, fisheye and 360° cameras. And because \((\theta,\phi)\) is in bijection with 3D directions, directions "behind" a 360° camera (where \(\theta\) crosses into the back half-space) can also be represented — something no representation parameterized on the image plane can do. The Radial Module then predicts just one scalar per ray (the radial distance) plus its uncertainty, so scale information is isolated entirely in this branch.

The decoupling pays off directly: distortion contaminates only the angular branch, which has the strong spherical-harmonics prior behind it, while scale flows only through the radial branch, which is why training can use a uniform radial-distance weighting (see the loss section). And since no explicit camera model is involved, inference performs neither calibration nor undistortion, avoiding the information loss that resampling introduces.

3. Wide field-of-view data recipe and geometry-consistent augmentation

There are two pain points here. The first is extreme supervision imbalance: 360° images make up less than 1% of the training set, and 360° data is inherently sparse along a trajectory (each viewpoint already sees the entire surroundings), so consecutive frames may overlap very little and multi-view correspondence is harder to learn than from pinhole sequences. The second is that overlap is itself a prerequisite for feed-forward multi-view learning, yet camera positions in wide field-of-view sequences are often very unevenly distributed, so drawing a handful of images into one batch at random can easily produce a nearly non-overlapping set, degrading the training signal.

On the data side, nine datasets (TartanAirV2, ASE, Hypersim, KITTI-360, 360Loc, Matterport3D, ScanNet++, EDEN, VKITTI — five synthetic and four real) are organized into pinhole / fisheye / 360° groups and trained jointly, so that camera-family diversity compensates for the small absolute number of 360° samples. On the sampling side, pairwise camera-position distances form a distance matrix, and a softmax over the negative distances yields a probability matrix, so views that are spatially closer — and therefore more likely to overlap — are sampled into the same batch with higher probability. On the augmentation side, the most important piece is camera augmentation: a pinhole image and its depth are unprojected into a point cloud and reprojected onto a randomly sampled distorted camera model (with softmax splatting to preserve detail), effectively manufacturing fisheye supervision out of abundant pinhole data. For ERP (equirectangular) images, the fact that a single ERP image covers the whole viewing sphere is exploited: azimuth \(\in[0,2\pi]\) and elevation \(\in[0,\pi]\) are sampled at random, and image, depth and pose are rotated accordingly, with the same transformation applied to all views of a scene within a batch to keep geometry consistent.

The three designs are complementary: the token provides the interface for "knowing which camera this is," the ray representation provides the capacity to "represent any camera," and the data recipe provides supervision the model can actually learn from. The ablation shows this complementarity most clearly in stability — trained on 360° data alone, mean Acc is 0.220 with a standard deviation of 0.187; adding pinhole data cuts the deviation to 0.099, and adding pinhole plus fisheye brings it down to 0.089 (Table 7a).

A Worked Example

Take the fisheye setting of Fig. 1: 20 fisheye images of one scene (all resized to 518×336), each accompanied by a "fisheye" token. The DINO backbone extracts per-pixel features and the aggregation module exchanges information across the N views once (this step, like the pose header, is already π3-pretrained). The Angular Module then outputs a set of spherical-harmonics coefficients per pixel under the fisheye token and expands them into that pixel's 3D ray direction — these directions need not be symmetric about the image center, so even heavily stretched pixels at the fisheye boundary receive the correct pointing; the Radial Module regresses one radial distance and uncertainty per ray, and the product of the two gives the point map in that view's local coordinate frame. The pose header simultaneously emits 20 poses, and a single globally shared scale aligns them into one world frame. With a 360° camera the flow is identical — only the token changes to "360°" — and the same weights handle images spanning the entire viewing sphere. With the 18 360° images of Fig. 1 the feed-forward pass still runs once, and for a large scene such as Matterport3D the whole chain takes about 3 seconds (Table 4).

Loss & Training

All supervision is defined in a "local point map plus one unknown scale" framework. Because each point map lives in its own camera frame, a scale ambiguity exists across views; Wid3R follows π3 in assuming a single unknown but consistent scale \(s\) shared by all views and solves the MoGe-style optimal alignment:

\[s^{*}=\arg\min_{s}\sum_{i=1}^{N}\sum_{j=1}^{HW}\frac{1}{\mathbf{D}_{i,j}}\left\|s\hat{\mathbf{P}}_{i,j}-\mathbf{P}_{i,j}\right\|_1\]

The weight is the radial distance \(\mathbf{D}_{i,j}\) along the ray rather than perpendicular depth — a necessary substitution for arbitrary camera models, since perpendicular depth is undefined under a wide field of view. The point map loss is then the weighted L1 with \(s^*\).

The remaining terms: the normal loss constrains the angle between predicted and ground-truth normals computed by an 8-neighbor cross-product convention; the pose loss is defined on the relative pose \(\mathbf{T}_i^{-1}\mathbf{T}_j\), with the rotation term using the geodesic distance (through the trace of the rotation-matrix product) and the translation term using a Huber loss to suppress outliers, scaled by the same \(s^*\) so that translation and point map live at one scale; the ray loss follows UniK3D's asymmetric angular loss to compensate for the scarcity of wide field-of-view training images, weighting the \(\theta\) and \(\phi\) angular components separately and asymmetrically in the sign of the error (the paper gives \(\beta=0.75\); that equation is corrupted by OCR in the cached text, so the exact form ⚠️ refer to the original paper); the radial loss is an L1 on radial distance; and the uncertainty loss makes the predicted uncertainty fit the radial error itself, i.e. teaches the network to anticipate where it will be wrong.

The total loss is a weighted sum with \(\lambda_{normal}=10\), \(\lambda_{pose}=0.1\), \(\lambda_{ray}=1.0\), \(\lambda_{rad}=1.0\), and \(\lambda_{uncer}=0.1\). Optimization uses Adam with an initial learning rate of \(5\times10^{-5}\) and exponential decay per iteration, with inputs resized to 518×336; the encoder, feature aggregation and pose header are initialized from π3 pretrained weights while the remaining components train from scratch, end to end. GPU memory caps the maximum frames per batch — 12 on an RTX A6000 and up to 24 on an H100 — and this is not a trivial engineering detail: the ablation shows that raising frames per batch from 12 to 24 lowers mean Acc from 0.248 to 0.197 and raises N.C. from 0.706 to 0.729 (Table 7c).

Key Experimental Results

Main Results

All evaluations are zero-shot: models are trained only on the composed wide-angle data and evaluated directly on FIORD (fisheye), Zip-NeRF (fisheye) and Stanford2D3D (360°) without any fine-tuning. "Ours (π3)" is a control trained on the same data but keeping the original π3 point-map head, which separates the contribution of the data from that of the ray representation and tokens.

Table 2, zero-shot pose estimation (RRA@30 / RTA@30 / AUC@30, higher is better):

Method FIORD RRA@30↑ FIORD RTA@30↑ FIORD AUC@30↑ Zip-NeRF RRA@30↑ Zip-NeRF RTA@30↑ Zip-NeRF AUC@30↑ S2D3D RRA@30↑ S2D3D RTA@30↑ S2D3D AUC@30↑
VGGT 82.34 84.01 44.63 64.42 65.29 20.68 19.30 33.14 2.60
π3 85.48 87.42 48.35 82.65 86.66 40.94 19.94 31.99 2.06
Ours (π3 baseline) 99.87 96.39 72.56 83.04 89.48 64.44 82.91 87.83 62.14
Ours (Wid3R) 100.0 93.38 68.20 93.34 95.15 74.61 94.05 93.29 79.93

Table 3 cross-checks this with absolute and relative errors (lower is better): on Stanford2D3D, Wid3R cuts π3's ATE from 2.83 to 1.11 and RPE rot from 91.13 to 18.99; on Zip-NeRF, ATE drops from 1.83 (VGGT) to 0.49 and RPE rot from 34.41 to 7.72; on FIORD, ATE goes from 0.53 (π3) to 0.44 and RPE rot from 13.45 to 3.27.

Table 5, point map estimation (Acc./Comp./N.C.; Acc. and Comp. lower is better, N.C. higher is better; means shown, see the paper for medians):

Method ScanNet++ Acc.↓ ScanNet++ Comp.↓ ScanNet++ N.C.↑ Matterport3D Acc.↓ Matterport3D Comp.↓ Matterport3D N.C.↑ Stanford2D3D Acc.↓ Stanford2D3D Comp.↓ Stanford2D3D N.C.↑
VGGT 0.135 0.068 0.704 0.327 1.756 0.530 0.387 2.115 0.536
π3 0.086 0.037 0.739 0.315 1.308 0.539 0.380 0.745 0.557
Ours (π3 baseline) 0.027 0.014 0.781 0.161 0.142 0.610 0.211 0.273 0.596
Ours (Wid3R) 0.018 0.012 0.803 0.094 0.087 0.790 0.197 0.172 0.729

Ablation Study

Table 7 ablates point map estimation on the Stanford2D3D 360° setting (this dataset itself is not in the training data), separating the effects of the camera token, the training data composition, and the number of frames per batch:

Study Frames per batch Camera token Pinhole Fisheye Sphere Acc. mean↓ Acc. std↓ Comp. mean↓ Comp. std↓ N.C. mean↑
(a) 2~12 ✓ ✗ ✗ ✓ 0.220 0.187 0.355 0.416 0.710
(a) 2~12 ✓ ✓ ✗ ✓ 0.254 0.099 0.166 0.243 0.698
(a) 2~12 ✓ ✗ ✓ ✓ 0.593 0.101 0.120 0.136 0.686
(a) 2~12 ✓ ✓ ✓ ✓ 0.248 0.089 0.146 0.164 0.706
(b) 2~12 ✗ ✓ ✓ ✓ 0.284 0.103 0.152 0.178 0.680
(c) 2~24 ✓ ✓ ✓ ✓ 0.197 0.113 0.172 0.257 0.729

Key Findings

  • For 360°, the data composition mainly buys stability and completeness, not just mean accuracy. Trained on 360° alone, mean Acc is 0.220 with a standard deviation of 0.187 and mean Comp 0.355; adding pinhole data drops the deviation to 0.099, and adding fisheye on top brings it to 0.089 with mean Comp down to 0.146. Mixed-camera training therefore works by letting shared parameters draw extra supervision from other camera types, mitigating the overfitting caused by scarce 360° supervision.
  • Fisheye data helps 360° the most and is also the least stable. In the 360° + fisheye row, mean Comp falls to the lowest value in the whole table (0.120) and the median Acc of 0.126 is among the best, yet mean Acc rises to 0.593 (std 0.101), meaning a few scenes produce very large outlier errors — mean and median diverge sharply on this row, so the table cannot be read through means alone.
  • The camera token helps consistently, but modestly. Full data with the token (0.248 / 0.146 / 0.706) versus without it (0.284 / 0.152 / 0.680): all three metrics move the right way, consistent with the token providing only a camera-family prior while the real knowledge lives in the shared backbone.
  • Frames per batch is a performance curve capped by hardware. Going from 12 to 24 frames lowers mean Acc from 0.248 to 0.197 and raises N.C. from 0.706 to 0.729, but mean Comp actually worsens from 0.146 to 0.172 and its standard deviation grows from 0.164 to 0.257. The authors' explanation is that more views let the model learn richer multi-view attention patterns, which also implies the method has not yet saturated on consumer-grade GPU memory.
  • Large-scale localization: registration rate on par with SfM, mixed accuracy, two orders of magnitude faster. On three Matterport3D scenes (Table 4), Wid3R registers every image in all three (37/37, 20/20, 31/31), matching EDM + IM360; AUC@10 is clearly better on Scene2 (84.77 versus 66.87) but slightly lower on Scene1 (72.81 versus 84.53) and Scene3 (89.40 versus 92.18). The decisive difference is runtime: roughly 3 seconds for the whole chain versus about 3–30 minutes for the SfM pipelines, because it needs neither all-pairs dense matching nor iterative optimization.
  • Monocular 360° depth: strong distribution metrics, unremarkable absolute error. On Matterport3D (Table 6), Wid3R reaches δ1.25 = 0.948, the highest in the table (Depth Anywhere 0.917, ACDNet 0.900), ties Depth Anywhere on δ1.25² at 0.976, but trails on δ1.25³ (0.984 versus 0.991); its AbsRel of 0.227 is worse than 360° depth specialists such as Depth Anywhere (0.085) and DAC (0.156). Given that Wid3R is a multi-view model trained on a mix of pinhole, fisheye and 360° data, the result shows it learns usable 360° depth structure without being the monocular depth state of the art.

Highlights & Insights

  • The ray representation turns "camera model" from something the network fits into part of the representation. Because pixel-to-direction is expanded over a fixed spherical-harmonics basis with only the coefficients predicted, the network learns a low-dimensional correction to "roughly what this camera looks like" instead of fitting a nonlinear mapping from scratch; and since \((\theta,\phi)\) is in bijection with 3D directions, the back half-space of a 360° camera is representable at all — something a point-map head parameterized on the image plane simply cannot do.
  • Decoupling angle from radius confines distortion and scale to separate branches. The elegance of the split is that it solves two problems at once: distortion (which only affects direction) gets the spherical-harmonics prior as a backstop, while scale (which only affects radius) can be supervised with a camera-agnostic radial-distance weighting. Replacing perpendicular depth with radial distance in the scale-alignment loss is a tiny but essential change that any wide field-of-view geometric supervision can reuse directly.
  • Camera tokens plus camera augmentation convert a scarce-supervision problem into a data-sharing problem. With 360° labels under 1% of the corpus, brute force is hopeless; the paper instead shares weights across all three camera types, distinguishes them by token, and manufactures wide field-of-view supervision by reprojecting pinhole data into fisheye models. The ablation shows the biggest gain is in variance rather than the mean, which is a useful lesson for any multi-task training setup where one data type is extremely rare.
  • The efficiency advantage of feed-forward geometry over SfM is amplified in the wide field-of-view regime. 360° SfM must do all-pairs dense matching (EDM) plus bundle adjustment — accurate but minute-scale — whereas Wid3R compresses the same task to seconds without losing registration rate, showing that "replace iterative optimization with a large model" works in the wide field-of-view setting too.

Limitations & Future Work

  • The authors acknowledge that the method does not explicitly model dynamic scene elements and plan to add datasets with dynamic content and dedicated modules for scene dynamics. This is a very real limitation for in-the-wild 360° capture (pedestrians, vehicles, reflective surfaces).
  • The token can only express camera families seen during training (pinhole / fisheye / 360°); an unseen projection model (a particular non-central camera, an extreme distortion model) cannot be expressed precisely by interpolation. Inference also requires knowing which camera family an image came from, even though full calibration is not needed. A natural improvement is to replace the discrete token with a continuous condition on camera parameters (focal-length ratio, distortion coefficients), so the model can interpolate within a camera family as well.
  • 360° data is under 1% of training images. Mixed-camera training mitigates this, but the ablation also shows that the fisheye + 360° combination raises mean Acc from 0.220 to 0.593 (despite the best median), so unstable outlier failures remain in the few-shot regime and stability is not fully solved.
  • On monocular 360° depth the AbsRel of 0.227 is clearly behind specialized methods (Depth Anywhere at 0.085), and on large-scale localization Scene1 and Scene3 are below EDM + IM360, so the representation does not dominate every benchmark: its strengths are generality, zero-shot transfer and speed rather than raw accuracy on every axis.
  • The method rests on π3 pretrained weights (encoder, feature aggregation, pose header) and therefore still depends on large-scale pinhole pretraining. The paper reports no fully-from-scratch training, and it gives neither the spherical-harmonics degree \(L\) nor a sensitivity analysis over it.
  • vs DUSt3R / VGGT / π3: they make multi-view geometry a feed-forward regression task, but assume pinhole or already rectified inputs, output point maps, and align scale through a single shared factor. Wid3R keeps π3's permutation-equivariant backbone (and thus its insensitivity to reference-view choice) and its scale-alignment idea, but swaps the output head for spherical-harmonics rays plus radial distance and adds camera tokens. The difference lies in the representation and the conditioning rather than the backbone, so Wid3R's changes can be read as a general "output layer + input condition" upgrade for this family of feed-forward models.
  • vs UniK3D / DAC: UniK3D also works in ray space with spherical harmonics, but it performs monocular point estimation only, with no poses and no joint multi-view reconstruction; Wid3R extends it to multi-view reconstruction and adds the camera token and the radial uncertainty branch. DAC takes the other route, projecting various camera models into a common equirectangular space before estimating depth, i.e. a unified parameter domain rather than a unified ray space. The task settings differ (multi-view versus single-view), so absolute numbers are not directly comparable.
  • vs IM360 + EDM (the 360° SfM pipeline): they obtain accurate poses and sparse/dense point clouds via dense matching, triangulation and bundle adjustment, at the cost of all-pairs matching and iterative optimization (minute-scale). Wid3R predicts point maps in a single feed-forward pass, finishing in seconds with an equal registration rate, better AUC on some scenes and slightly lower on others. The trade-off is the classic one of accuracy ceiling versus throughput and deployment cost.
  • vs MoGe / Metric3D (monocular point maps with canonical cameras): they handle camera geometry through a canonical camera transformation and optimal alignment, but their home turf is monocular metric depth on perspective images. Wid3R's contribution is to upgrade "camera geometry" from a canonical camera transform to an explicit camera model condition plus a ray representation, placed in the setting of joint multi-view pose and point map estimation.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ First multi-view feed-forward method for distorted wide field-of-view imagery including 360°; the combination of a ray-decoupled representation and camera model tokens is clean and previously unexplored.
  • Experimental Thoroughness: ⭐⭐⭐⭐ Four lines of evaluation (pose, point map, large-scale localization, monocular depth) plus ablations over data composition, tokens and frame count; but there is no finer-grained ablation of the spherical-harmonics degree or token design, and several ablation rows have means and medians that diverge sharply.
  • Writing Quality: ⭐⭐⭐⭐ The motivation chain is clear (the pinhole bias of feed-forward models → the need for wide field-of-view → ray representation) and the method sections are well organized; some equations are degraded in the public PDF, and reproduction details (harmonic degree, token dimension) require the supplementary material.
  • Value: ⭐⭐⭐⭐⭐ Directly widens the applicability of feed-forward geometry models to fisheye and panoramic imagery, and requires neither calibration nor undistortion at inference — attractive for real deployments in robotics, AR/VR and large-scale mapping.