MAC-Splat: Multi-Attribute Consistency for High-Fidelity Sparse-View Reconstruction¶

Conference: ECCV2026
Paper: ECCV Paper
Area: 3D Vision
Keywords: Gaussian Splatting, sparse-view reconstruction, cross-view matching, semantic guidance, multi-attribute consistency

TL;DR¶

MAC-Splat connects 3D Gaussians predicted from different views using semantically enriched 2D matches and jointly regularizes position, shape, and appearance, reaching 21.34 dB PSNR on the ScanNet++ Very Wide split, 5.18 dB above Splatt3R.

Background & Motivation¶

Generalizable Gaussian Splatting aims to predict a scene representation directly from a few images, avoiding expensive optimization for each new scene. However, predicting Gaussians from sparse images does not establish the correct 3D surface: many combinations of position, scale, and opacity can produce similar training images. With large viewpoint changes, occlusion and view-dependent appearance further weaken correspondences, so low training-image error can conceal floaters, duplicate surfaces, and distorted thin structures. PixelSplat and MVSplat incorporate multi-view geometry, while Splatt3R and NoPoSplat reduce dependence on input poses, but supervision of rendered images alone still struggles to eliminate these 3D ambiguities.

Adding depth, normals, or foundation-model features can improve reconstruction without directly answering a crucial question: do Gaussians representing the same surface location in two images have compatible 3D attributes? Making both outputs look plausible can still allow them to form inconsistent local explanations. Directly enforcing agreement between matched Gaussians also carries a risk, because an incorrect match can pull primitives on different objects together. The paper therefore addresses both ends of supervision: improve the reliability of the links between primitives, then use those links to constrain 3D parameters rather than only projected colors.

The authors combine MASt3R geometric descriptors with frozen DINOv3 semantic features to establish cross-view anchors from trustworthy correspondences. Semantic information neither generates missing images nor supplies object-category labels here; it helps descriptors distinguish locations that look similar but belong to different structures. Consistency supervision then checks not only whether centers coincide, but also whether principal-axis scales, opacity, and color representations agree. Core Idea: turn semantically enriched 2D correspondences into direct training signals on multiple attributes of 3D Gaussians, allowing different views to correct their explanations of the same local surface.

Method¶

Overall Architecture¶

The inputs are sparse scene images; the method establishes image-pair correspondences and predicts a Gaussian representation for novel view synthesis with a feed-forward network. MASt3R supplies the geometric backbone and cross-view interaction, while frozen DINOv3 provides dense semantic features for each image. Residual Semantic Fusion modifies only the final decoder-layer descriptors; Correspondence Filtering retains reciprocal nearest-neighbor pixel pairs with sufficiently high confidence. The Gaussian prediction head also produces parameter maps aligned with the descriptor grid, including 3D centers, covariances, opacity, and spherical harmonic coefficients. Each retained pixel pair selects a pair of Gaussians from these maps, which are placed in a common world frame before Multi-Attribute Consistency is evaluated. Meanwhile, target views rendered from the Gaussians retain image reconstruction and perceptual supervision, so 3D agreement supplements rather than replaces 2D fitting.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Sparse input images"] --> Features["MASt3R features<br/>Frozen DINOv3 features"]
    Features --> Fusion["Residual Semantic Fusion"]
    Fusion --> Matches["Correspondence Filtering"]
    Input --> Gaussians["Backbone and Gaussian head<br/>Predict parameter maps"]
    Matches --> Pair["Sample matched Gaussians<br/>Transform to world coordinates"]
    Gaussians --> Pair
    Pair --> MAC["Multi-Attribute Consistency"]
    Gaussians --> Render["Novel view rendering"]
    MAC -.-> Train["Joint training supervision"]
    Render --> Photo["Target-image supervision"]
    Photo -.-> Train

The MAC branch in the diagram is a training loss, not an iterative optimization loop required at test time. Training aligns Gaussians with ground-truth camera-to-world transformations; at test time MASt3R estimates input-view poses without ground-truth input poses or external SfM. Evaluation still uses ground-truth target cameras to specify the novel views to render, which does not contradict the absence of ground-truth input poses. The final output is a Gaussian scene representation that supports novel view rendering, not the 2D match list itself.

Key Designs¶

1. Residual Semantic Fusion: retain geometric descriptors while improving semantic distinctiveness

The original MASt3R descriptors support geometric matching, and replacing them outright with semantic features could disrupt pretrained local correspondence capabilities. The method first bilinearly resizes DINOv3 features to the decoder grid and projects them to the descriptor dimension with a linear layer. It then concatenates geometric and aligned semantic features and feeds them into a small MLP that predicts a residual added to the original descriptor. This MLP is shared across all spatial locations and operates only at the final decoder layer, rather than adding a semantic branch at every layer. Semantic information consequently acts as a correction to geometric features, preserving the backbone's representational foundation for matching.

Freezing the matching head does not make correspondences fixed: matching depends on distances between input descriptors, and residual fusion changes those distances. Likewise, a frozen confidence head can produce different scores when it receives new descriptors. DINOv3 itself remains frozen, and the authors explicitly identify the residual fusion module and Gaussian prediction head as trainable. The paper does not fully enumerate optimizer groups for all other backbone parameters, so it does not justify asserting that the entire MASt3R backbone is updated or frozen. The purpose of this branch is to improve anchors for downstream 3D supervision, not to add a separate semantic segmentation task.

2. Correspondence Filtering: connect 3D primitives only through trustworthy pixel pairs

The fused descriptor maps first produce reciprocal nearest neighbors under cosine distance. A location's best match in the other image must also select the original location as its best match; one-way similarity is insufficient to establish an anchor. This bidirectional condition removes some ambiguities in repetitive textures and occluded regions, but does not guarantee that every remaining match is correct. The frozen per-pixel confidence head then scores both endpoints, and their product becomes the match's joint confidence. Only matches above a confidence threshold survive; the same product subsequently serves as a soft weight in the MAC loss. Reliability control therefore combines hard filtering with continuous weighting among retained matches.

The 2D anchors share a grid with the Gaussian parameter maps, and differentiable bilinear grid sampling retrieves the corresponding Gaussian parameters. This step explicitly turns pixel correspondence into parameter correspondence, allowing 3D attribute discrepancies to affect the prediction branch through back-propagation. Differentiable sampling does not make discrete nearest-neighbor selection differentiable, and the paper does not claim otherwise. Each sampled image pair produces its own match set, while multiple pairs can provide overlapping constraints on the same Gaussians. If filtering leaves no matches, the MAC term becomes zero and training falls back to image supervision instead of forcing agreement on unreliable candidates. The authors describe robustness to threshold choice, but provide no threshold-sweep table in the main text, so this remains a methodological claim rather than comprehensive sensitivity evidence.

3. Multi-Attribute Consistency: constrain different 3D explanations of the same surface

Coordinate-induced differences must first be removed: centers are rotated and translated into the world frame, whereas covariance matrices transform only with rotation. The position term applies a Huber penalty to the L1 norm of the difference between world-space centers, and is evaluated only when both centers lie in front of their respective cameras. Huber reduces the influence of large residuals relative to a purely quadratic penalty, preventing a few anomalous correspondences from dominating updates. This supervision directly reduces the freedom to place the same local surface at different depths, targeting drift and duplicate surfaces. Coincident centers do not guarantee compatible Gaussian extents, motivating a separate shape term. The shape term compares log covariance eigenvalues; the following readable notation follows the Huber and L1 definitions in the paper's Eq. (5):

\[ \ell_{\mathrm{shape}}^{(k)} =\mathcal H\left(\left\|\log\boldsymbol{\lambda}^{1,k} -\log\boldsymbol{\lambda}^{2,k}\right\|_1\right). \]

Here \(\boldsymbol{\lambda}^{1,k}\) and \(\boldsymbol{\lambda}^{2,k}\) are eigenvalue vectors of the paired world-space covariances, not loss weights. Log differences compare relative principal-axis scales, so applying the same global scaling to both endpoints leaves the discrepancy unchanged. Using eigenvalues alone also makes this term rotation invariant, at the cost of not constraining principal-axis orientation. Common-scale invariance must be distinguished from constraining only anisotropy: two isotropic Gaussians with different scales still incur a penalty. The appearance term adds L1 penalties on differences in opacity and spherical harmonic coefficients, reducing incompatible color or transmission explanations even when centers and shapes agree. This is an appearance-parameter constraint, not a requirement that final pixel colors be identical across views. The main text does not fully specify directional alignment of spherical harmonic coefficients across coordinate frames, leaving that implementation detail to be checked separately. The three attribute terms are combined with internal weights, summed using joint match confidences, and divided by the sum of confidences plus a small constant preventing division by zero. This normalization prevents the number of valid matches alone from arbitrarily changing the loss magnitude, but weight reliability still depends on descriptors and confidence estimation.

Loss & Training¶

The 2D objective combines an L2 reconstruction term and LPIPS on rendered target images, while the 3D objective is the MAC loss above. The overall objective can be summarized from the prose as follows; the cached Eq. (9) has missing characters, so this is not presented as an exact transcription of its damaged typesetting:

\[ \mathcal L_{\mathrm{total}} =\mathcal L_{\mathrm{photo}}+0.25\,\mathcal L_{\mathrm{MAC}}. \]

Internal MAC weights for position, shape, and appearance are 1.0, 0.02, and 0.1, respectively, and remain fixed across experiments. Training uses ScanNet++, 4 NVIDIA RTX 4090 GPUs, and AdamW with a learning rate of \(10^{-5}\). An auxiliary MASt3R correspondence loss could be added, but its weight is zero in every reported result. The improvements therefore cannot be attributed to an additional explicit matching loss, but arise from fused features and 3D consistency together with rendering supervision. The authors explain resistance to degeneracy through an inverse-eigenvalue factor in the gradient of the log term for small eigenvalues. However, this effect requires a nonzero discrepancy; if both matched Gaussians shrink together while retaining identical eigenvalues, the shape loss can remain zero. It more precisely discourages inconsistent degeneration across views, rather than serving alone as a barrier guaranteeing that Gaussians never collapse.

Key Experimental Results¶

Main Results¶

The following selection comes from Table 1, page 10; PSNR is in dB, higher PSNR/SSIM is better, and lower LPIPS is better. Metrics are averaged over target views and test scenes in sRGB on full images by default, not within visibility masks. Coverage thresholds \((\phi,\psi)\) for Close, Medium, Wide, and Very Wide are \((0.9,0.9)\), \((0.7,0.7)\), \((0.5,0.5)\), and \((0.3,0.3)\), respectively; lower values indicate less overlap.

Split	Method	PSNR	SSIM	LPIPS
Close	Splatt3R	18.89	0.757	0.233
Close	PixelSplat	23.98	0.817	0.169
Close	MAC-Splat	22.76	0.810	0.146
Medium	Splatt3R	18.79	0.760	0.228
Medium	PixelSplat	20.31	0.783	0.227
Medium	MAC-Splat	22.60	0.816	0.135
Wide	Splatt3R	16.71	0.686	0.289
Wide	PixelSplat	18.46	0.770	0.235
Wide	MAC-Splat	22.09	0.818	0.142
Very Wide	Splatt3R	16.16	0.662	0.289
Very Wide	PixelSplat	15.36	0.673	0.329
Very Wide	MAC-Splat	21.34	0.818	0.158

Methods retain their official checkpoints and pose requirements, so this is not a controlled architectural comparison with matched training budgets and identical input-pose information. The paper renders MASt3R and DUSt3R as colored point clouds; their gap to Gaussian rendering should not be interpreted directly as a gap in geometric estimation ability.

Ablation Study¶

The following selection from Table 4, page 14, retains the Close and Very Wide difficulty extremes; other losses, training schedules, and hyperparameters remain fixed. Removing MAC retains standard image losses; removing DINOv3 retains MAC but removes the Residual Semantic Fusion branch.

Split	Config	PSNR	SSIM	LPIPS
Close	Without MAC	19.94	0.791	0.193
Close	Without DINOv3	20.57	0.749	0.237
Close	Full model	22.76	0.810	0.146
Very Wide	Without MAC	17.12	0.695	0.278
Very Wide	Without DINOv3	20.23	0.785	0.203
Very Wide	Full model	21.34	0.818	0.158

Table 2, page 11, also reports zero-shot transfer to DTU: MAC-Splat trains only on ScanNet++ without DTU fine-tuning, while baselines use their official pretrained models. All methods use official DTU camera intrinsics and a common input resolution, with metrics averaged over all target views.

Method	PSNR	SSIM	LPIPS
PixelSplat	13.33	0.517	0.329
MVSplat	14.03	0.522	0.334
Splatt3R	12.13	0.454	0.437
MAC-Splat	17.32	0.605	0.311

Key Findings¶

From Close to Very Wide, MAC-Splat loses only 1.42 dB PSNR, whereas PixelSplat loses 8.62 dB; PixelSplat nevertheless has higher Close PSNR and SSIM, so MAC-Splat is not best on every metric in every setting.
On Very Wide, removing MAC lowers PSNR by 4.22 dB relative to the full model, while removing DINOv3 lowers it by 1.11 dB; this supports the importance of 3D constraints for wide baselines, but the two ablations cannot isolate the three attribute terms.
In the observable-region evaluation of Table 3, page 13, Very Wide PSNR is 19.06/13.04 dB for MAC-Splat/Splatt3R, a 6.02 dB gap, indicating that gains are not confined to image completion in unobserved regions.
The visibility mask requires visibility in at least one context view and depth consistency; this must not be described as evaluation only on the intersection visible in both inputs.

Highlights & Insights¶

Moving supervision from rendered images to corresponding Gaussians directly penalizes 3D explanations that image losses struggle to distinguish; image supervision still anchors the overall representation to real appearance.
Semantic features determine where geometric constraints should be applied rather than simply adding a perceptual loss, explicitly linking foundation-model features to geometric failure modes.
Hard match gating, soft confidence weights, and Huber penalties for position and shape jointly handle unreliable anchors; setting MAC to zero for empty matches also defines a clear fallback behavior.

Limitations & Future Work¶

The authors propose unknown-pose and dynamic-scene extensions; although current inference estimates input poses, training still uses ground-truth poses, so it is not already fully pose-unsupervised learning.
The main text lacks separate ablations of position, shape, and appearance, as well as systematic measurements of matching accuracy, confidence calibration, latency, and threshold sensitivity.
Reader analysis: matching eigenvalues does not align covariance orientations, and common collapse may not trigger a penalty; orientation or minimum-scale constraints are possible extensions, not validated results of this paper.
Reader analysis: regions without overlap still rely heavily on image priors when trustworthy anchors are unavailable, and input-pose errors affect world-frame alignment; targeted pose-perturbation and empty-match tests are needed.
Several cached equations contain extraction artifacts, so this note uses only mechanisms supported by prose and readable expressions; spherical harmonic direction handling, the exact confidence threshold, and some training details still require implementation evidence.

vs Splatt3R: both use geometric foundation models to support Gaussian prediction from sparse inputs; MAC-Splat emphasizes explicit multi-attribute training constraints between corresponding primitives rather than merely replacing the renderer.
vs PixelSplat / MVSplat: these represent generalizable reconstruction using multi-view geometry; MAC-Splat is more stable on low-overlap splits, but official input-pose protocols differ, limiting conclusions to the reported evaluation conditions.
vs depth or normal priors: external priors constrain surface layout, whereas this method connects specific primitive identities across views; the approaches may be complementary, but the paper does not report gains from combining them.
Transferable insight: for point clouds or other explicit 3D representations, establish reliable local identity correspondences before constraining attributes beyond position; coordinate invariance and shared degeneracy must be reconsidered when transferring this idea.

Rating¶

Novelty: 4/5. The contribution centers on combining semantically enriched matching with primitive-level multi-attribute supervision, rather than introducing a wholly new geometric backbone.
Experimental Thoroughness: 3/5. The paper covers overlap difficulty, observable regions, cross-dataset transfer, and two module ablations, but lacks attribute-level ablations and efficiency evidence.
Writing Quality: 3/5. The task and module relationships are clear, but shape-invariance and anti-degeneracy arguments need qualification, and several implementation details remain underspecified.
Value: 4/5. It offers a reusable direct 3D supervision strategy for low-overlap reconstruction, with practically informative wide-baseline results.