General Self-Calibration with Varying Intrinsics¶

Conference: ECCV 2026
Paper: ECCV Version
Code: https://github.com/Rowing0914/Self-Calibration-Varying-Intrinsics
Area: 3D Vision
Keywords: Camera Self-Calibration, Varying Intrinsics, Kruppa Equations, Dual Image of the Absolute Conic (DIAC), Minimal Relaxations

TL;DR¶

This paper introduces a unified algebraic framework for multiview camera self-calibration with varying intrinsics by pushing structured parameter-space priors forward into dual image of the absolute conic (DIAC) space and sampling locally independent square subsystems via Jacobian rank analysis to reliably solve overdetermined Kruppa systems.

Background & Motivation¶

Self-calibration is a foundational component in unstructured multiview 3D computer vision, converting projective reconstructions into physically meaningful metric models with metric consistency up to an unknown similarity transformation. Reliable self-calibration is essential for robust downstream tasks such as Structure-from-Motion (SfM), Visual SLAM, augmented reality, and 3D metrology. Traditional self-calibration theory has predominantly operated under the assumption of static camera intrinsics across views or relied on complete-intersection arguments—which assume that every added geometric equation independently reduces the solution dimension by one. However, modern capture scenarios frequently violate these idealized assumptions due to optical zoom, continuous autofocus, and post-capture electronic image stabilization (EIS) that dynamically crops and shifts frames, altering camera intrinsics continuously.

When intrinsics vary dynamically, classical formulations face significant algebraic hurdles. Practical task-specific calibration priors (such as zero skew, fixed aspect ratio, or shared principal points across views) are naturally and intuitively expressed in the intrinsic parameter space ($K$-space). In contrast, pairwise epipolar relations—such as Kruppa equations—naturally constrain the dual image of the absolute conic (DIAC, $\omega^* \sim K K^\top$). This creates a fundamental algebraic gap between the spaces where priors are formulated and where epipolar geometry is resolved. Furthermore, blindly combining projective Kruppa constraints across views in the varying-intrinsic regime yields overdetermined polynomial systems riddled with subtle algebraic dependencies, causing conventional algebraic solvers or unprincipled heuristics to suffer from catastrophic failure or combinatorial root explosion. Existing approaches have largely been restricted to isolated hand-crafted regimes (such as shared focal length or known centered principal points) and lack a general mathematical framework.

To overcome these barriers, this work presents a systematic strategy to push polynomial intrinsic priors from $K$-space directly into DIAC space and to extract well-conditioned, square minimal subsystems via local Jacobian analysis. Core idea: map arbitrary polynomial intrinsic priors from parameter space to DIAC space via pushforward varieties, and construct locally independent minimal relaxation subsystems using rank-revealing Jacobian factorizations to achieve general and well-conditioned self-calibration under varying intrinsics.

Method¶

Overall Architecture¶

The input consists of pairwise fundamental matrices $F_{ji}$ and corresponding epipoles $e'_{ji}$ estimated across views along with user-specified intrinsic prior constraints; the output is the set of recovered camera intrinsic matrices $K_i$ for each view. The pipeline operates in three interconnected algebraic phases: First, structured intrinsic priors in parameter space are pushed forward into DIAC space using rational quadratic parameterizations and symbolic Gröbner bases, defining the pushforward variety. Second, pairwise projective Kruppa constraints are formulated via 2x2 minors of matched matrix columns and stacked together with the pushforward variety constraints, producing a globally overdetermined polynomial system. Third, rank-revealing QR decomposition on the system Jacobian at generic configurations samples locally independent square subsystems (minimal relaxations), which are solved using homotopy continuation or algebraic elimination, followed by Cholesky/RQ decomposition to extract the camera intrinsics.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multiview Epipolar Input<br/>Fundamental matrices Fij and epipoles"] --> B["Pushforward Prior Mapping<br/>Parameter priors mapped into DIAC variety"]
    B --> C["Kruppa System Assembly<br/>Projective epipolar and DIAC prior constraints"]
    C --> D["Jacobian Minimal Relaxation<br/>Rank-revealing QR selects independent square subsystems"]
    D --> E["Numerical Solving & Factorization<br/>Recovers per-view intrinsic matrices Ki"]

Key Designs¶

1. Pushforward Prior Mapping: Bridging Parameter Space and DIAC Space

To resolve the discrepancy between parameter-space priors and conic-space Kruppa constraints, the framework defines intrinsic priors as an algebraic set $\mathcal{P}_{\mathcal{K}} = \{ K=(K_1, \dots, K_m) \in (\mathbb{C}^5)^m \mid \mathcal{F}_{\mathrm{task}}(K) = 0 \}$. Recognizing the rational quadratic map $\pi_{\omega^*}: K_i \mapsto \omega_i^* \sim K_i K_i^\top$, the method constructs the pushforward variety $\mathcal{P}_{\mathcal{W}} := \overline{\pi_{\omega^*}(\mathcal{P}_{\mathcal{K}})} \subset (\mathbb{P}^5)^m$ via its Zariski closure. Using symbolic Gröbner basis elimination, polynomial equations that vanish on $\mathcal{P}_{\mathcal{W}}$ are derived explicitly. The dimension $d = \dim(\mathcal{P}_{\mathcal{W}})$ directly determines the true remaining geometric degrees of freedom of the camera network after enforcing the priors, establishing the exact theoretical threshold of independent geometric equations needed to yield a 0-dimensional, finitely solvable system.

2. Projective Kruppa Formulation for Independent Conics: Retaining View-Specific Variables

Unlike classical Kruppa solvers that enforce a single global $\omega^*$ across all images, this approach models each view $i$ with its own symmetric projective $3 \times 3$ DIAC variable $\omega_i^*$ (with the scale pinned by $(\omega_i^*)_{33} = 1$). For every view pair $(i, j)$, the projective equality is: $$ F_{ji} \omega_i^ F_{ji}^\top \;\sim\; [e'{ji}]\times \omega_j^ [e'{ji}]\times $$ Denoting the symmetric matrices on the left and right as $A_{ji}$ and $B_{ji}$, their six unique upper-triangular entries are flattened into vectors $\mathbf{a}_{ji}, \mathbf{b}_{ji} \in \mathbb{R}^6$. Collinearity is enforced by forming the $6 \times 2$ matrix $C_{ji} = [\mathbf{a}_{ji}, \mathbf{b}_{ji}]$ and setting all its $2 \times 2$ minors to zero: $$ f_{ji}^{(p,q)} = a_{ji,p} b_{ji,q} - a_{ji,q} b_{ji,p} = 0, \quad 1 \le p < q \le 6 $$ This generates $\binom{6}{2} = 15$ quadratic polynomial constraints per image pair. Stacking these constraints across selected image pairs $\mathcal{V}$ alongside the DIAC pushforward prior equations yields an overdetermined polynomial system in the unknown conics $(\omega_1^*, \dots, \omega_m^*)$.

3. Jacobian Minimal Relaxation Sampling: Extracting Well-Conditioned Square Subsystems

Because combining Kruppa equations across multiple pairs yields an overdetermined system containing subtle algebraic dependencies, naive root-finding struggles. The method adopts a minimal relaxation paradigm by analyzing the Jacobian of the full equation set $J = \left. \frac{\partial \mathcal{G}}{\partial \mathbf{x}} \right|_{(\mathbf{p}_0, \mathbf{x}_0)}$ evaluated at a generic configuration $(\mathbf{p}_0, \mathbf{x}_0)$. To find an independent subsystem of size $n = \dim(\mathcal{X})$, column-pivoted QR factorization is applied to $J^\top$, selecting the first $r = \operatorname{rank}(J) = n$ pivot rows corresponding to linearly independent constraints in $\mathcal{G}$. To explore the combinatorial space of minimal relaxations and guard against ill-conditioned geometries, random row permutations and positive diagonal row scalings are applied, sampling a small bank of $N=10$ independent minimal relaxations. The condition numbers $\kappa(J_{\mathcal{F}}) = \sigma_{\max} / \sigma_{\min}$ of the resulting square subsystems consistently concentrate within a benign range ($10^3 \sim 10^4$), ensuring stable numerical solving via homotopy continuation or polynomial elimination before Cholesky/RQ recovery of physical intrinsics.

Loss & Training¶

This work presents an algebraic geometric initialization method and does not involve neural network training or loss minimization. For real-world execution, the minimal solver is embedded into an MSAC (M-estimator Sample Consensus) robust estimation loop: in each iteration, an 8-point correspondence sample estimates candidate fundamental matrices, the minimal relaxation is evaluated, and the solution minimizing epipolar reprojection residual and algebraic error across all views is selected over 20 iterations. The recovered intrinsic matrices provide an accurate initialization for standard non-linear Bundle Adjustment in pipelines like COLMAP.

Key Experimental Results¶

Main Results¶

The framework is evaluated across controlled synthetic noise benchmarks as well as real dynamic video sequences from the InFlux dataset, encompassing various varying-intrinsic tasks (ff000-f: unknown shared focal length; ff000-none: independent varying focal length; ffuv0-uv: fixed aspect ratio, zero skew, shared principal point; fguv0-uv: varying focal length, aspect ratio, zero skew, shared principal point).

Table 1 details the focal-length relative error $\Delta_{fg}$ and principal-point relative error $\Delta_{uv}$ under synthetic Gaussian pixel noise at $\sigma = 1.0$ px:

Task Config	Solver	Views $m$	Focal Error $\Delta_{fg}$ ($\sigma=1.0$ px)	Principal Point Error $\Delta_{uv}$ ($\sigma=1.0$ px)	Note
ff000-f	Linear DAQ (Pollefeys)	2	0.457	−	Classical linear DAQ baseline
ff000-f	Ours (Kruppa-DIAC)	2	0.511	−	Matches DAQ performance closely
ff000-none	Pairwise SVD (Lourakis)	2	>1.00 (degraded)	−	Pairwise decoupling lacks multiview consistency
ff000-none	Ours (Kruppa-DIAC)	2	0.485	−	Stable solving for dynamic focal lengths
ffuv0-uv	Pairwise (Lourakis)	3	0.805	0.797	Degrades due to center principal point assumption
ffuv0-uv	Ours (Kruppa-DIAC)	3	1.737	1.507	High algebraic ambiguity under weak cross-view constraints
fguv0-uv	Ours (Kruppa-DIAC)	4	0.547	0.413	Robust and accurate across noise levels

Table 2 reports the unweighted sequence-level median errors (Mean $\pm$ Std) on five challenging real-world InFlux video sequences featuring dynamic zoom and principal point motion (original Table 2):

Method / Task Config	Focal Error $\Delta_{fg}$ (Mean $\pm$ Std) ↓	Principal Point Error $\Delta_{uv}$ (Mean $\pm$ Std) ↓	Empirical Behavior
ff000-f (Kruppa, Ours)	0.725 ± 0.169	−	Preserves parity with classical static solvers
ff000-f (Pollefeys DAQ)	0.857 ± 0.116	−	Linear DAQ baseline
ff000-none (Ours)	0.604 ± 0.162	−	Stable focal recovery under genuine zoom
ffuv0-uv (Kruppa, Ours)	0.790 ± 0.104	0.312 ± 0.086	Substantially better principal-point accuracy
ffuv0-uv (Lourakis)	0.805 ± 0.140	0.797 ± 0.001	High error due to rigid center-point assumption
fguv0-uv (Ours)	0.748 ± 0.105	0.464 ± 0.133	Fully general varying-intrinsic formulation

Ablation Study¶

The algebraic complexity and solution space cardinality across different intrinsic priors and variable sharing configurations were thoroughly analyzed using numerical monodromy and Bernstein–Kushnirenko–Khovanskii (BKK) polyhedral bounds (original Table 1):

Task	Shared Variables	Views $m$	Variety Dim $\dim(\mathcal{P}_{\mathcal{W}})$	# Subsystems	Solution Count (Max / Min #sol.)	Nature of Complexity
ff000	f	2	1	15	2 / 1	Monodromy root count; minimal algebraic ambiguity
ff000	none	2	2	105	1 / 1	Independent focal lengths; unique isolated solution
ffuv0	fuv	3	3	$1.42 \times 10^4$	16 / 16	Shared focal and principal point; exactly 16 roots
ffuv0	none	5	15	$1.62 \times 10^{20}$	372,918 / 372,191	Unshared parameters; massive combinatorial explosion
ffuv0	uv	3	5	$1.22 \times 10^6$	40 / 40	Varying focal with shared principal point; 40 roots
fguv0	uv	4	10	$5.72 \times 10^{12}$	144 / 98	Pragmatic varying-intrinsic setup; moderate root count

Furthermore, an ablation simulating Electronic Image Stabilization (EIS) cropping (original Table 3) was performed to test off-center optical axes. While COLMAP's image-center heuristic suffered a severe breakdown ($\Delta_{uv} = 2.483 \pm 0.608$), the proposed Kruppa algebraic relaxations recovered accurate principal points ($\Delta_{uv} = 0.395 \pm 0.475$ for ffuv0-uv and $0.665 \pm 0.106$ for fguv0-uv), demonstrating resilience to arbitrary optical offsets.

Key Findings¶

Cross-view parameter sharing dictates algebraic tractability: removing sharing constraints (e.g., ffuv0-none) triggers an exponential explosion in BKK bounds and root counts (>370,000 roots), rendering numerical recovery fragile, whereas enforcing shared principal points collapses root counts to 40–144, restoring robust solvability.
Naive degree-of-freedom counting fails in varying-intrinsic regimes: the projective structure of Kruppa relations combined with quadratic DIAC mappings induces hidden algebraic dependencies, disproving the assumption that constraints always behave as complete intersections.
Jacobian-based minimal relaxation sampling yields well-conditioned linear subsystems: a modest sample of $N=10$ relaxations reliably identifies square subsystems with condition numbers around $10^3 \sim 10^4$, delivering stability even under 1.0 px epipolar noise.

Highlights & Insights¶

Algebraic Pushforward of Intrinsic Priors: Establishes a rigorous algebraic bridge between human-interpretable parameter-space constraints and the projective variety of dual absolute conics using polynomial pushforward mappings and Gröbner bases.
Rank-Revealing Jacobian Relaxation: Employs column-pivoted QR on the measurement Jacobian to dynamically extract square, minimally independent subsystems from overdetermined, algebraically entangled Kruppa systems.
Robustness to Non-Centered Optical Axes: Eliminates the fragile heuristic assumption that principal points lie at the image center, enabling accurate calibration for videos altered by electronic stabilization or sensor cropping.

Limitations & Future Work¶

Absence of Critical Motion Sequence Handling: The current algebraic formulation focuses on generic camera trajectories; degenerate motion paths (such as pure rotations or optical axes intersecting on a conic) remain unaddressed.
Sensitivity to Initial Epipolar Quality: As a multi-stage algebraic approach, the accuracy of the formulation depends on the precision of initial fundamental matrix estimation from pairwise matches.
Future Directions: Integrating the algebraic minimal relaxation directly into differentiable or end-to-end Bundle Adjustment pipelines, and extending the pushforward paradigm to non-pinhole camera models such as radial distortion and omnidirectional lenses.

vs Classical DAQ Metric Upgrade (Pollefeys et al. [24]): Pollefeys et al. require an explicit projective reconstruction and rely on dual absolute quadrics with complete-intersection assumptions, primarily suited for static focal lengths; this work operates directly on pairwise Kruppa relations and readily accommodates arbitrary varying-intrinsic configurations.
vs Pairwise SVD Decoupling (Lourakis & Deriche [22]): Lourakis solves independent pairwise Kruppa equations and averages them, discarding global multiview consistency and relying on centered principal point guesses; this paper couples multiview DIAC constraints globally, yielding superior accuracy under genuine principal point shifts.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ Formulates the first pushforward framework for varying-intrinsic calibration with Jacobian-based minimal relaxation.
Experimental Thoroughness: ⭐⭐⭐⭐☆ Rigorous evaluation bridging algebraic root bounds, synthetic noise sweeps, and real InFlux dynamic video datasets.
Writing Quality: ⭐⭐⭐⭐⭐ Mathematically rigorous, clearly structured, and articulate in explaining complex algebraic geometry concepts.
Value: ⭐⭐⭐⭐☆ Solves long-standing algebraic ambiguity in dynamic camera calibration, providing a powerful initializer for practical SfM and SLAM systems.