Skip to content

title: >- [Paper Note] Disentangling Rotation and Translation from SE(3)-Equivariant Features for Shape Assembly description: >- [ECCV 2026][3D Vision][Shape Assembly] SOT encoder explicitly disentangles rotation and translation into SO(3)- and T(3)-equivariant features via null-space projection and direct subtraction. tags: - ECCV 2026 - 3D Vision - Shape Assembly - SE(3) Equivariance - Feature Disentanglement date: 2026-09-19 content_hash: 3ae58a51a35ab45a

Disentangling Rotation and Translation from SE(3)-Equivariant Features for Shape Assembly

Conference: ECCV 2026
Paper: ECCV Official
Code: https://github.com/maroo-sky/SOT
Area: 3D Vision
Keywords: 3D Shape Assembly, SE(3) Equivariance, Feature Disentanglement, Vector Neurons, Rigid Pose Estimation

TL;DR

To resolve the performance degradation and poor generalization caused by entangled rotation and translation in 3D shape assembly, this paper proposes the SOT encoder, which decouples rigid motions into an SO(3)-equivariant rotation feature via null-space projection and a T(3)-equivariant translation feature via translation loss and direct algebraic subtraction.

Background & Motivation

In real-world environments where broken parts or structural components are spatially dispersed, 3D shape assembly requires inferring geometric compatibilities and accurately predicting 6D rigid poses—specifically decomposing motions into 3D rotations and 3D translations. Practical applications span pairwise object insertion, multi-part furniture assembly, and the reconstruction of complex fractured artifacts. To achieve robustness against arbitrary spatial poses, modern assembly pipelines increasingly adopt SE(3)-equivariant backbones (such as VN-DGCNN or SE(3)-Transformers) that naturally preserve transformation semantics under rigid transformations.

However, existing frameworks predominantly rely on a single SE(3)-equivariant feature or coupled dual representations that mix rotation and translation together. Geometrically, the special Euclidean group SE(3) is a semidirect product of the compact, non-linear rotation group SO(3) and the non-compact Euclidean translation group T(3). When both factors are entangled within a single representation space, translation offsets easily distort the orientation-sensitive rotational features, leading to catastrophic failure under wide-range translational variations. Prior works such as VNT attempted to eliminate translation by strictly enforcing a hard constraint where weight matrix rows sum to one (\(\sum_l \mathbf{W}_{kl} = 1\)), which severely cripples neural network expressiveness and feature capacity. Meanwhile, dual-pipeline registration networks like FINet rely solely on task losses for soft separation, suffering from substantial factor leakage.

To resolve this fundamental tension between representation capacity and rigorous geometric disentanglement, this paper explores an explicit structural decoupling mechanism without imposing destructive parameter constraints. Core idea: propose the SOT encoder, which isolates SO(3)-equivariant rotation features by projecting onto the orthogonal null space of translation via RETI-VN and a weak row-sum regularizer, while extracting T(3)-equivariant translation features by suppressing rotation residuals through translation supervision and directly subtracting the rotation representation from the global SE(3) feature.

Method

Overall Architecture

The SOT (SO(3)- and T(3)-decomposed) encoder is designed as a plug-and-play feature extractor that decouples input transformed point clouds \(\mathbf{P}' = \mathbf{P}\mathbf{R} + \mathbf{1}_n^\top \mathbf{t}\) into separate SO(3)-equivariant and T(3)-equivariant features. The pipeline first transforms raw coordinates into vector-list features using an initial vector feature extractor. In the rotation branch, stacked RETI-VN layers project features onto the orthogonal complement of the translation subspace, assisted by a soft row-sum regularization loss \(\mathcal{L}_w\) to output a clean SO(3)-equivariant rotation feature \(f'(\mathbf{P}')\). In the translation branch, standard VNT layers extract full SE(3) representations \(V'\), from which the rotation feature \(f'(\mathbf{P}')\) is directly subtracted (\(V' - f'(\mathbf{P}')\)) under translation loss supervision to isolate the T(3)-equivariant translation feature \(f(\mathbf{P}')\). Both disentangled features are subsequently fed into arbitrary downstream shape assembly networks to regress the final assembly pose \((\hat{\mathbf{R}}, \hat{\mathbf{t}})\).

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Transformed Point Cloud<br/>P' = PR + 1 t"] --> B["Initial Vector Feature Extractor<br/>VFeat(P')"]
    B --> C["Rotation Branch: RETI-VN<br/>Orthogonal complement projection"]
    B --> D["Translation Branch: VNT<br/>Global SE(3) feature extraction"]
    C --> E["Row-Sum Regularizer Lw<br/>Suppress translation leakage"]
    C --> F["Direct Subtraction<br/>V' - f'(P') cancels rotation"]
    D --> F
    E --> G["SO(3)-Equivariant Feature<br/>f'(P')"]
    F --> H["T(3)-Equivariant Feature<br/>f(P')"]
    G --> I["Downstream Shape Assembly<br/>Predict rotation R and translation t"]
    H --> I

Key Designs

1. RETI-VN Layer: Eliminating translation via orthogonal null-space projection and weak row-sum regularization Prior rotation-equivariant layers (e.g., VNT) enforce a strict normalization condition on the weight matrix \(\sum_l \mathbf{W}_{kl} = 1\) to mathematically eliminate translation, which drastically restricts network expressiveness and limits generalization. To overcome this limitation, SOT introduces the Rotation-Equivariant and Translation-Invariant Vector-Neuron (RETI-VN) layer under weak constraints. Given transformed point clouds \(\mathbf{P}\mathbf{R} + \mathbf{1}_n^\top \mathbf{t}\), RETI-VN parameterizes the translation subspace with a learnable matrix \(\mathbf{W}_Q \in \mathbb{R}^{m \times r}\) and derives an orthonormal basis \(\mathbf{Q}_t \in \mathbb{R}^{m \times r}\) via QR factorization (\(\mathbf{Q}_t^\top \mathbf{Q}_t = \mathbf{I}_r\)). It then applies the complementary projection operator \(\mathbf{I}_m - \mathbf{Q}_t \mathbf{Q}_t^\top\) to extinguish translational energy: $$ \text{RETI-VN}(\mathbf{P}') = \mathbf{W}\mathbf{P}\mathbf{R} + (\mathbf{I}m - \mathbf{Q}_t \mathbf{Q}_t^\top) \tilde{\mathbf{W}} \mathbf{1}_n^\top \mathbf{t} $$ Because the geometric projection alone spans an estimated subspace, translation components can still leak through unmodeled directions. The authors observe that translation sensitivity is directly governed by the row sums of the weight matrix \(\sum_j \mathbf{W}_{i,j}\). Therefore, a soft regularizer \(\mathcal{L}_w\) is introduced to encourage each row sum to vanish: $$ \mathcal{L}_w = \frac{1}{m} \sum \right| $$ This soft penalty eliminates translation leakage without damaging parameter representation capacity, progressively driving the layer toward ideal translation invariance and pure SO(3) equivariance as layers deepen.} \left| \sum_{j} \mathbf{W}_{i,j

2. Translation Supervision Mechanism: Suppressing rotation-dependent residuals and theoretical bounds of composite maps Most assembly models attempt to extract translation features purely by applying a translation regression loss. The authors analyze the theoretical properties of composite SE(3)-equivariant maps \(g = h \circ f\). Under standard point-cloud actions, the mapping satisfies \(g(\mathbf{P}\mathbf{R} + \mathbf{1}_n^\top \mathbf{t}) = g(\mathbf{P})\mathbf{R} + \mathbf{1}_n^\top \mathbf{t}\). Minimizing the translation prediction loss against ground truth translates to: $$ \mathcal{L}_{\text{trans}} = | g(\mathbf{P}\mathbf{R} + \mathbf{1}_n^\top \mathbf{t}) - \mathbf{1}_n^\top \mathbf{t} |_2 = | g(\mathbf{P})\mathbf{R} |_2 = | g(\mathbf{P}) |_2 $$ This demonstrates that minimizing translation loss is mathematically equivalent to suppressing the rotation-dependent residual \(g(\mathbf{P})\mathbf{R}\). However, Proposition 2 establishes that requiring \(g(\mathbf{P}) = h(f(\mathbf{P})) = \mathbf{0}\) for all \(\mathbf{P}\) is impossible under SE(3) equivariance because it inevitably implies \(\mathbf{0} = \mathbf{1}_n^\top \mathbf{t}\) for non-zero translations. Thus, a single forward branch supervised by translation loss alone cannot fundamentally decouple translation from rotation.

3. Direct Discrepancy-Based Subtraction: Decoupling T(3)-equivariant representations To overcome the theoretical impossibility of zeroing out the composite SE(3) map, SOT adopts an algebraic discrepancy formulation. Instead of forcing a single network to learn isolated translation from scratch, the translation branch first extracts a complete SE(3)-equivariant representation \(V' = \text{VNT}(\mathbf{P}')\), and then directly subtracts the purified SO(3)-equivariant feature \(f'(\mathbf{P}')\) extracted by the rotation branch: $$ f(\mathbf{P}') = V' - f'(\mathbf{P}') $$ The translation objective is reformulated to supervise the discrepancy between the SE(3) feature and the RETI feature: $$ \mathcal{L}_{\text{trans}} = | h(f(\mathbf{P}\mathbf{R} + \mathbf{1}_n^\top \mathbf{t}) - f'(\mathbf{P}\mathbf{R} + \mathbf{1}_n^\top \mathbf{t})) - \mathbf{1}_n^\top \mathbf{t} |_2 = | h(f(\mathbf{P}) - f'(\mathbf{P}))\mathbf{R} |_2 $$ By subtracting \(f'(\mathbf{P}')\), the rotational geometric manifold common to both representations cancels out, leaving behind a clean feature space dedicated to encoding affine translation vectors. This algebraic shortcut relieves the regression head from disentangling complex rotational couplings internally.

Loss & Training

The overall training objective combines the downstream shape assembly loss \(\mathcal{L}_h\) with the translation suppression regularization term \(\mathcal{L}_w\): $$ \mathcal{L} = \mathcal{L}_h + \mathcal{L}_w $$ where rotation loss \(\mathcal{L}_{\text{rot}} = \| h \circ f'(\mathbf{P}\mathbf{R} + \mathbf{1}_n^\top \mathbf{t}) - \mathbf{R} \|_2\) supervises the rotation branch and \(\mathcal{L}_{\text{trans}}\) supervises the subtracted translation branch. The model is trained for 100 epochs on 2BY2 and Breaking Bad, and for 200 epochs on ShapeNet. Pose-regression baselines (NSM, SE(3)-Assembly, 2BY2NET) are optimized with Adam (learning rate \(1 \times 10^{-4}\), weight decay \(1 \times 10^{-6}\)) without learning rate decay. Experiments run on 4 NVIDIA GeForce RTX 3090 GPUs with a total batch size of 64. Under the realistic wide-range setting, translation offsets are sampled uniformly from \(\mathbf{t}_i \sim \mathcal{U}[-2|\mathbf{C}_{P_i}|, 2|\mathbf{C}_{P_i}|]\) along each axis to prevent trivial centroid overfitting.

Key Experimental Results

Main Results

SOT is evaluated as a drop-in replacement across three benchmarks: 2BY2, Breaking Bad (everyday category), and ShapeNet, integrated with direct pose regression architectures (NSM, SE(3)-Assembly, 2BY2NET). Baselines include DGCNN, VN-DGCNN, and FINet-EB. Metrics comprise rotation RMSE (RMSE-R in degrees), translation RMSE (RMSE-T), Chamfer Distance (CD), and Part Accuracy (PA, thresholded at CD \(< 0.1\)).

Method Encoder 2BY2 RMSE-R ↓ 2BY2 RMSE-T ↓ 2BY2 CD ↓ 2BY2 PA ↑ Breaking Bad RMSE-R ↓ Breaking Bad RMSE-T ↓ Breaking Bad CD ↓ Breaking Bad PA ↑ ShapeNet RMSE-R ↓ ShapeNet RMSE-T ↓ ShapeNet CD ↓ ShapeNet PA ↑
NSM DGCNN 37.4801 0.4019 0.7360 0.5488 78.3134 0.1295 0.0781 0.8824 54.9841 0.1238 0.0961 0.8596
NSM VN-DGCNN 37.0824 0.4197 0.5914 0.5664 87.3287 0.1328 0.0979 0.8393 64.9645 0.1297 0.1124 0.8272
NSM FINet-EB 44.3693 0.4853 0.7518 0.5165 85.8648 0.1412 0.0879 0.8655 65.7310 0.1325 0.1198 0.8158
NSM SOT (Ours) 29.1815 0.3982 0.3490 0.6121 71.9768 0.1100 0.0769 0.8887 44.9236 0.1157 0.0893 0.8694
SE(3)-Assembly DGCNN 43.1994 0.4403 1.1484 0.4403 75.9024 0.1005 0.1957 0.7425 46.6635 0.1322 0.1410 0.8061
SE(3)-Assembly VN-DGCNN 36.4079 0.3569 1.4027 0.5243 55.3831 0.0692 0.1087 0.8402 75.7370 0.1086 0.1579 0.7719
SE(3)-Assembly FINet-EB 39.1817 0.4113 2.2576 0.5536 53.5764 0.0814 0.1104 0.8423 59.1983 0.1038 0.1497 0.7814
SE(3)-Assembly SOT (Ours) 32.3640 0.3039 0.6200 0.5911 49.2427 0.0684 0.0842 0.8662 37.1323 0.0957 0.1100 0.8383
2BY2NET DGCNN 30.4387 0.3386 2.2349 0.5207 72.0284 0.0865 0.1813 0.7575 82.8681 0.1301 0.2872 0.6425
2BY2NET VN-DGCNN 30.3856 0.4391 6.1800 0.3812 62.0458 0.0724 0.1683 0.7976 62.2055 0.0765 0.1983 0.7829
2BY2NET FINet-EB 29.7778 0.3597 4.9334 0.5330 59.8457 0.0685 0.1396 0.8181 60.2706 0.1906 0.1462 0.7945
2BY2NET SOT (Ours) 22.4719 0.2857 1.5265 0.5623 53.7092 0.0568 0.0873 0.8533 51.6656 0.0799 0.1109 0.8179

Ablation Study

Ablations on 2BY2 with SE(3)-Assembly evaluate each core component alongside the translation leakage metric \(\mathcal{D}_{rt-r} = \| f'(\mathbf{P}\mathbf{R} + \mathbf{1}_n^\top \mathbf{t}) - f'(\mathbf{P}\mathbf{R}) \|_2\) (measuring rotation feature sensitivity to translation offsets).

Config RMSE-R (°) ↓ RMSE-T ↓ CD ↓ PA ↑ \(\mathcal{D}_{rt-r}\) ↓ Note
Full Model (Ours) 32.3640 0.3039 0.6200 0.5911 0.0381 Dual-branch RETI-VN + \(\mathcal{L}_w\) + direct subtraction
w/o regularizer \(\mathcal{L}_w\) 34.0900 0.3161 0.9088 0.5687 0.0682 Omitting \(\mathcal{L}_w\) surges translation leakage \(\mathcal{D}_{rt-r}\) by 79.0%
w/o projector \(\mathbf{Q}_t\) 32.8937 0.3352 0.7042 0.5652 0.0427 Without null-space projection, translation error and CD degrade
w/o direct subtraction (DS) 32.8855 0.3315 0.7656 0.5376 - Without feature subtraction, PA drops by 5.35%

Key Findings

  • Translation leakage quantitatively verified: Removing row-sum regularizer \(\mathcal{L}_w\) causes translation leakage \(\mathcal{D}_{rt-r}\) to spike from 0.0381 to 0.0682, demonstrating that soft weight regularization is essential to suppress numerical translation bleed-through that escapes subspace projection.
  • Superiority over restrictive equivariant layers: Comparing rotation branch instantiations on 2BY2, replacing RETI-VN with VNT spikes RMSE-R to 84.4165° in SE(3)-Assembly and 88.3118° in NSM, whereas standard VN achieves 38.0648° and 51.1724° respectively, and RETI-VN achieves the lowest errors of 32.3640° and 29.1815°. This verifies that VNT's rigid row-sum constraint destroys feature expressiveness.
  • Remarkable efficiency in parameter budget: On ModelNet40 rigid pose regression, SOT achieves the lowest rotation error (6.1905° vs baselines 7.9962°~8.9837°) and translation error (0.0304 vs 0.0313~0.0550) using only 34.79K parameters—less than half that of VN-DGCNN (72.21K) and FINet-EB (77.86K), and an order of magnitude smaller than DGCNN (309.63K).

Highlights & Insights

  • Geometric null-space projection: Deriving an adaptive translation subspace via learnable QR factorization and zeroing translation responses through complementary orthogonal projection provides a clean geometric inductive bias without sacrificing weight freedom.
  • Overcoming the composite equivariant mapping paradox: The authors provide an elegant algebraic proof (Proposition 2) establishing why composite SE(3)-equivariant mappings cannot zero out under non-zero translations, neatly resolving this theoretical deadlock by directly subtracting SO(3) features from SE(3) features.
  • Broad versatility across assembly regimes: SOT serves as an agnostic backbone, demonstrating consistent performance leaps across direct pose regression (NSM, SE(3)-Assembly, 2BY2NET), dense feature matching (CMNet), pairwise assembly, and multi-part assembly (2 to 20 parts).

Limitations & Future Work

  • Subspace dimension hyperparameter sensitivity: The optimal subspace rank \(r\) is tuned empirically (best at \(r=13\) for channel size \(m=21\)), which lacks dynamic rank adaptation across shapes of drastically differing geometric complexity.
  • Ambiguities in rotational symmetry: For parts with continuous or discrete rotational symmetries (such as cylinders or circular plates), disentangled orientation features can suffer from multi-modal alignment ambiguities.
  • Future directions: Extending SOT from rigid geometry to articulated assemblies and deformable contact interfaces, or integrating disentangled representations with generative diffusion/flow-matching reassembly frameworks.
  • vs VN-DGCNN / Vector Neurons: Standard VN only supports SO(3) equivariance and is highly vulnerable to arbitrary spatial translations; SOT introduces projection and subtraction branches to achieve joint SO(3) and T(3) disentangled equivariance.
  • vs VNT: VNT imposes an inflexible row-sum=1 constraint that drastically impairs representation capacity; SOT replaces hard constraints with orthogonal null-space projection and weak soft regularization, preserving full parameter expressiveness.
  • vs FINet / DBDNet: FINet utilizes dual branches for point-cloud registration but relies on end-to-end task loss for separation, leading to severe representation entanglement; SOT provides structural null-space isolation and direct algebraic feature cancellation.

Rating

  • Novelty: ⭐⭐⭐⭐☆ Strong algebraic grounding combining Lie group factor decomposition with orthogonal null-space projections.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluations covering pairwise, multi-part, dense matching, pose regression, and feature equivariance/invariance metrics.
  • Writing Quality: ⭐⭐⭐⭐⭐ Clear mathematical derivations, well-structured proofs, and informative visual diagrams.
  • Value: ⭐⭐⭐⭐⭐ An effective plug-and-play backbone that establishes new state-of-the-art benchmarks for 3D robotic assembly.