Inclusive Interactive Collisions for Multi-View Consistent Compositional 3D Generation

Conference: ECCV 2026
Paper: ECCV 2026 Poster 3923
Code: https://github.com/threestudio-project/threestudio
Area: 3D Vision
Keywords: compositional 3D generation, 3D Gaussian Splatting, interaction region collisions, multi-view adaptive SDS, multi-view consistency

TL;DR

I2C-3D presents a coarse-to-fine optimization framework for compositional 3D asset generation, using an Inclusive Interactive Collisions strategy to naturally gather Gaussian primitives into plausible contact zones and Multi-View Adaptive Score Distillation Sampling to resolve cross-view inconsistency and the Janus problem.

Background & Motivation

Recent advancements in generative diffusion models and differentiable rendering have enabled high-quality single-object 3D asset synthesis through Score Distillation Sampling (SDS) and 3D Gaussian Splatting (3DGS). However, synthesizing multi-object compositional scenes characterized by intricate physical interactions remains a formidable challenge. Feed-forward 3D reconstruction models are primarily trained on isolated single-object datasets, failing when confronted with multi-object topologies. Concurrently, optimization-based methods struggle to distill compositional priors from pre-trained 2D diffusion models, which lack fine-grained spatial controllability and frequently produce attribute leakage, spatial disarray, and cross-view geometry tearing (the notorious Janus problem).

A deeper bottleneck lies in modeling the distribution of 3D Gaussian primitives in interactive boundary regions. Empirical investigation demonstrates that Gaussian primitives between interacting entities concentrate densely around the midpoint of their connecting line, forming a roughly cylindrical contact zone. Existing compositional approaches either optimize isolated objects before rigid assembly—yielding unrealistic penetrations or floating gaps—or perform naive global SDS optimization, which disperses Gaussians haphazardly and induces dark floating artifacts that degrade surface coherence.

To overcome these physical and geometric discrepancies, this paper moves beyond purely semantic 2D priors by introducing explicit bounding-box constraints and geometric collision energy based on the triangle inequality theorem directly into 3DGS optimization, paired with cross-view modulated score distillation. The core idea is to decouple compositional 3D generation into LLM-driven local-to-global layout planning followed by fine-grained Gaussian refinement, guiding primitives into plausible midpoint contact regions via an Inclusive Interactive Collisions (I2C) strategy while enforcing multi-view consistency through Multi-View Adaptive Score Distillation Sampling (MV-ASDS).

Method

Overall Architecture

I2C-3D operates through a coarse-to-fine two-stage optimization pipeline. In the coarse stage, an LLM parses multi-object text prompts to predict relative spatial arrangements and bounding box dimensions, establishing a local-to-global 3D layout. This layout is projected to a frontal 2D blueprint to condition a layout-guided diffusion model; foreground instances are isolated via Segment Anything (SAM) and reconstructed into initial 3D Gaussians using the Large Multi-View Gaussian Model (LGM) without 2D inpainting. In the refinement stage, the Inclusive Interactive Collisions (I2C) strategy enforces in-box volume fullness while steering outlying primitives toward the contact interface; simultaneously, Multi-View Adaptive Score Distillation Sampling (MV-ASDS) modulates cross-attention maps of spatial and instance tokens across viewpoints, jointly optimizing scene-level and entity-level Gaussians to produce consistent, high-fidelity compositional assets.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multi-object text prompt"] --> B["Stage 1: Local-global 3D layout generation<br/>LLM infers relative topology and bounding boxes"]
    B --> C["Stage 2: Single-object generation & composition<br/>Layout-guided diffusion + SAM + feed-forward LGM"]
    C --> D["Inclusive Interactive Collisions (I2C)<br/>In-box volume constraint + triangle collision guidance"]
    C --> E["Multi-View Adaptive Score Distillation Sampling (MV-ASDS)<br/>Spatial/instance attention modulation + multi-view denoising"]
    D --> F["High-fidelity multi-view consistent 3D scene<br/>Physically plausible interaction & flexible 3D editing"]
    E --> F

Key Designs

1. Local-global 3D layout generation and feed-forward object initialization: resolving spatial ambiguity without inpainting artifacts

Directly predicting 3D global coordinates from text prompts using LLMs leads to bounding box overlaps or excessive gaps due to a scarcity of paired 3D spatial pretraining data. To address this, the pipeline decomposes layout estimation into a two-step local-global procedure: the LLM first parses object nouns, relative spatial relations, and bounding box dimensions in local coordinates, and subsequently maps them into a coherent global world coordinate frame conditioned on viewpoint prompts. During coarse initialization, the 3D layout is rendered into a 2D bounding-box layout to guide a diffusion model, and SAM extracts individual masks without running 2D inpainting. Reconstructing initial Gaussians directly from these segmented, partially occluded views via pre-trained LGM prevents 2D inpainting hallucination from polluting 3D geometry across perspectives.

2. Inclusive Interactive Collisions (I2C): dual geometric energy guiding Gaussian density and physical contact

To ensure each object maintains a full volumetric shape while forming a coherent, non-penetrating contact interface, I2C introduces two complementary geometric constraints supported by K-means clustering initialized at bounding-box centers. For the vast majority of Gaussians assigned to each entity cluster, an In-Box loss prevents thin, deflated shapes:

\[\mathcal{L}_{\text{IB}} = \sum_{i=1}^{k}\sum_{j=1}^{N}\Big[\text{clamp}\big(\max(m_i^j - B_i^{\max}, 0)\big) + \text{clamp}\big(\max(B_i^{\min} - m_i^j, 0)\big)\Big]\]

where \(m_i^j\) denotes the local coordinate of the \(j\)-th primitive in instance \(i\), and \(B_i^{\max}, B_i^{\min}\) denote the longest and shortest bounding box extents. This term penalizes primitives escaping the box until over 95% reside within it. For the remaining primitives situated outside the box near adjacent entities, an Interaction Collision loss is formulated based on the geometric triangle inequality (\(d_{k,i} + d_{k,j} \ge d_{i,j}\)):

\[\mathcal{L}_{\text{IC}} = \sum_{k=1}^{N} \max\big(0, (d_{k,i} + d_{k,j}) - d_{i,j} - \tau\big)\]

where \(d_{i,j}\) is the Euclidean distance between bounding box centers, \(d_{k,i}, d_{k,j}\) are distances from primitive \(k\) to the two centers, and \(\tau\) is a scale-adaptive margin set to 5% of the maximum bounding box diagonal. The loss analytically pulls outlying Gaussians toward the segment connecting the two centers, generating a dense, physically grounded collision zone without interpenetration. The combined objective is \(\mathcal{L}_{\text{I2C}} = \lambda_{\text{IB}}\mathcal{L}_{\text{IB}} + \lambda_{\text{IC}}\mathcal{L}_{\text{IC}}\).

3. Multi-View Adaptive Score Distillation Sampling (MV-ASDS): disentangled token modulation and cross-view prior distillation

Standard SDS treats all tokens uniformly, causing structural blur when spatial tokens are amplified or severe multi-view hallucinations (Janus artifacts) when evaluated per viewpoint. ASDS dynamically modulates cross-attention maps based on token roles:

\[A'_i := \begin{cases} k \cdot A_i, & \text{if } i = i_{\text{spa}} \\ A_i \cdot M_i + \frac{A_i}{k} \cdot (1 - M_i), & \text{if } i = i_{\text{ins}} \\ A_i, & \text{otherwise} \end{cases}\]

where \(M_i\) represents the 2D foreground mask of instance token \(i_{\text{ins}}\), and \(k=1.25\). This modulation amplifies global spatial attention while confining instance features strictly within respective object masks. To eliminate multi-view hallucinations, the framework incorporates a multi-view diffusion model (Stable Zero123) to compute view-conditioned score distillation gradients:

\[\nabla_\theta \mathcal{L}_{\text{MV-ASDS}} = \mathbb{E}_{t,\epsilon,v} \left[ w(t) \big(\epsilon_{\phi^*}(x_t; y, t, v) - \epsilon\big) \frac{\partial g(\theta, v)}{\partial \theta} \right]\]

Crucially, MV-ASDS is applied not only to the composed scene but also independently to individual entities after scaling each bounding box into a canonical volume space via translation \(\beta = \text{Mean}(B_i)\) and scaling \(\gamma = B_{\text{global}}/B_i\), guaranteeing geometric fidelity and texture sharpness across all camera angles.

Loss & Training

The framework is implemented in PyTorch using ThreeStudio. The optimization runs for 1,000 iterations. Loss hyperparameters are set to \(\lambda_{\text{IB}} = 0.15\) and \(\lambda_{\text{IC}} = 1.0\), with the In-Box primitive ratio threshold set to \(0.95\). Guidance is provided by Stable Zero123 at timestep \(t=50\), with attention boost factor \(k=1.25\).

Key Experimental Results

Main Results

Experiments are conducted on the ComboVerse benchmark comprising 50 text prompts and corresponding real and synthetic images. Evaluation includes CLIP-Score for semantic alignment, GPT-4V scoring across four dimensions (Prompt Alignment, Spatial Arrangement, Geometric Fidelity, Scene Quality), Janus Rate (JR), and multi-view consistency accuracy (ACC).

Method	3D Rep.	CLIP-Score ↑	GPT-4V Prompt Align. ↑	GPT-4V Spatial Arrang. ↑	GPT-4V Geom. Fidelity ↑	GPT-4V Scene Quality ↑	JR (%) ↓	ACC (%) ↑	Training Time ↓
DreamFusion	NeRF	0.273	50.6	51.3	50.9	28.5	55.6	42	6.0h
Latent-NeRF	NeRF	0.299	67.9	70.6	68.4	67.7	58.6	44	15min
SJC	NeRF	0.303	65.2	69.7	67.3	71.2	61.1	48	2.0h
Magic3D	NeRF	0.276	55.1	53.4	54.5	52.4	49.4	52	5.3h
GaussianDreamer	3DGS	0.295	78.4	75.3	74.3	67.5	62.5	48	15min
GSGEN	3DGS	0.304	80.9	72.9	73.7	70.2	64.2	50	3.0h
GraphDreamer	3DGS	0.286	78.5	80.3	81.1	79.6	69.7	48	3.0h
I2C-3D (Ours)	3DGS	0.314	92.7	84.5	82.6	85.3	17.3	88	28min

Ablation Study

Ablation investigations examine key modular components and the contact margin parameter \(\tau\) (refer to Section 4.4, Fig. 10, and Fig. 11):

Config / Component	Empirical Observation & Metrics	Underlying Physical/Visual Cause
Full Model	Plump geometry, natural boundary deformation, clean contact zones without dark artifacts	Balanced synergy between I2C boundary guidance and MV-ASDS cross-view prior
w/o MV-ASDS	Severe multi-view distortion, missing textures (e.g. round pillows turn square from side view)	Absence of multi-view diffusion prior leads to single-view over-smoothing and Janus collapse
w/o I2C Loss	Underfilled shapes, excessive outlier Gaussians, prominent black artifacts at contact boundaries	Lacks analytic distance pull and box constraints, leading to disordered Gaussian scattering
w/o ASDS (Attention Modulation)	Blurred surface textures, loss of fine patterns (e.g. stool grain and floral embroidery fade)	Mutual interference between spatial and instance tokens dilutes localized feature gradients
Margin \(\tau = 0.02\) (too small)	Unrealistic severe object penetration and mesh clipping	Insufficient margin allows Gaussians to encroach freely into neighboring entity volumes
Margin \(\tau = 0.5\) (too large)	Unnatural visible gaps between interacting objects	Overly strong repulsion prevents contact, violating the physical premise of touch/support
Margin \(\tau = 0.2\) (optimal adaptive)	Stable, smooth contact interfaces with realistic resting contact	5% bounding box diagonal margin provides an optimal buffer for physical contact modeling

Key Findings

• Substantial reduction in Janus artifacts: I2C-3D dramatically lowers the Janus Rate (JR) to 17.3% compared to \(55\%\sim69\%\) in prior baselines, while pushing multi-view consistency accuracy (ACC) to 88% (versus \(48\%\) in GraphDreamer).
• Geometric prior outperforms implicit 2D guidance: Explicitly constraining Gaussian distributions via bounding boxes and triangle distance minimization eliminates floating black clusters and delivers realistic contact interfaces.
• Progressive compositional 3D editing: By maintaining existing object coordinate systems, the pipeline supports incremental object insertion (e.g., sequentially adding a nightstand, flowerpot, and apple to a double bed, scaling up to 6–8 objects) without degrading existing geometry.

Highlights & Insights

• Geometric collision modeling via triangle inequality: Exploiting the condition that the sum of distances from a point to two centers achieves its minimum on their connecting segment provides an elegant, physics-free mechanism to concentrate boundary Gaussians.
• Dual token cross-attention modulation: Selectively boosting spatial relational tokens while dampening instance tokens outside their corresponding masks offers a lightweight, training-free way to prevent semantic bleeding in multi-object diffusion.
• Inpainting-free initialization pipeline: Generating multi-view Gaussians directly from un-inpainted masked instances via LGM avoids propagating synthetic 2D hallucinations into multi-view 3D space.

Limitations & Future Work

• Dependence on 2D segmentation fidelity: Coarse bounding box and cluster centers heavily rely on LLM spatial reasoning and SAM masks; segmentation errors directly degrade Gaussian cluster initializations.
• Rigid-body assumption in complex interactions: The bounding-box center midpoint assumption suits rigid and semi-rigid objects, but may be sub-optimal for highly deformable bodies (e.g., fluid splashes or cloth draping over concave meshes).
• Future directions: Integrating differentiable contact mechanics or signed distance field (SDF) normal regularizations could enable micro-level surface deformation and non-convex interactive asset generation.

Related Work & Insights

• vs ComboVerse / GraphDreamer: While ComboVerse and GraphDreamer employ spatially-aware SDS or scene graphs, they lack explicit geometric contact forces, frequently resulting in hovering entities or noisy boundaries. I2C-3D stabilizes interaction boundaries through explicit triangle-distance energy.
• vs Layout-your-3D / REPARO: Layout-driven mesh generation pipelines often struggle with intricate multi-view texture synthesis. I2C-3D leverages 3DGS alongside multi-view diffusion distillation to achieve superior geometric coherence and visual fidelity.

Rating

• Novelty: ⭐⭐⭐⭐☆ (Ingenious formulation of contact boundary collisions based on elementary triangle inequality properties in 3DGS)
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ (Comprehensive GPT-4V multi-metric evaluation, progressive editing demonstrations, and granular ablations on the ComboVerse benchmark)
• Writing Quality: ⭐⭐⭐⭐☆ (Clear formulation of the two-stage coarse-to-fine pipeline, clean mathematical notation, and thorough ablation analyses)
• Value: ⭐⭐⭐⭐☆ (Offers a practical and robust paradigm for multi-object compositional 3D asset generation and interactive editing)

Related Papers

• [ECCV 2026] Interact3D: Compositional 3D Generation of Interactive Objects
• [CVPR 2026] Multi-view Consistent 3D Gaussian Head Avatars 'without' Multi-view Generation
• [ECCV 2026] Multi-View Foundation Models
• [ECCV 2026] Geometrically Consistent Multi-View Scene Generation from Freehand Sketches
• [ECCV 2026] MVGS: Multi-view Regulated Gaussian Splatting for Novel View Synthesis