BrepLLM: Enabling Large Language Models to Understand Boundary Representations¶

Conference: ECCV 2026
arXiv: 2512.16413
Project Page: https://user-deng.github.io/BrepLLM/
Code: None
Area: 3D Vision / Multimodal VLM
Keywords: B-rep, CAD understanding, large language models, cross-modal alignment, 3D Vision

TL;DR¶

BrepLLM is the first framework that enables large language models to directly parse and reason over native B-rep (Boundary Representation) CAD models. By performing adaptive UV sampling to convert B-rep models into face-edge topology graphs, and using a hierarchical BrepEncoder to extract decoupled face, edge, and topology features, followed by CLIP contrastive pretraining and two-stage progressive LLM fine-tuning, it comprehensively outperforms point cloud- and image-based baseline models on 3D object description and classification tasks.

Background & Motivation¶

CAD is an indispensable tool in fields such as manufacturing, aerospace, and architecture, and B-rep (Boundary Representation) is the mainstream data format for industrial CAD. It precisely encodes parameterized surfaces, watertight topology, and explicit topological adjacency, offering a much higher information density than point clouds or triangular meshes. Enabling LLMs to directly understand native B-rep models is of great value to industrial design and intelligent manufacturing.

However, existing integration of LLMs with CAD follows a "procedural proxy" route: converting CAD models into command sequences, sketch operation sequences, or program code, which are then executed by a CAD kernel. For example, CAD-MLLM generates CAD command sequences, CadVLM and CAD-LLM generate sketch sequences, and cadrille and CAD-Coder use LLMs to generate CAD programs. These methods operate on the modeling process rather than B-rep entities (faces, edges, and topology), thereby failing to perform true geometric or topological reasoning. A deeper issue is that modeling sequence data is extremely scarce (the DeepCAD dataset is only one-fifth the size of the ABC dataset), while native B-rep models, as the industry-standard format, are abundant and easy to obtain at scale.

Core Idea: Decouple the face, edge, and topological information of B-rep models into hierarchical graph token sequences. Through "cross-modal contrastive pretraining + two-stage progressive fine-tuning," the LLM is enabled to directly "see and understand" native CAD geometry—without intermediate format conversion and without relying on scarce modeling history data.

Method¶

Overall Architecture¶

The core problem BrepLLM aims to solve is that B-rep contains complex geometric (coordinates, normal vectors, and curvatures on parametric surfaces) and topological (face-edge-face adjacency relations) information, whereas the input to an LLM is a sequence of tokens—representing a huge modality gap. BrepLLM bridges this gap in two major stages: first, without involving the LLM, it uses contrastive learning to align the overall representation of the B-rep with text embeddings into a shared semantic space; then, it connects the pretrained geometric encoder to the LLM, progressively mapping the fine-grained node tokens of the B-rep into the LLM's semantic space in two steps.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["B-rep Model"] --> B["Adaptive UV Sampling<br/>Face Area-Driven + Edge Length Adaptive"]
    B --> C["Face-Edge Topology Graph<br/>Face Nodes (10D) / Edge Adjacency (8D)"]
    C --> D["Hierarchical BrepEncoder<br/>Face Features / Edge Conditional Features / Global Topology<br/>Three-branch Decoupling → Concatenated 128D"]
    D --> E["Global token h_cls<br/>+ Node Token Sequence"]
    E -->|"Stage 1: CLIP Contrastive Pretraining"| F["h_cls ↔ Frozen CLIP Text Encoder<br/>InfoNCE Loss Alignment"]
    E -->|"Stage 2: LLM Fine-tuning"| G["MLP Projection + Q-Former<br/>128→1408D, 32 Learnable Queries"]
    G --> H["LoRA Fine-tuning LLM<br/>Qwen3-8B"]
    H --> I["Text Output"]

The first stage, "Cross-Modal Alignment Pretraining": performing adaptive UV sampling on the raw B-rep to discretize continuous surfaces into a face-edge topology graph (where sampled points on faces form node features, and face adjacency relations form edges). The hierarchical BrepEncoder extracts the global token \(h_{\text{cls}}\) and the node token sequence for each face \([h_1, \dots, h_n]\) from the graph. Using a CLIP-style symmetric contrastive loss (InfoNCE), \(h_{\text{cls}}\) is mapped through a projection layer to align with the text embeddings produced by a frozen CLIP text encoder (ViT-L/14).

The second stage, "Two-Stage LLM Fine-Tuning": keeping the BrepEncoder frozen, the node token sequence is projected through a two-layer MLP (from 128 to 1408 dimensions) and fed into a pretrained Q-Former (with 32 learnable queries). The fixed-length tokens output by the Q-Former are then linearly mapped into the LLM's latent space. Stage I only trains the MLP projection head, leveraging the alignment prior of the 2D-VLM to establish an initial bridge from geometry to visual semantic space; Stage II uses LoRA to jointly fine-tune key sub-layers of the Q-Former and a portion of the LLM's parameters, ultimately enabling the model to master generation capabilities "from B-rep structures to natural language."

Key Designs¶

1. Adaptive UV Sampling and Graph Representation Construction: Transforming Continuous B-reps into Discrete Graphs Consumable by Neural Networks

The geometric information of a B-rep is defined over continuous parameter domains—each face is a NURBS/analytic surface, and each edge is a parametric curve, which cannot be directly input into neural networks. An intuitive approach would be uniform sampling, but this leads to wasted computation: over-sampling small faces and under-sampling large ones. This paper proposes area-driven adaptive UV sampling to resolve this contradiction.

For face \(\mathcal{S}\), within the parameter domain \(\Omega_{\mathcal{S}} = [u_{\min}, u_{\max}] \times [v_{\min}, v_{\max}]\), the number of sampled points \(N_{\mathcal{S}}\) is determined by linear interpolation of its surface area:

\[N_{\mathcal{S}} = N_{\min}^{\text{face}} + \frac{A_{\mathcal{S}} - A_{\min}}{A_{\max} - A_{\min}} \cdot (N_{\max}^{\text{face}} - N_{\min}^{\text{face}})\]

where \(A_{\min}/A_{\max}\) represents the minimum and maximum face areas in the model. Each sampled point \((u_k, v_l)\) extracts 10-dimensional features: 3D coordinates \(\mathbf{P} \in \mathbb{R}^3\), unit normal vector \(\mathbf{n} \in \mathbb{S}^2\), mean curvature \(H\), visibility mask \(\mathbf{V} \in \{0,1\}\), face type \(\mathbf{t}\), and normalized area \(a = A_{\mathcal{S}}/A_{\max}\).

For edge \(\mathcal{C}\), the number of sampled points \(M_{\mathcal{C}}\) is linearly interpolated based on edge length. Each sampled point extracts 8-dimensional features: 3D coordinates \(\mathbf{Q}\), unit tangent vector \(\boldsymbol{\tau}\), edge type \(\mathbf{c}\), and normalized length \(b\). Both face and edge sampling counts are bounded within the range \([32, 64]\), striking a balance between geometric accuracy and computational overhead.

A topology graph is constructed with faces as nodes and adjacency relations as edges. In this way, the B-rep is converted into a heterogeneous graph where each node carries a 10D geometric feature and each edge carries an 8D boundary feature, providing a structured input for the subsequent hierarchical encoding.

2. Hierarchical BrepEncoder: A Face/Edge/Topology Three-branch Decoupled Graph Encoder

The information in a B-rep naturally possesses a hierarchical structure: fine-grained local geometry (coordinates, normal vectors, curvatures) within faces, boundary adjacency relations between faces, and overall global topology for the model. A single, flat encoder cannot simultaneously capture information at all three granularities. The BrepEncoder designed in this paper utilizes three parallel feature extraction branches corresponding to these three levels, which are finally concatenated into a unified node representation.

Face Features \(F_f\) (finest granularity): The attributes (10D) of all UV sampled points inside each face are fed into PointTransformerV3. Through self-attention, information propagates among sampled points, outputting a 32D feature vector \(F_f \in \mathbb{R}^{32}\) that captures sub-face local geometric details.

Edge Conditional Features \(F_e\) (edge granularity): An Edge Encoder is designed to encode the 5 attributes (coordinates, tangent vector, edge type, normalized length) of each edge into edge-level geometric features. These features serve as kernel weights for NNConv layers during face-to-face message passing, conditioning the influence of adjacent faces on the center face based on the shape of their shared boundary. This outputs a 32D feature vector \(F_e \in \mathbb{R}^{32}\).

Global Topology Features \(F_t\) (coarsest granularity): The 3D coordinates of faces and edges are first encoded using 2D CNNs and 1D CNNs, respectively, and then fed into two EGATConv (Edge-conditioned Graph Attention Convolution) layers. Through multi-head attention over multiple hops across the entire graph, global topological information is progressively aggregated into each face node, outputting a 64D feature vector \(F_t \in \mathbb{R}^{64}\).

The outputs of the three branches are concatenated along the dimension to form the final representation of each face node \(i\):

\[h_i = [F_t^{(i)} \| F_e^{(i)} \| F_f^{(i)}] \in \mathbb{R}^{128}\]

The BrepEncoder eventually outputs two components—a global graph feature \(h_{\text{cls}}\) (obtained by global attention pooling over all nodes, used for CLIP contrastive pretraining) and a node token sequence \([h_1, \dots, h_n]\) (used for LLM fine-tuning). This "one encoder, two output granularities" design avoids the overhead of training separate global and local encoders.

For cross-modal contrastive pretraining, \(h_{\text{cls}}\) is mapped via a projection layer to the text embedding dimension \(D\), producing \(\mathbf{z}_{\text{brep}}\); the text description is passed through the frozen CLIP text encoder to yield \(\mathbf{z}_{\text{text}}\). Within a batch, a symmetric InfoNCE loss is employed:

\[\mathcal{L}_{\text{CLIP}} = -\frac{1}{2N}\sum_{i=1}^{N}\left(\log P_{ii} + \log Q_{ii}\right)\]

where \(P_{ij} = \frac{\exp(S_{ij}/\tau)}{\sum_{k} \exp(S_{ik}/\tau)}\), \(Q_{ij} = \frac{\exp(S_{ij}/\tau)}{\sum_{k} \exp(S_{kj}/\tau)}\), \(S_{ij} = \hat{\mathbf{z}}_{\text{brep},i} \cdot \hat{\mathbf{z}}_{\text{text},j}\) is the cosine similarity after L2 normalization, and \(\tau\) is a learnable temperature parameter. In this stage, only the BrepEncoder and the projection layer are trained, while the CLIP text encoder remains frozen. This avoids expensive training of large-scale text models while pulling the geometric representations into the semantic neighborhood "comprehensible" to language models.

3. Two-Stage Progressive LLM Fine-Tuning: Bridging with 2D-VLM Priors First, then Fine-Tuning 3D-Language Alignment

Directly feeding the 128D geometric tokens of the BrepEncoder into the LLM for training will lead to instability or even divergence due to excessive modality discrepancy. The two-stage strategy proposed in this paper cleverly employs an intermediary—the pretrained Q-Former (from BLIP-2 / TinyGPT-V) and its underlying 2D-VLM alignment prior.

Stage I (Geometry-to-Vision Bridging): The BrepEncoder, Q-Former, and LLM are frozen, and only a two-layer MLP projection head (128D → 1408D) is trained. The projected geometric token sequence serves as the value input to the Q-Former. The Q-Former uses 32 learnable query vectors via cross-attention to compress the variable-length sequence into 32 fixed-length tokens. These tokens are linearly mapped into the LLM latent space and trained with an autoregressive cross-entropy loss. The elegance of this phase lies in: the Q-Former has already learned the mapping "from visual tokens to language space" during pretraining on 2D image-text data. Now, by simply swapping the input from "image patch features" to "B-rep geometric features," the model essentially leverages the alignment prior of the 2D-VLM to find "the closest known semantic anchor" for the 3D geometric features, significantly reducing the difficulty of aligning from scratch.

Stage II (3D-Language Alignment Fine-Tuning): Building upon Stage I, LoRA is used to jointly fine-tune key sub-layers of the Q-Former and a small portion of the LLM's parameters (while the BrepEncoder remains frozen). A smaller learning rate (\(5 \times 10^{-6}\)) is applied, allowing the model to progressively absorb B-rep-specific geometric and topological patterns on top of the established semantic bridge, ultimately learning to precisely describe the CAD model's structure and modeling steps in natural language.

Loss & Training¶

First Stage (Cross-Modal Alignment Pretraining): The symmetric InfoNCE contrastive loss is used (see equation in Key Designs 2). An AdamW optimizer is employed with an initial learning rate of \(1 \times 10^{-4}\) and weight decay of 0.01. Linear warmup is applied for the first 5 epochs, followed by cosine annealing. The batch size is 256, trained for 200 epochs using mixed-precision training and gradient clipping.

Second Stage (LLM Fine-Tuning): A weight decay of 0.05 is used. Stage I has a learning rate of \(3 \times 10^{-5}\) and is trained for 1 epoch (70,000 iterations, first 7,000 warmup); Stage II has a learning rate of \(5 \times 10^{-6}\) and is trained for 3 epochs (50,000 iterations per epoch, 5,000 warmup). The LLM backbone uses Qwen3-8B, with the Q-Former and projection head initialized from TinyGPT-V pretrained weights. All experiments are conducted on 4 x 80GB A800 GPUs.

In addition, the Brep2Text dataset is constructed: based on 134,722 B-rep models from the Text2CAD corpus, Qwen-Max is used to automatically generate corresponding questions for the "abstract level" (what it is) and "introductory level" (how to make it) descriptions of each model, resulting in a total of 269,444 high-quality QA pairs. 200 models are strictly reserved as the test set.

Key Experimental Results¶

Main Results¶

3D Object Description (Table 1): BrepLLM comprehensively outperforms the baselines across all evaluation dimensions. The Qwen3-30B score is 61.36 (4.78 points higher than the strongest baseline, MiniGPT-3D), while the Sentence-BERT and SimCSE similarities reach 75.42 and 77.23 respectively (gains of 3.78 and 4.10). Human evaluation shows a precision of 83.62 and the lowest hallucination rate (0.83). Notably, the 8.4B parameter BrepLLM outperforms the 13B PointLLM and ShapeLLM across all metrics.

Model	LLM Params	Qwen3-30B	Sentence-BERT	SimCSE	Correct Attributes	Hallucinated Attributes ↓	Precision
LLaVA-13B	13B	36.73	45.67	47.16	2.16	1.58	63.15
Qwen3-VL-8B	8B	55.48	67.65	68.85	3.97	0.86	78.76
PointLLM-7B	7B	46.81	65.72	66.05	3.32	1.13	74.60
PointLLM-13B	13B	49.65	66.78	67.32	3.46	1.28	72.99
ShapeLLM-7B	7B	48.32	67.14	68.77	3.79	0.96	79.78
ShapeLLM-13B	13B	51.36	68.36	70.12	3.74	1.35	73.47
MiniGPT-3D	2.7B	56.58	71.64	73.13	4.01	1.04	79.40
BrepLLM	8.4B	61.36	75.42	77.23	4.63	0.83	83.62

3D Object Classification (Table 2): Open-ended generative classification capabilities are evaluated using two prompt formats (instruction-based "What is this?" and fill-in-the-blank "This is an object of..."). BrepLLM achieves an average accuracy of 60.0%, outperforming the strongest baseline MiniGPT-3D by 3.75 percentage points. Furthermore, its performance is highly consistent across both prompt formats (with only a 1.0 percentage point difference), demonstrating the model's robustness to prompt variations.

Model	LLM Params	Input Modality	I (%)	C (%)	Average
LLaVA-13B	13B	Single-view image	46.5	44.5	45.5
Qwen3-VL-8B	8B	Three-view images	54.0	55.0	54.5
PointLLM-7B	7B	3Dpoint cloud	52.5	51.0	51.75
PointLLM-13B	13B	3Dpoint cloud	53.0	51.5	52.25
ShapeLLM-7B	7B	3Dpoint cloud	53.5	52.5	53.0
ShapeLLM-13B	13B	3Dpoint cloud	54.5	53.0	53.75
MiniGPT-3D	2.7B	3Dpoint cloud	56.0	56.5	56.25
BrepLLM	8.4B	B-rep	60.5	59.5	60.0

Ablation Study¶

Ablation Item	Config	Stage I (%)	Stage I+II (%)	Δ (I+II)
Full model	Full configuration	43.83	61.36	—
Adaptive UV sampling	Remove adaptive sampling, replace with uniform sampling	40.59	59.21	-2.15
Hierarchical BrepEncoder	Remove hierarchy, replace with flat encoding	39.64	57.39	-3.97
Training Strategy	Stage I only (no LLM fine-tuning)	43.83	—	—
Training Strategy	Stage II only (skipping Stage I)	—	59.12	-2.24

Key Findings¶

Hierarchical BrepEncoder contributes the most: Removing it causes a 3.97% drop in Stage I+II performance, indicating that the face/edge/topology three-layer decoupled representation is key to capturing multi-granularity geometric information of B-rep, and a single flat encoder loses a large amount of topological structure.
Adaptive UV sampling primarily functions in the early stages: It brings a 3.24% improvement in Stage I and maintains a stable 2.15% contribution in Stage I+II, indicating its critical role in perceiving fine-grained geometric structures during early model training, which remains irreplaceable even as semantic understanding capability is enhanced through training.
Two-stage progressive training is indispensable: Skipping Stage I to directly perform Stage II results in a 2.24% drop, indicating that the 2D-VLM alignment prior provides critical semantic anchors for subsequent 3D-language alignment. Only training Stage I (43.83%) is far from sufficient, demonstrating that 3D-language alignment is the primary driver of performance.
Native B-rep understanding > point cloud/image proxies: Under the same parameter scale, BrepLLM significantly outperforms all point cloud- and image-based baselines, validating that directly operating on B-rep entities (faces, edges, and topology) is far more effective than reducing CAD dimensionality to point clouds or rendered images for proxy reasoning.
Qualitative Results: Faced with complex geometries, BrepLLM accurately identifies fine-grained details such as beveled side edges, internal triangular cutouts, a rounded rectangular base, and the spatial composition of cylinders of varying sizes at both ends, while the baseline models' descriptions are relatively vague or omit key geometric features.

Highlights & Insights¶

A two-stage cross-modal bridging of "align globally first, then fine-tune decoding": The first stage utilizes a CLIP contrastive loss to align overall B-rep representations with text, while the second stage employs a Q-Former to progressively align fine-grained node tokens. This strategy of "global anchoring + local injection" avoids training collapse caused by an excessive modality gap when directly training a geometric encoder and an LLM end-to-end. This can be extended to align other non-natural language modalities (e.g., molecular graphs, circuit diagrams) with LLMs.
Navigating the "sea" of 3D geometry with the "ship" of 2D-VLMs: Stage I freezes the Q-Former and only trains the MLP projection, essentially reusing the "visual token → language" mapping capability learned by the Q-Former on large-scale image-text data, treating B-rep features as a type of "unfamiliar visual token." This approach of leveraging the priors of pretrained interfaces is more stable and converges faster than training a cross-modal bridging layer from scratch, representing a low-cost and highly efficient strategy when introducing entirely new modalities to LLMs.
Area/length-driven adaptive sampling represents a practical tradeoff between "expressive power and computational complexity": Unlike wasteful uniform sampling or complex curvature-adaptive sampling (which requires calculating second derivatives), area/length interpolation incurs nearly zero computational cost while ensuring that both large and small faces receive sampling densities matching their geometric information content. This approach can be directly transferred to other 3D deep learning tasks that require continuous geometry discretization.

Limitations & Future Work¶

Currently only supports object-level B-reps, being unable to handle assemblies (multiple parts + mating constraints), which are the objects with actual reasoning value in industrial scenarios. The authors do not discuss how to extend the face-edge graph to multi-part graphs.
Descriptions in the Brep2Text dataset are automatically generated by Qwen-Max, making the quality limited by the generative model itself, with potential semantic drift or deviations from real engineering semantics. There is a lack of manually annotated engineering-grade ground truth (such as GD&T tolerance information, material properties, and manufacturing process constraints).
The coverage of downstream tasks is narrow: Currently, only description and classification tasks have been evaluated, without involving more engineering-valuable tasks such as B-rep editing, feature recognition (e.g., recognizing manufacturing features like chamfers/fillets/ribs), B-rep generation, and design intent reasoning.
The barrier of computational resources is high: The two-stage training (200 epochs of pretraining + 4 epochs of fine-tuning) is completed on 4 A800 GPUs, presenting a certain barrier for academic replication.

vs CAD-MLLM / CadVLM / CAD-LLM: These works cast CAD modeling as a generation problem of command or sketch sequences, operating on procedural spaces rather than the B-rep entities themselves. The key difference of BrepLLM lies in directly modeling the faces, edges, and topology of B-reps, independent of scarce modeling history data, thereby enabling true geometric reasoning instead of sequence pattern matching.
vs PointLLM / ShapeLLM / MiniGPT-3D: These 3D MLLMs take point clouds as input, which inevitably loses the continuity and topological constraints of parametric surfaces. BrepLLM demonstrates sustained and significant performance gains, indicating that point clouds serve as an information bottleneck when used as proxy representations for CAD models—native B-rep is a better choice for downstream tasks requiring precise geometric understanding.
vs BrepGen / SolidGen / AutoBrep: These works focus on B-rep generation and reconstruction using diffusion or autoregressive models. The hierarchical encoder and cross-modal alignment concepts of BrepLLM can be conversely applied to generative tasks—for instance, using the BrepEncoder as a VAE encoder and the LLM as the backbone of a conditional generator.
Insight: The core methodology of this work—"decoupling structured data (B-rep) into hierarchical token sequences + leveraging 2D priors for progressive alignment"—can be generalized to other structured modalities (program dependency graphs, knowledge graphs, molecular structures) linked to LLMs, reducing the labeling and computational costs of all-modal training.

Rating¶

Novelty: ⭐⭐⭐⭐ Enables the LLM to directly understand B-rep instead of through proxy formats for the first time; the problem definition is clean and possesses practical industrial value; concrete technical components (adaptive UV sampling, hierarchical encoder, two-stage progressive alignment) are not revolutionary innovations, but their combined design is highly rational.
Experimental Thoroughness: ⭐⭐⭐⭐ Covers description and classification tasks with multiple baselines and evaluation protocols (LLM as a judge + traditional metrics + human evaluation), and the ablation studies cover three core components and one training strategy with detailed qualitative comparisons; lacks scaling experiments on larger LLMs (8B+ variants).
Writing Quality: ⭐⭐⭐⭐ Overall clear, with excellent correspondence between figures and text in the methodology section, and the data generation pipeline and evaluation prompts in the appendix are transparent and reproducible; minor typesetting issues exist in some equations (loss of subscript/superscript formatting when LaTeX is converted to plain text).
Value: ⭐⭐⭐⭐ Opens up a new path for applying LLMs in industrial CAD scenarios, with the Brep2Text dataset potentially serving as a standardized benchmark in this direction; the strategies of hierarchical encoding and progressive alignment offer methodological reference value to the 3D MLLM community.