Pointer-CAD v2: Plan-Then-Construct CAD Generation with Dimension-Aware Parametric Precision¶

Conference: ECCV 2026
arXiv: 2606.29301
Code: https://github.com/Snitro/Pointer-CAD-v2
Area: 3D Vision
Keywords: CAD Generation, Parametric Modeling, Plan-Then-Construct, Pointer Mechanism, Geometric Precision

TL;DR¶

Pointer-CAD v2 proposes the Plan-Then-Construct framework, which decouples CAD generation into two stages: "first plan dimension parameters, then construct geometry by retrieving parameters via pointers." This completely eliminates the precision loss caused by numerical parameter quantization in traditional command sequence methods, significantly outperforming all baselines across vertex, edge, and face-level geometric precision metrics.

Background & Motivation¶

Background: Mainstream CAD generation paradigms are categorized into command sequences and code representations. Code representations directly generate CADQuery or FreeCAD API scripts, naturally supporting precise numerical values but suffering from low token efficiency (~4x longer than command sequences). Command sequences describe parametric operations using dedicated tokens, ensuring high token efficiency, but continuous geometric quantities must be quantized into a finite vocabulary, leading to precision loss. Pointer-CAD extends the expressive power of command sequences using a pointer mechanism to reference intermediate geometric entities, but it still fails to solve the precise dimension control problem.

Limitations of Prior Work: Command sequence methods discretize continuous parameters like lengths and angles into tokens, introducing quantization errors. On the surface, these tiny discrepancies are almost invisible in terms of shape-level metrics (Chamfer Distance, IoU). However, in industrial manufacturing, millimeter-level dimensional deviations are sufficient to cause part assembly failure. Worse, existing evaluation metrics (CD, IoU) are insensitive to such parameter errors, leading the research community to overlook the dimensional precision problem for a long time.

Key Challenge: In command sequence methods, parameters and geometric operations share the same token space, forcing continuous values to be quantized. This is a structural contradiction that cannot be resolved by hyperparameter tuning. Code representations can express precise values, but the token cost is prohibitively high. A fundamental trade-off exists between token efficiency and parameter precision.

Goal: To enable command sequence models to express continuous, precise, and dimensioned geometric parameters without sacrificing token efficiency. Specifically, this is decomposed into three sub-problems: (1) how to explicitly express dimensional parameters in the text domain; (2) how to accurately reference these parameters during command sequence generation; and (3) how to evaluate the parametric fidelity of generative models rather than merely assessing shape similarity.

Key Insight: The authors observe that parameter reasoning and geometric construction are two tasks of different natures. The former requires numerical reasoning with units in the text space, while the latter requires entity selection and operation ordering in the geometric space. Forcing both into the same token prediction task is the fundamental cause of precision loss. If the model can first "think through" all parameters in the text domain and then "follow the plan" in the geometric domain, both numerical precision and token efficiency can be retained.

Core Idea: Plan-Then-Construct. In each modeling step, the model first generates a structured text "design plan" containing complete dimensional parameters. Then, the parameters in the plan are extracted into a dictionary. During the construction stage, precise values are retrieved from the dictionary via a pointer mechanism to populate the command sequence, thereby completely bypassing parameter quantization.

Method¶

Overall Architecture¶

The core problem addressed by Pointer-CAD v2 is the precision loss caused by forcing continuous geometric parameters to be quantized into discrete tokens in command sequence CAD generation. The overall approach is to decouple parameter reasoning from geometric construction. Inheriting the step-by-step modeling paradigm of Pointer-CAD, each operation step (sketch-extrude / chamfer / fillet) is decomposed into two phases: the Plan phase generates a structured design plan with dimension annotations in the text domain; the Construction phase extracts a parameter dictionary from the plan, retrieves precise numerical values via pointers to fill in the command sequence, updates the B-rep after execution, and proceeds to the next step.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Text Prompt + Current B-rep"] --> B["Plan Phase<br/>Generate Dimension-Aware Design Plan"]
    B --> C["Parameter Extraction & Encoding<br/>Length/Angle Dict + Log Normalization + Frequency Encoding"]
    C --> D["Construction Phase<br/>Retrieve Parameters via Pointers to Generate Command Sequence"]
    D --> E["Execute Command to Update B-rep"]
    E -->|"Incomplete, Next Step"| A
    E -->|"All Steps Completed"| F["Output CAD Model"]

Key Designs¶

1. Dimension-Aware Design Plan: Explicitly expressing dimensioned continuous parameters in the text domain

In traditional command sequence methods, a length of 5.3mm is quantized to token ID #4237, which is mapped back to 5.0mm during decoding—precision is lost at the step where "continuous values become finite categories." In the Plan phase of Pointer-CAD v2, instead of predicting tokens, the model generates a structured "design plan" in the text domain. In this plan, each referencable parameter is wrapped in <> with explicit type annotations (L for length, A for angle) and physical units (e.g., mm, degrees, meters), and arithmetic references between parameters of the same type are supported (e.g., <L2> = <L1> × 2). This plan is a sequence of text tokens and does not involve any quantization, preserving the original continuous value of each parameter.

2. Based on Pointer-Based Parameter Retrieval: Extraction, encoding, and similarity matching

After the Plan phase generates the plan, all <L> and <A> parameters are extracted into length and angle dictionaries. The next issue is how to enable the model to "point back" to these precise values during command sequence generation. The authors design a parameter encoding and retrieval pipeline: (1) Parameter extraction and normalization: Extracted parameters are uniformly converted to standard units (length $\rightarrow$ meters, angle $\rightarrow$ degrees), and a sign-preserving log-normalization is applied: $ \tilde{x}=\frac{\text{sign}(x)\cdot\log(1+|x|)}{\max_{x'\in\mathcal{P}}|\text{sign}(x')\cdot\log(1+|x'|)|+\epsilon} $ to alleviate scale imbalance among parameters of different magnitudes; (2) Frequency encoding: Fourier feature mapping is applied to the normalized value $ \tilde{x} $ using exponentially growing frequency bands $ \mathbf{b}=[2^0,2^1,\dots,2^{K-1}] $: $ \phi(\tilde{x})=[\sin(\tilde{x}\cdot s\cdot\mathbf{b}),\cos(\tilde{x}\cdot s\cdot\mathbf{b})] $ to enhance the expressive capability across different numerical ranges; (3) Gated MLP embedding: The raw normalized value $ \tilde{x} $ and frequency feature $ \phi(\tilde{x}) $ are separately projected to a shared embedding space using a gated MLP: $ \mathcal{H}(\mathbf{u})=\text{LayerNorm}(\sigma(W_g\mathbf{u})\odot\text{MLP}(\mathbf{u})) $ then averaged to obtain $ \mathbf{e}=\frac{1}{2}(\mathcal{H}(\tilde{x})+\mathcal{H}(\phi(\tilde{x}))) $, followed by RoPE encoding to incorporate parameter identity and positional information. During command sequence construction, whenever a numerical parameter is needed, the model predicts a query embedding $ \tilde{\mathbf{e}} $, performs cosine similarity matching with all candidates in the parameter dictionary: $ \text{similarity}(\tilde{\mathbf{e}},\mathbf{e}_i)=\frac{\tilde{\mathbf{e}}\cdot\mathbf{e}_i}{|\tilde{\mathbf{e}}|\cdot|\mathbf{e}_i|} $, and directly inserts the candidate with the highest score into the command sequence. Throughout this process, parameter values always exist in continuous form, completely eliminating quantization errors.

3. Three-Level Hierarchical Geometric Precision Metrics: From "shape similarity" to "parameter correctness"

Existing metrics (Chamfer Distance, IoU) measure shape similarity after normalization, which is virtually insensitive to millimeter-level dimensional discrepancies. The authors define a set of hierarchical geometric precision metrics from coarse to fine, all sharing the Accuracy Ratio formula: $ Acc=\frac{N_{match}}{N_{gt}}\times 100 $: (1) Vertex Precision: A predicted vertex matches a GT vertex if their Euclidean distance is within a tolerance of $ \epsilon=0.001\times\min(\mathbf{b}{gt}^{max}-\mathbf{b}) $; (2) }^{minEdge Precision: Requires not only endpoint matching but also alignment of the underlying geometric parameters (e.g., center and radius of an arc, normal of a circle); (3) Face Precision: Requires all constituent edges to match and the face types (plane / cylinder / cone / sphere / torus) to be identical. Additionally, RMR@3 (Repairable Model Rate) is introduced, where models with at most 3 erroneous faces are considered repairable, measuring practical engineering usability. All evaluations are conducted under the original scale and pose of the models without normalization, forcing the model to demonstrate true scale understanding capability.

A Complete Example¶

Suppose the text prompt requires generating a "square base with a side length of 50mm, extruded upward by 30mm." In the Plan phase, the model generates the plan: "(1) Draw a square sketch on the XY plane with side length <L1>=50mm; (2) Extrude along the Z+ direction by a distance of <L2>=30mm." Parameters are extracted into the length dictionary {L1: 0.05, L2: 0.03} (uniformly converted to meters), yielding embedding vectors after log-normalization and frequency encoding. In the Construction phase, the model sequentially generates <ss> (start sketch) $\rightarrow$ four line commands (endpoint coordinates retrieve existing B-rep vertices via pointers) $\rightarrow$ <se> (start extrude) $\rightarrow$ <lv> (predicts a query embedding, which has the highest cosine similarity with the L2 embedding, selecting 0.03m) $\rightarrow$ <em> (end). Both parameter values are retrieved via continuous exact matching, avoiding the issue of "token #4237 decoded as 5.0 instead of 5.3".

Loss & Training¶

The training objective combines four loss terms: $ \mathcal{L}=\lambda_t\cdot\mathcal{L}_t+\lambda_l\cdot\mathcal{L}_l+\lambda_v\cdot\mathcal{L}_v+\lambda_p\cdot\mathcal{L}_p $, where $ \mathcal{L}_t $ and $ \mathcal{L}_l $ are cross-entropy losses for Plan tokens and Label tokens, respectively, and $ \mathcal{L}_v $ and $ \mathcal{L}_p $ are contrastive losses (cosine similarity + softmax) for value tokens and pointer tokens. The hyperparameters are set to $ \lambda_t=0.2,\lambda_l=0.3,\lambda_v=0.2,\lambda_p=0.3 $, where the label and pointer loss weights are higher because they are more critical to command generation accuracy. The decoding side uses three independent heads: a Label Head predicting label tokens (including <lv>/<av> to indicate length/angle values), a Value Head predicting 128-dimensional query embeddings for parameter retrieval, and a Pointer Head predicting 128-dimensional query embeddings for B-rep geometric entity retrieval. The backbone LLM is Qwen2.5-0.5B, trained on an H800 GPU for 10 epochs.

Key Experimental Results¶

Main Results¶

In the table below (OmniCAD-Plan dataset, containing 202K models), Pointer-CAD v2 significantly outperforms CADmium and Pointer-CAD across all three geometric precision levels and RMR@3. The 1.5B version improves average accuracy by 13.49 percentage points compared to CADmium-1.5B.

Method	RMR@3	Vertex Precision	Edge Precision	Face Precision
CADmium-0.5B	78.60	76.37	72.41	66.56
Pointer-CAD-0.5B	66.03	58.87	59.18	55.88
Ours-0.5B	87.29	90.10	92.21	89.05
CADmium-1.5B	81.15	79.36	74.03	67.99
Pointer-CAD-1.5B	72.10	64.67	62.40	59.50
Ours-1.5B	91.25	92.76	94.23	91.14

On the more complex OmniCAD-Plan+ (containing chamfer/fillet, 209K models), the 0.5B model improves face precision from CADmium's 26.06% to 84.16%, and the 1.5B model's RMR@3 reaches 90.97%, proving the strong robustness of the framework for complex operations. Compared with general LLMs (GPT-5.2, Gemini 3 Pro, Claude Opus 4.5, Qwen3-235B), Ours-1.5B outperforms the strongest general model, Gemini 3 Pro, by 21.30 percentage points in average three-level precision. On traditional metrics (F1, CD), Line and Circle F1 scores are nearly saturated, but Ours improves Arc F1 from Pointer-CAD's 51.00 to 63.59. The CD improvement is marginal—this precisely confirms that traditional metrics are insensitive to parameter errors, highlighting the necessity of new metrics.

Ablation Study¶

Variant	RMR@3	Vertex Precision	Edge Precision	Face Precision	Description
Full model	87.29	90.10	92.21	89.05	Full model
w/o Ref.	65.55	66.30	71.14	66.30	Removing pointer retrieval, using direct regression + nearest value matching instead; drops by 21.74
w/o Plan	80.93	76.13	75.50	71.38	Removing Plan phase, extracting parameters directly from the raw prompt; drops by 6.36
w/o Freq.	86.77	89.02	90.67	87.31	Removing frequency encoding; slight drop of 0.52
w/o Norm.	86.76	89.63	91.96	88.51	Removing log-normalization; slight drop of 0.48

Key Findings¶

Removing pointer retrieval (w/o Ref.) results in the most severe performance drop (RMR@3 drops from 87.29 to 65.55), indicating that direct numerical regression is highly susceptible to interference from neighboring values, and the pointer mechanism is core to guaranteeing precision. Removing the Plan phase drops performance by 6.36 points, as raw prompts often do not contain all construction parameters (e.g., positioning parameters inferred from existing geometry), making explicit reasoning and completion necessary.
The contributions of frequency encoding and log-normalization are minimal (<1.2%), suggesting that the pointer retrieval mechanism has low sensitivity to numerical scales and can retrieve correct parameters effectively even without elaborate encoding—representing a positive robustness signal.
As the model scales from 0.5B to 7B, the performance gap between Ours and CADmium continues to widen (from 13.12% to 15.36%), illustrating that the Plan-Then-Construct framework can effectively leverage larger model capacities.
The difference in traditional CD metrics between Ours and Pointer-CAD is very narrow (3.15 vs. 3.69), which directly reflects how parameter precision improvements fail to be captured by shape-level metrics.

Highlights & Insights¶

The Plan-Then-Construct decoupling paradigm is elegant and generalizable: Separating "parameter reasoning" from "geometric construction" into two phases, where the Plan phase performs numerical reasoning in the text domain and the Construction phase handles entity operations in the geometric domain, bridged by pointers. This "think through parameters first, then construct accordingly" pattern not only solves the quantization issue but also makes plans naturally editable—updating parameter values in the plan directly yields the modified CAD model.
Using pointers for parameter retrieval instead of regression: Regression-based prediction of continuous values is prone to drift. However, encoding parameter values as embeddings and letting the model retrieve the most similar one from candidate sets converts regression into retrieval, significantly boosting stability. This is validated by the 21.74-point drop in the "w/o Ref." variant.
Log-normalization + frequency encoding parameter representation pipeline: This is a neat technology transfer from the NeRF community—applying Fourier Feature tricks used for continuous coordinates to handle CAD parameter values spanning several orders of magnitude (from micrometers to meters) serves as an excellent cross-domain inspiration.
RMR metric closely aligns with engineering realities: Instead of providing overly rigid "perfect matching" metrics, using RMR@3 to evaluate "whether post-processing can salvage it" is much more likely to be adopted by the industry than pure academic metrics.

Limitations & Future Work¶

Plan quality is constrained by the LLM: Plan annotations rely on auto-generation by Qwen3-32B. The failure rate of Plan generation is relatively high for complex sketch-extrude pairs (the paper indicates a 22.08% reduction in such operations in OmniCAD-Plan+ compared to raw data), meaning current frameworks may have insufficient coverage for complex sketches. If the Planner itself fails in complex scenes, no amount of precision in the construction phase can save it.
Step-by-step generation efficiency: Following the sequential paradigm of Pointer-CAD, each step requires both Plan and Construct phases, making the inference latency of long-sequence models (multi-step CAD) several times that of code-based methods. The paper does not discuss inference speed.
End-to-end training vs. independent modules: In the current framework, Plan and Construction share the same backbone and are of sequential generation in a single forward pass, so errors in the Plan phase propagate to Construction. The paper does not explore error recovery mechanisms (e.g., verifying/correcting the Plan once generated before entering Construction).
Limited operation types: Only sketch-extrude, chamfer, and fillet operations are currently supported. More general CAD operations such as loft, sweep, and boolean are not yet covered, leaving a gap before achieving a fully covered engineering CAD workflow.

vs Pointer-CAD: Pointer-CAD introduces a pointer mechanism to reference existing geometric entities (edges/faces). Pointer-CAD v2 extends this to dual-use—referencing both geometric entities and numerical parameters. The pointer in v1 answers "which face to fillet," while the pointer in v2 also answers "what the fillet radius is." The relationship is inheritance and extension, with the planning phase in v2 being entirely new.
vs CADmium / Code representation methods: Code-based methods naturally support precise parameters (API parameters are floats) but consume excessive tokens and are bound by LLMs' code generation capabilities. Pointer-CAD v2 achieves equivalent precision to code-based methods within command sequences while preserving token efficiency advantages, essentially combining the best of both worlds.
vs General LLM-based direct CAD code generation: Even with prompt-guided CADQuery code generation, the geometric precision of models like GPT-5.2, Gemini 3 Pro, Claude Opus 4.5, and Qwen3-235B remains significantly weaker than the specialized 0.5B model, with a gap of over 13 points in RMR@3. This implies that CAD modeling requires specialized representations and training paradigms, and the code generation capabilities of general LLMs are insufficient to bridge the domain gap.

Rating¶

Novelty: ⭐⭐⭐⭐ The idea of decoupling parameter reasoning and geometric construction is novel, and the Plan-Then-Construct paradigm opens up a new direction for command sequence methods; the core mechanism (pointer-based parameter retrieval) is an extension of Pointer-CAD, not a ground-up innovation.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Extremely comprehensive: dual datasets, multiple model sizes (0.5B-7B), diverse baselines (command sequences, code, general LLMs), multiple metrics (new vs. traditional), ablation, tolerance analysis, and qualitative comparisons.
Writing Quality: ⭐⭐⭐⭐ Clear structure, well-motivated (approached from industrial scenarios), and high-quality figures/tables; however, some technical details (e.g., the exact prompt format for Plan generation, validation/retry logic for parameters) require reading the supplementary materials to fully grasp.
Value: ⭐⭐⭐⭐ Addresses the long-ignored precision problem in command sequence CAD generation, and the proposed hierarchical evaluation metrics are expected to become new community standards; the Plan-Then-Construct framework has strong potential to migrate to other sequence modeling tasks requiring precise parameter generation.