Pointer-CAD v2: Plan-Then-Construct CAD Generation with Dimension-Aware Parametric Precision¶

Conference: ECCV 2026
arXiv: 2606.29301
Code: https://github.com/Snitro/Pointer-CAD-v2
Area: 3D Vision
Keywords: CAD Generation, Parametric Modeling, Plan-Then-Construct, Pointer Mechanism, Geometric Precision

TL;DR¶

Pointer-CAD v2 proposes the Plan-Then-Construct framework, which decouples CAD generation into two stages: "first plan dimension parameters, then construct geometry using point-based parameter retrieval". This thoroughly eliminates the precision loss caused by quantitative parameter discretization in traditional execution sequence methods, significantly outperforming all baselines on vertex, edge, and face-level geometric precision metrics.

Background & Motivation¶

Background: The mainstream paradigms of CAD generation are categorized into two types: command sequence representations and code representations. Code representations directly generate CADQuery or FreeCAD API scripts, which naturally support precise values but suffer from low token efficiency (approximately four times longer than command sequences). Command sequence representations utilize dedicated tokens to describe parametric operations, achieving high token efficiency; however, continuous geometric quantities must be quantized into a finite vocabulary, leading to precision loss. Although Pointer-CAD extends the expressive capability of execution sequences through pointer mechanisms by supporting references to intermediate geometric entities, it still fails to resolve the exact dimension control problem.

Limitations of Prior Work: Command sequence methods discretize continuous parameters such as length and angle into tokens, introducing quantization errors. On the surface, these tiny discrepancies are almost imperceptible under shape-level metrics (e.g., Chamfer Distance, IoU). However, in industrial manufacturing, millimeter-level dimensional deviations are sufficient to cause assembly failures of parts. Even worse, existing evaluation metrics (CD, IoU) are insensitive to these parameter errors, causing the research community to overlook the dimension precision problem for a long time.

Key Challenge: In command sequence methods, parameters and geometric operations share the same token space, forcing continuous values to be quantized. This represents a structural conflict that cannot be resolved by hyperparameter tuning. While code representations can express exact values, the token overhead is prohibitive. Thus, a fundamental trade-off exists between token efficiency and parameter precision.

Goal: To enable command sequence models to express continuous, precise, and unit-aware geometric parameters without sacrificing token efficiency. Specifically, this is decomposed into three sub-problems: (1) how to explicitly express dimensional parameters in the text domain; (2) how to precisely reference these parameters during the generation of command sequences; and (3) how to evaluate the parametric fidelity of generative models rather than relying solely on shape similarity.

Key Insight: The authors observe that parameter reasoning and geometric construction are two tasks of fundamentally different natures. The former requires numerical reasoning with units in the textual space, while the latter requires entity selection and operation ordering in the geometric space. Forcing both into the same token prediction task is the root cause of precision loss. If the model can first "think through" all parameters in the text domain and then "follow the blueprint" in the geometric domain, both numerical precision and token efficiency can be preserved simultaneously.

Core Idea: Plan-Then-Construct. In each modeling step, the model first generates a structured text "design plan" containing complete dimensional parameters. Then, the parameters in the plan are extracted into a dictionary. During the construction stage, precise values are retrieved from the dictionary via a pointer mechanism and filled into the command sequences, thereby entirely bypassing parameter quantization.

Method¶

Overall Architecture¶

The core problem addressed by Pointer-CAD v2 is the precision loss caused by forcing continuous geometric parameters to be quantized into discrete tokens in command-sequence-based CAD generation. The overall approach is to decouple parameter reasoning from geometric construction. The method inherits the step-by-step modeling paradigm of Pointer-CAD, dividing each operation step (sketch-extrude / chamfer / fillet) into two stages: the Plan stage generates a structured design plan with dimensional annotations in the text domain; the Construction stage extracts a parameter dictionary from the plan, retrieves precise numerical values via pointers to populate the command sequences, and updates the B-rep after execution before proceeding to the next step.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Text Prompt + Current B-rep"] --> B["Plan Stage<br/>Generate Dimension-Aware Design Plan"]
    B --> C["Parameter Extraction & Encoding<br/>Length/Angle Dictionary + Log Normalization + Frequency Encoding"]
    C --> D["Construction Stage<br/>Retrieve Parameters via Pointers to Generate Command Sequence"]
    D --> E["Execute Command to Update B-rep"]
    E -->|"Not finished, next step"| A
    E -->|"All steps completed"| F["Output CAD Model"]

Key Designs¶

1. Dimension-Aware Design Planning: Explicitly Expressing Continuous Parameters with Units in the Text Domain

In traditional command sequence methods, a length of 5.3mm is quantized to token ID #4237, which maps back to 5.0mm during decoding—precision is lost during the step where "continuous values are converted into finite classes". The Plan stage of Pointer-CAD v2 does not predict tokens; instead, it generates a structured 'design plan' in the text domain. In this plan, each referencable parameter is wrapped in <> to explicitly annotate its type (L for length, A for angle) and physical units (e.g., mm, degrees, meters), and arithmetic references between parameters of the same type are supported (e.g., <L2> = <L1> × 2). This plan is a sequence of text tokens and does not involve any quantization, allowing each parameter to retain its original continuous value.

2. Pointer-Based Parameter Retrieval: Three Steps of Extraction, Encoding, and Similarity Matching

After generating the plan in the Plan stage, all <L> and <A> parameters are extracted into a length dictionary and an angle dictionary. The subsequent challenge is how to enable the model to 'point back' to these precise values when generating command sequences. The authors design a parameter encoding and retrieval pipeline: (1) Parameter extraction and normalization: Extracted parameters are uniformly converted to standard units (length $\rightarrow$ meters, angle $\rightarrow$ degrees), and a sign-preserving log normalization is applied: $$ \tilde{x}=\frac{\text{sign}(x)\cdot\log(1+|x|)}{\max_{x'\in\mathcal{P}}|\text{sign}(x')\cdot\log(1+|x'|)|+\epsilon} $$, mitigating scale imbalance among parameters of different magnitudes; (2) Frequency encoding: Fourier feature mapping is applied to the normalized value $ \tilde{x} $ using exponentially growing frequency bands $ \mathbf{b}=[2^0,2^1,\dots,2^{K-1}] $: $ \phi(\tilde{x})=[\sin(\tilde{x}\cdot s\cdot\mathbf{b}),\cos(\tilde{x}\cdot s\cdot\mathbf{b})] $, enhancing the expressive capability across different numerical ranges; (3) Gated MLP embedding: The original normalized value $ \tilde{x} $ and the frequency features $ \phi(\tilde{x}) $ are projected into a shared embedding space using gated MLPs $ \mathcal{H}(\mathbf{u})=\text{LayerNorm}(\sigma(W_g\mathbf{u})\odot\text{MLP}(\mathbf{u})) $, and then averaged as $ \mathbf{e}=\frac{1}{2}(\mathcal{H}(\tilde{x})+\mathcal{H}(\phi(\tilde{x}))) $. RoPE is then added to encode parameter identity and positional information. During command sequence construction, whenever a numerical parameter is needed, the model predicts a query embedding $ \tilde{\mathbf{e}} $ and performs cosine similarity matching with all candidates in the parameter dictionary: $ \text{similarity}(\tilde{\mathbf{e}},\mathbf{e}_i)=\frac{\tilde{\mathbf{e}}\cdot\mathbf{e}_i}{|\tilde{\mathbf{e}}|\cdot|\mathbf{e}_i|} $. The highest scorer is directly inserted into the command sequence. Throughout this process, parameter values consistently exist in continuous forms, completely eliminating quantization errors.

3. Three-Level Hierarchical Geometric Precision Metrics: From "Shape Similarity" to "Parameter Correctness"

Existing metrics (Chamfer Distance, IoU) measure normalized shape similarity and are almost insensitive to millimeter-level dimensional discrepancies. The authors define a set of coarse-to-fine hierarchical geometric precision metrics, where all metrics share the Accuracy Ratio formula $ Acc=\frac{N_{match}}{N_{gt}}\times 100 $: (1) Vertex precision: A predicted vertex is considered matched if its Euclidean distance to the nearest target point is within the tolerance $ \epsilon=0.001\times\min(\mathbf{b}{gt}^{max}-\mathbf{b}) $; (2) }^{minEdge precision: Requires not only matching endpoints but also matching underlying geometric parameters (such as the center and radius of an arc, and the normal of a circle); (3) Face precision: Requires all constituent edges to match and the face type (plane/cylinder/cone/sphere/torus) to be consistent. Additionally, RMR@3 (Repairable Model Rate) is introduced: models with no more than 3 incorrect faces are considered repairable, measuring practical engineering usability. All evaluations are conducted under the original model dimensions and positions without normalization, forcing the models to demonstrate true scale understanding capabilities.

A Complete Example¶

Suppose the text description requires generating a "square base with a side length of 50mm, extruded upward by 30mm". In the Plan stage, the model generates the plan: "(1) Draw a square sketch on the XY plane with side length <L1>=50mm; (2) Extrude along the Z+ direction by a distance <L2>=30mm." Parameters are extracted into the length dictionary {L1: 0.05, L2: 0.03} (uniformly converted to meters), and then log-normalized and frequency-encoded to obtain embedding vectors. In the Construction stage, the model sequentially generates <ss> (start sketch) $\rightarrow$ four line commands (endpoint coordinates retrieved from existing B-rep vertices via pointers) $\rightarrow$ <se> (start extrude) $\rightarrow$ <lv> (predict query embedding, matching L2 with the highest cosine similarity, selecting 0.03m) $\rightarrow$ <em> (end). The retrieval of both parameter values is a continuous exact match, no longer suffering from the "token #4237 decoded as 5.0 instead of 5.3" issue.

Loss & Training¶

The training objective combines four loss terms: $ \mathcal{L}=\lambda_t\cdot\mathcal{L}_t+\lambda_l\cdot\mathcal{L}_l+\lambda_v\cdot\mathcal{L}_v+\lambda_p\cdot\mathcal{L}_p $, where $ \mathcal{L}_t $ and $ \mathcal{L}_l $ represent the cross-entropy losses for the Plan tokens and Label tokens, respectively, while $ \mathcal{L}_v $ and $ \mathcal{L}_p $ denote the contrastive losses (cosine similarity + softmax) for the value tokens and pointer tokens. The hyperparameters are set to $ \lambda_t=0.2,\lambda_l=0.3,\lambda_v=0.2,\lambda_p=0.3 $, where label and pointer losses are assigned higher weights because they are critical to command generation accuracy. The decoding side uses three independent heads: the Label Head to predict label tokens (including <lv>/<av> to indicate the current value is a length/angle), the Value Head to predict a 128-dimensional query embedding for parameter retrieval, and the Pointer Head to predict a 128-dimensional query embedding for B-rep geometric entity retrieval. The backbone LLM is Qwen2.5-0.5B, trained on H800 GPUs for 10 epochs.

Key Experimental Results¶

Main Results¶

In the table below (OmniCAD-Plan dataset, featuring 202K models), Pointer-CAD v2 significantly outperforms CADmium and Pointer-CAD across all three geometric precision levels and RMR@3. The 1.5B version improves the average accuracy by 13.49 percentage points compared to CADmium-1.5B.

Method	RMR@3	Vertex Precision	Edge Precision	Face Precision
CADmium-0.5B	78.60	76.37	72.41	66.56
Pointer-CAD-0.5B	66.03	58.87	59.18	55.88
Ours-0.5B	87.29	90.10	92.21	89.05
CADmium-1.5B	81.15	79.36	74.03	67.99
Pointer-CAD-1.5B	72.10	64.67	62.40	59.50
Ours-1.5B	91.25	92.76	94.23	91.14

On the more complex OmniCAD-Plan+ dataset (which includes chamfer/fillet, 209K models), the 0.5B model elevates face precision from CADmium's 26.06% to 84.16%, and the 1.5B model achieves an RMR@3 of 90.97%, illustrating the robust handling of complex operations by the proposed framework. Compared with general LLMs (GPT-5.2, Gemini 3 Pro, Claude Opus 4.5, Qwen3-235B), Ours-1.5B outperforms the strongest general model, Gemini 3 Pro, by 21.30 percentage points in average three-level precision. In terms of traditional metrics (F1, CD), the F1 scores for Line and Circle are already close to saturation. Ours improves the Arc F1 from Pointer-CAD's 51.00 to 63.59, while CD shows marginal improvement—which precisely demonstrates that traditional metrics are insensitive to parametric precision, underscoring the necessity of the newly proposed metrics.

Ablation Study¶

Variant	RMR@3	Vertex Precision	Edge Precision	Face Precision	Description
Full model	87.29	90.10	92.21	89.05	Full model
w/o Ref.	65.55	66.30	71.14	66.30	Remove pointer retrieval, use direct regression + nearest value matching instead, decreases by 21.74
w/o Plan	80.93	76.13	75.50	71.38	Remove the Plan stage, extract parameters directly from the raw prompt, decreases by 6.36
w/o Freq.	86.77	89.02	90.67	87.31	Remove frequency encoding, slight drop of 0.52
w/o Norm.	86.76	89.63	91.96	88.51	Remove log normalization, slight drop of 0.48

Key Findings¶

Removing pointer retrieval (w/o Ref.) results in the most severe performance drop (RMR@3 falls from 87.29 to 65.55), indicating that direct regression of numerical values is highly susceptible to interference from adjacent values, and the pointer mechanism is the cornerstone of precision. Removing the Plan stage yields a drop of 6.36 points because the raw prompt often lacks administrative parameters (such as positioning parameters derived from existing geometries) that need to be explicitly reasoned and completed.
The contributions of frequency encoding and log normalization are marginal (<1.2%), suggesting that the pointer retrieval mechanism is relatively insensitive to numerical scales, and can successfully retrieve correct parameters even without fine-grained encoding—a positive signal of robustness.
As the model size scales from 0.5B to 7B, the performance gap between Ours and CADmium continues to widen (from 13.12% to 15.36%), indicating that the Plan-Then-Construct framework effectively leverages larger model capacities.
The difference in traditional CD metrics between Ours and Pointer-CAD is very minor (3.15 vs 3.69), which directly reflects the issue that improvements in parameter precision are not captured by shape-level metrics.

Highlights & Insights¶

The decoupling concept of Plan-Then-Construct is clever and generalizable: Separating 'parameter reasoning' from 'geometric construction' into two phases allows the Plan stage to perform numerical reasoning in the text domain while the Construction stage handles entity operations in the geometric space, bridged by pointers. This 'think through the parameters first, then construct accordingly' pattern not only solves the quantization problem but also renders plans natively editable—modifying parameter values in the plan directly yields the modified CAD model.
Using pointers for parameter retrieval instead of regression: Regressing continuous values is prone to drift, but encoding parameter values as embeddings and allowing the model to retrieve the most similar one from candidate sets transforms the regression problem into a retrieval task, significantly enhancing stability. This is validated by the 21.74-point drop in the w/o Ref. ablation variant.
Parameter representation pipeline using log normalization + frequency encoding: This is a tech transfer from the NeRF community—applying the Fourier Feature technique, typically used for handling continuous coordinates, to CAD parameter values spanning several orders of magnitude (from micrometers to meters) is an excellent example of cross-domain adaptation.
RMR metrics align closely with engineering reality: Instead of imposing overly strict metrics that demand 'flawlessness', RMR@3 measures 'whether post-processing can salvage the model'. This is highly likely to be adopted by industry over purely academic metrics.

Limitations & Future Work¶

Plan quality is constrained by the LLMs: Plan annotations rely on automatic generation by Qwen3-32B, which exhibits a high failure rate for complex sketch-extrude pairs (the paper indicates a 22.08% reduction in such operations in OmniCAD-Plan+ compared to the source data), suggesting that the current framework may lack coverage of complex sketches. If the Planner itself fails in complex scenarios, no amount of precision in the construction phase can salvage the output.
Step-by-step generation efficiency: Inheriting the step-by-step paradigm of Pointer-CAD, each step requires both Plan and Construct phases, leading to inference latencies for long-sequence models (multi-step CAD) that are several times that of code-based methods. The paper does not discuss inference speed.
End-to-end training vs. independent modules: In the current framework, Plan and Construction share the backbone but generate sequentially within a single forward pass; errors in the Plan stage propagate to the Construction stage. The paper does not explore error-recovery mechanisms between the Plan and Construction phases (e.g., verifying/correcting the Plan before entering Construction).
Restricted operation types: Currently, only three operations (sketch-extrude, chamfer, fillet) are supported. More general CAD operations such as loft, sweep, and Boolean operations have not yet been covered, leaving a gap before fully matching engineering CAD workflows.

vs Pointer-CAD: Pointer-CAD introduced the pointer mechanism for referencing existing geometric entities (edges/faces). Pointer-CAD v2 expands its usage to dual purposes: referencing both geometric entities and numerical parameters. Where v1's pointers resolved 'which face to fillet', v2's pointers additionally resolve 'what the fillet radius is'. The relationship between the two is inheritance and extension, with the planning stage in v2 being entirely novel.
vs CADmium / Code representation methods: Code-based methods naturally support precise parameters (API arguments are floats) but suffer from high token consumption and are limited by the LLM's code generation capability. Pointer-CAD v2 achieves equivalent precision to code-based methods within command sequences while maintaining the advantage of token efficiency, essentially combining the best of both worlds.
vs Direct CAD code generation by general LLMs: Even when prompted to generate CADQuery code, general LLMs (such as GPT-5.2, Gemini 3 Pro) perform significantly worse in geometric precision than the specially trained 0.5B model, with an RMR@3 gap exceeding 13 points. This indicates that CAD modeling requires specialized representations and training paradigms, and the code generation capabilities of general LLMs are insufficient to bridge the domain gap.

Rating¶

Novelty: ⭐⭐⭐⭐ The idea of decoupling parameter reasoning from geometric construction is novel, and the Plan-Then-Construct paradigm opens up a new direction for command sequence methods. However, the core mechanism (pointer retrieval of parameters) is an extension of Pointer-CAD rather than an entirely ground-up innovation.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Extremely comprehensive evaluation featuring dual datasets, multiple model parameters (0.5B-7B), diverse baselines (command sequences, code, general LLMs), multiple metrics (newly proposed + traditional), ablation studies, tolerance analysis, and qualitative comparisons.
Writing Quality: ⭐⭐⭐⭐ Clear structure, well-justified motivation (approached from industrial scenarios), and high-quality figures and tables. However, some technical details (e.g., the specific prompt formats for Plan generation, retry logic for parameter validation) require referring to the supplementary material to be fully understood.
Value: ⭐⭐⭐⭐ Solves the long-overlooked precision problem in command sequence CAD generation, and the proposed hierarchical evaluation metrics are highly likely to become new standards in the community. The Plan-Then-Construct framework also holds potential to be adapted to other sequence modeling tasks requiring precise parameter generation.