RIGOR: Atomic Semantic Normalization and Declarative Constraint Reasoning for Geometry Problem Solving¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/dongyuhang22/RIGOR
Area: LLM Reasoning
Keywords: Geometry Problem Solving, Neuro-Symbolic System, Atomic Semantic Normalization, Declarative Constraint Reasoning, SMT Solver
TL;DR¶
Addressing the two foundational bottlenecks of deductive database systems—coupled textual descriptions and eager intermediate variable elimination—RIGOR normalizes complex problem texts into atomic statements via an instruction-tuned GSN and solves non-linear geometric constraints declaratively via SMT query-and-refutation, setting new state-of-the-art results across GeoQA, Geometry3K, and PGPS9K.
Background & Motivation¶
Geometry Problem Solving (GPS) stands as a quintessential challenge in artificial intelligence, demanding seamless multi-modal perception, spatial diagram parsing, and rigorous multi-step deductive reasoning. Existing neuro-symbolic methods grounded in Deductive Databases (DD) decompose GPS into formal parsing and symbolic deduction: a parser maps diagrams and problem texts into formal first-order predicates, after which a symbolic engine applies geometric theorems iteratively to derive intermediate quantities and establish the computational goal. Compared to sequence-generation neural models that directly decode answers or program tokens, this neuro-symbolic paradigm ensures formal logical rigor, eliminates arithmetic hallucinations, and provides verifiable inference steps.
However, existing DD-based frameworks break down when deployed on realistic geometry problems due to two deeply entrenched assumptions. First, on the text parsing front, methods rely on rigid regular-expression templates under the implicit assumption that problem texts consist of isolated atomic statements corresponding directly to individual predicates. In practice, natural geometric problem descriptions routinely compress multiple relations around shared primitives into coupled compound expressions (e.g., "in circle O, chord AD // radius OC"). Parsing such descriptions directly would demand combinatorially many handcrafted pattern templates covering arbitrary linguistic compositions, causing parsers to miss latent geometric premises and shatter downstream reasoning chains. Second, on the symbolic reasoning front, traditional engines adopt an eager elimination paradigm, where newly derived algebraic equations immediately trigger iterative symbolic substitution and elimination of intermediate variables. Because geometric constraints (such as the Pythagorean theorem and trigonometric formulas) are fundamentally non-linear and tightly coupled, eager elimination rapidly blows up algebraic complexity and spawns exponential case branching, frequently causing solver timeouts or premature heuristics pruning.
The pivotal insight of this work is to shift both interfaces: rather than expanding text parsing templates to accommodate arbitrary compound phrases, the text should be normalized into a compact, finite vocabulary of predicate-level atomic statements; and rather than explicitly eliminating intermediate algebraic variables step by step, target derivation should be reformulated as an entailment query over a global constraint system in Nonlinear Real Arithmetic (NRA). Core idea: train a compact Geometric Semantic Normalizer (GSN) on a synthesized 15K dataset to project compound geometric text into atomic statements for reliable rule-based formalization, coupled with a Declarative Symbolic Engine (DSE) that accumulates constraints globally and determines target values through SMT-based query-and-refutation.
Method¶
Overall Architecture¶
RIGOR consists of two primary stages: multi-modal parsing and declarative symbolic reasoning. For input diagrams, visual primitives and topological relations are extracted using PGDPNet complemented by a classical vision fallback pipeline (SAM2 segmentation, Hough transform, and Harris corner alignment). For problem texts, the instruction-tuned GSN decouples natural compound sentences into an ordered sequence of standardized atomic statements, which a rule-based parser deterministically converts into formal logical predicates. The resulting premise set is handed to the Declarative Symbolic Engine (DSE). Within DSE, forward deduction rules iteratively fire over geometric facts and accumulate algebraic constraints until reaching a deductive fixpoint. Known variable values are subsequently propagated to simplify the equations. Finally, an off-the-shelf SMT solver (Z3) extracts a candidate target value from the simplified constraint pool and rigorously proves its uniqueness by refuting alternative assignments, while recursively backtracking over an unsatisfiable core on the dependency graph to synthesize a faithful, step-by-step reasoning trace.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Multi-Modal Input<br/>Geometry Diagram + Problem Text"] --> B["Atomic Semantic Normalization<br/>GSN maps text to predicate-level atomic statements"]
B --> C["Rule-Based Formalization<br/>Deterministic mapping to formal logic predicates"]
C --> D["Declarative Constraint Accumulation<br/>DSE forward chaining until deductive fixpoint S_final"]
D --> E["Value Propagation<br/>Univariate equation reduction to simplify into C_red"]
E --> F["Query-and-Refute Target Derivation<br/>SMT candidate extraction + refutation proof of uniqueness"]
F --> G["Proof-Grounded Trace Reconstruction<br/>UNSAT core backtracking over dependency graph"]
Key Designs¶
1. Atomic Semantic Normalization via GSN-15K: Taming Combinatorial Text Parsing To eliminate the severe text-to-predicate mismatch where compound geometric descriptions break regular-expression rules, the authors formulate text parsing as an atomic projection task into a finite semantic space. Recognizing that local geometric phrases exhibit strong syntactic modularity despite sharing primitives across the problem, the authors construct GSN-15K through a decomposition–recombination strategy: an advanced LLM (DeepSeek-V3) first decomposes raw problem texts from GeoQA into self-contained segments, maps them into atomic statements, and abstracts concrete entities into placeholders to build a segment-level template library. The pipeline then samples template counts and structures matching the empirical distribution of real exam questions, instantiates them with sampled primitives to synthesize 15,000 paired training examples, and applies targeted manual curation. A compact LLaMA-2-7B model is instruction-tuned on GSN-15K using LoRA under the standard autoregressive negative log-likelihood objective: $\(\mathcal{L} = -\sum_{t=1}^{|S|} \log P(s_t \mid s_{<t}, T; \theta)\)$ By normalizing complex compound sentences into atomic clauses (e.g., transforming "in circle O, chord AD // radius OC" into three discrete statements: "AD is a chord of circle O", "C is on circle O", and "AD is parallel to OC"), downstream rule-based parsing attains near-perfect formalization fidelity without requiring combinatorial pattern engineering or risking LLM syntax hallucinations.
2. Declarative Constraint Accumulation and Value Propagation: Circumventing Eager Algebraic Blowup To resolve the algebraic complexity explosion caused by eager elimination in traditional deductive engines, DSE formulates target resolution declaratively over a global constraint system in Nonlinear Real Arithmetic (NRA). During deductive forward chaining, engine rules \(r: A_r \to (B_F^r, B_C^r)\) match existing fact sets \(F_t\) and simultaneously accumulate new geometric facts \(\Delta F\) and algebraic constraints \(\Delta C\) without resolving intermediate variables, iterating until reaching a deductive fixpoint \(S_{\text{final}} = (F_{\text{final}}, C_{\text{final}})\). The constraint set is subsequently augmented with geometric quantity domain constraints \(D_{\text{domain}}\) (e.g., strict non-negativity for line segments). Before dispatching the potentially heavy constraint pool to the SMT solver, DSE executes an efficient value propagation pass: it maintains a set of resolved variables \(V_{\text{known}}\), identifies constraints \(c(x) \in C_{\text{final}}\) that collapse into univariate equations uniquely determining an admissible quantity \(x_u\), updates \(V_{\text{known}} \leftarrow V_{\text{known}} \cup \{x_u\}\), and substitutes known values throughout \(C_{\text{final}}\) until no further reductions are possible. This preprocessing yields a reduced constraint system \(C_{\text{red}}\), cutting penalized solver runtime by 57.5% on non-linear length problems.
3. Query-and-Refute Target Derivation and Proof-Grounded Trace Reconstruction: Verified Uniqueness with Explainability
To guarantee computational uniqueness without sequential symbolic substitution, DSE leverages an SMT solver (Z3) to execute query-and-refute entailment checking. The engine first queries the solver over \(C_{\text{red}}\) to obtain a satisfying model and reads off a candidate value \(v\) for computational target \(g\). To formally verify that \(v\) is the unique mathematically valid solution rather than an arbitrary artifact of an under-determined system, DSE submits the refutation query \(C_{\text{red}} \land (g \neq v)\). If the solver returns UNSAT, by logical duality it rigorously proves that \(C_{\text{red}} \models (g = v)\), verifying \(v\) as the unique entailed target. Furthermore, DSE extracts the unsatisfiable core returned by the SMT solver—identifying the exact subset of proof-relevant constraints \(K \subseteq C_{\text{red}}\) strictly required to entail the target. By mapping \(K\) back onto the deductive Dependency Graph \(G\) maintained during forward chaining and recursively backtracking through antecedent premise nodes, DSE reconstructs an interpretable, mathematically grounded derivation trace with zero step-level hallucinations.
Loss & Training¶
- Training Objective: GSN is fine-tuned via standard autoregressive language modeling on the paired text-to-atomic sequences of GSN-15K using token-level cross-entropy loss.
- Optimization & Hyper-parameters: Built upon LLaMA-2-7B using LoRA with rank \(r=128\) and scaling factor \(\alpha=256\). Optimization utilizes AdamW with a learning rate of \(1 \times 10^{-4}\), batch size of 2, for 3 training epochs with random seed 42.
- Symbolic Reasoning Infrastructure: DSE is implemented on top of Newclid and integrates Z3 (v4.12+) as the backend NRA SMT solver. The per-problem timeout threshold is set to 1200 seconds. Diagram parsing combines PGDPNet with SAM2-based fallback segmentation.
Key Experimental Results¶
Main Results¶
RIGOR is comprehensively evaluated against general-purpose multi-modal LLMs (MLLMs) and state-of-the-art specialist GPS models across both Sequence Generation (SG) and Deductive Database (DD) paradigms on GeoQA, Geometry3K, and PGPS9K.
| Benchmark / Model | Paradigm | Angle (Comp / Choice %) | Length (Comp / Choice %) | Overall (Comp / Choice %) |
|---|---|---|---|---|
| GeoQA | ||||
| GPT-4o | Generalist MLLM | 36.2 / 54.0 | 43.6 / 62.9 | 39.5 / 58.0 |
| NGS | Specialist (SG) | 55.6 / 65.7 | 34.7 / 47.8 | 46.3 / 57.7 |
| Geoformer | Specialist (SG) | 57.3 / 67.1 | 33.2 / 47.8 | 46.6 / 58.5 |
| SCA-GPS | Specialist (SG) | 54.9 / 65.5 | 35.3 / 49.0 | 46.2 / 58.1 |
| DualGeoSolver | Specialist (SG) | 55.2 / 65.7 | 36.5 / 50.4 | 46.8 / 58.9 |
| G-LLaVA (7B) | Specialist (SG) | 55.6 / 70.5 | 30.9 / 55.8 | 44.6 / 63.9 |
| GeoX | Specialist (SG) | 62.8 / 71.5 | 44.2 / 55.8 | 54.5 / 64.5 |
| RIGOR (Ours) | Specialist (Neuro-Symbolic) | 71.0 / 78.7 | 43.9 / 59.3 | 58.9 / 70.0 |
| Geometry3K & PGPS9K | Geometry3K (Comp / Choice) | PGPS9K (Comp / Choice) | Average (Comp / Choice) | |
| GPT-4o | Generalist MLLM | 34.8 / 58.6 | 33.3 / 51.0 | 34.1 / 54.8 |
| InterGPS | Specialist (DD) | 43.4 / 57.5 | - | - |
| GeoDRL | Specialist (DD) | 57.9 / 68.4 | 55.6 / 66.7 | 56.8 / 67.6 |
| GeoX | Specialist (SG) | 58.6 / 72.5 | 52.7 / 63.3 | 55.7 / 67.9 |
| Pi-GPS | Specialist (DD) | 68.2 / 75.4 | 61.4 / 69.8 | 64.8 / 72.6 |
| RIGOR (Ours) | Specialist (Neuro-Symbolic) | 65.0 / 74.9 | 68.8 / 77.6 | 66.9 / 76.3 |
Ablation Study¶
The core architectural innovations—GSN semantic normalization and DSE declarative constraint reasoning—are ablated on the GeoQA dataset, evaluating completion accuracy (%) alongside solver runtime metrics (seconds).
| Ablation Configuration / Variant | Angle Acc. (%) | Length Acc. (%) | Overall Acc. (%) | Impact & Mechanism Analysis |
|---|---|---|---|---|
| Full RIGOR Model | 71.0 | 43.9 | 58.9 | Full system: GSN-15K tuned normalizer + DSE declarative solver |
| GSN Module Ablations | ||||
| w/o GSN (Rule text parser only) | 53.2 | 27.9 | 41.9 | Drops 17.0%; rigid regex misses compound geometric relations |
| w/ GSN (w/o Aug. no data synthesis) | 67.4 | 37.7 | 54.1 | Drops 4.8%; training on un-augmented seed pairs limits syntactic coverage |
| w/ DeepSeek-V3 (direct LLM normalizer) | 70.7 | 37.7 | 56.0 | Drops 2.9%; proprietary LLM outputs exceed strict regex parser grammar |
| DSE Module Ablations | ||||
| w/ ESE (InterGPS-style eager elimination) | 66.2 | 31.8 | 50.8 | Drops 8.1% overall (and 12.1% on length); non-linear algebraic blowup |
| w/ DSE (w/o Prop. no value propagation) | 71.0 | 42.1 | 58.1 | Drops 0.8%; unsimplified constraints trigger SMT timeouts |
| Runtime Pairwise Comparison (GeoQA) | Angle Problems | Length Problems | Overall Problems | Efficiency Insights |
| Mutual Runtime MR (DSE vs ESE) | 14.72s / 11.78s | 11.10s / 80.04s | 13.76s / 30.00s | DSE achieves 7.2× speedup on length problems; ESE faster on linear angles |
| Penalized Runtime PR (DSE vs ESE, 1200s timeout) | 31.12s / 106.81s | 77.89s / 429.51s | 47.12s / 217.21s | DSE requires only 21.7% of ESE runtime, dramatically reducing timeouts |
Key Findings¶
- Atomic semantic normalization is the primary bottleneck of neuro-symbolic GPS: Reverting from GSN to pure rule-based parsing incurs a devastating 17.0% drop in overall accuracy (from 58.9% to 41.9%), illustrating that regex failure on coupled statements has historically starved symbolic engines of essential premises.
- Declarative reasoning decisively overcomes non-linear algebraic explosion: On length problems—where non-linear algebraic relationships predominate—DSE surpasses the eager elimination baseline ESE by 12.1% in accuracy, delivers a 7.2× mutual runtime speedup (11.10s vs. 80.04s), and slashes penalized runtime from 429.51s to 77.89s.
- Domain-specific fine-tuning beats general-purpose frontier LLMs: The 7B GSN outperforms both prompt-driven DeepSeek-V3 normalization (56.0%) and direct end-to-end formal predicate generation (53.3%), demonstrating that constraining generation to a tight predicate-aligned vocabulary is superior to unconstrained linguistic fluency.
- Superior scaling over reasoning complexity: When evaluated against problem difficulty (human-annotated reasoning steps \(RS \ge 3\)), RIGOR demonstrates significantly flatter accuracy degradation compared to sequence generation baselines like GeoX and G-LLaVA, validating the robustness of fixpoint deductive accumulation.
Highlights & Insights¶
- Decoupling Linguistic Flexibility from Formal Precision: Instead of asking an LLM to generate error-prone code or forcing regex engines to handle arbitrary natural language, GSN acts as a smooth buffer: the neural model resolves grammatical complexity into standardized atomic statements, while the rule parser enforces 100% syntactic precision into symbolic predicates.
- Query-and-Refute Entailment via SMT Duality: Reframing mathematical target derivation from imperative variable substitution into declarative SMT entailment (\(C_{\text{red}} \land (g \neq v) \models \text{UNSAT}\)) guarantees mathematical uniqueness while delegating non-linear polynomial algebra entirely to state-of-the-art decision procedures.
- Faithful, Hallucination-Free Trace Synthesis: By combining the SMT solver's UNSAT core with forward-chaining provenance records on the dependency graph, RIGOR reconstructs step-by-step human-readable proofs that are strictly anchored in verified theorem deductions.
Limitations & Future Work¶
- Perceptual Fragility in Visual Parsing: Error analysis indicates that visual parsing failures account for 64.8% of all failure cases across the full RIGOR pipeline, primarily due to missed angle labels or distorted line segments in low-resolution or natural-scene images.
- Unidirectional Neuro-Symbolic Information Flow: The pipeline operates as a feed-forward sequence (text \(\to\) GSN \(\to\) DSE); when DSE detects an under-determined system or an unsatisfiable premise contradiction, it cannot pass diagnostic feedback back to GSN to iteratively re-parse ambiguous text.
- Restricted Expressivity for Advanced Geometry: The deductive rule library currently lacks formal axiomatizations for dynamic moving-point optimization and extremum problems, which account for 13.9% of unsolvable benchmark cases.
Related Work & Insights¶
- vs. Sequence Generation Methods (GeoX, G-LLaVA, NGS): Sequence-to-sequence neural architectures map inputs directly to answers or program traces; while linguistically flexible, they suffer from fatal arithmetic hallucinations and step-level logical invalidity on complex problems. RIGOR guarantees 100% deduction correctness by offloading all reasoning to a verified symbolic engine.
- vs. Eager Elimination DD Engines (InterGPS, Pi-GPS): InterGPS attempts to isolate variables step-by-step during deduction, which frequently encounters exponential algebraic explosion on non-linear polynomial constraints. RIGOR replaces eager substitution with declarative constraint accumulation, yielding an 8.1% accuracy surge and a 78% reduction in runtime.
- vs. Olympiad-Level Theorem Provers (AlphaGeometry, AlphaGeometry2): AlphaGeometry uses an LLM to propose auxiliary geometric constructions for proving Olympiad equality theorems. In contrast, RIGOR tackles the dual challenges of natural language premise coupling and non-linear numerical calculation in standard curricula, providing complementary advances in semantic parsing and SMT-based value solving.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Formulates atomic semantic normalization to solve compound predicate parsing and introduces declarative SMT query-and-refutation to replace eager elimination in GPS.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous validation across three benchmarks, paired with detailed ablation of every component, runtime efficiency distributions, and comprehensive error categorization.
- Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally clear mathematical formulations, crisp architectural exposition, and compelling narrative flow.
- Value: ⭐⭐⭐⭐⭐ Provides an elegant, robust, and extensible neuro-symbolic blueprint for multimodal mathematical problem solving with immediate applicability to intelligent tutoring systems.