Skip to content

CirCA: Aligning Credit Assignment with Circuit Semantics via Atomic Logic Blocks for RTL Generation

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/Taskii-Lei/CirCA
Area: LLM Reasoning
Keywords: Verilog Code Generation, Reinforcement Learning Post-Training, Credit Assignment, Atomic Logic Blocks, Circuit Semantics

TL;DR

Addressing the severe misattribution in RL-based Verilog generation caused by overly coarse sequence-level assignment or fragmented line-level splitting, CirCA introduces Atomic Logic Blocks (ALBs) as semantically complete optimization units, localizes faulty blocks through semantics-aware simulation and fingerprint-guided dependency cone matching, and integrates localized penalties into GRPO via asymmetric advantage correction.

Background & Motivation

Reinforcement learning (RL) post-training has emerged as a premier paradigm for boosting the functional correctness of large language models (LLMs) in automated hardware description language (HDL) generation, such as Verilog. Under frameworks like Group Relative Policy Optimization (GRPO), models learn to refine generation policies via intra-group relative rewards without requiring a dedicated value network. However, hardware code differs fundamentally from traditional sequential software programs: Verilog execution is governed by concurrent constructs and procedural control flows that describe hardware behavior through a set of concurrent local update rules. Each distinct code segment typically implements an isolated logical function and drives circuit outputs through heterogeneous dependency paths.

Under these strict circuit semantics, conventional RL credit assignment strategies face a severe representational mismatch. Sequence-level credit assignment, as conventionally adopted by GRPO, is excessively coarse-grained: it treats the entire generated module as a monolithic optimization unit and uniformly distributes identical advantage values to all output tokens. Consequently, when a generated module exhibits partially correct behavior alongside localized bugs, positive credit belonging to correct structures is blindly shared with faulty code regions. Conversely, line-level credit assignment attempted in prior work proves overly fragmented: a semantically coherent hardware function in Verilog frequently spans multiple physical lines, yet line-level attribution isolates each line into an independent unit, fragmenting credit belonging to an indivisible functional block and causing severe credit misattribution. Furthermore, the search space for line-level fault localization suffers combinatorial explosion with code length and fault density.

To bridge the gap between circuit semantics and RL credit assignment, optimization must operate on units that strictly preserve hardware functional integrity without coarse sequence-wide smearing or fragmented line-level scattering. Core idea: decompose Verilog programs into Atomic Logic Blocks (ALBs) representing the smallest semantically complete units of circuit logic, leverage semantics-aware simulation to obtain output-signal-level feedback, localize faulty ALBs via fingerprint-guided dependency cone matching, and apply asymmetric advantage correction to selectively penalize localized flaws only within globally favored trajectories.

Method

Overall Architecture

CirCA transforms the coarse sequence-level post-training paradigm into a structured, circuit-aligned credit assignment process structured across four sequential stages. Given a generated Verilog candidate alongside its reference golden design, the pipeline proceeds from syntax-preserving decomposition to localized gradient reweighting. First, an AST-guided analyzer breaks the Verilog code into macro-level concurrent parts and micro-level Atomic Logic Blocks (ALBs). Next, a semantics-aware testbench generator statically inspects clock, reset, and state transition specifications to drive protocol-valid simulation and capture per-signal cycle-accurate error rates. Then, for every erroneous output, backward signal dependency cones are constructed and structurally matched between generated and reference designs using canonicalized logic fingerprints, pinpointing the responsible faulty ALBs. Finally, an asymmetric advantage correction module integrates these localized responsibilities into GRPO by down-weighting the positive advantages of faulty ALBs while leaving negative trajectories untouched.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Raw Verilog & Reference Golden Design"] --> B["Atomic Logic Block Decomposition<br/>Concurrent isolation & procedural flattening"]
    B --> C["Semantics-Aware Signal-Level Feedback<br/>Protocol-driven stimuli simulation & error rate"]
    C --> D["Fingerprint-Guided Cone Matching<br/>Hash coarse filtering & recursive sub-cone isomorphism"]
    D --> E["Asymmetric Advantage Correction<br/>Selective damping on positive advantage trajectories"]
    E --> F["Circuit-Aligned Policy Update in GRPO"]

Key Designs

1. Atomic Logic Block Decomposition: Progressive Macro-Concurrent and Micro-Procedural Flattening

To overcome the dichotomy between sequence-level over-generalization and line-level fragmentation, CirCA formalizes the Atomic Logic Block (ALB) as the smallest semantically complete unit representing circuit logic. Verilog's two-tier execution structure is resolved through a progressive two-stage decomposition. At the macro level, a regex-based syntactic scanner identifies module headers, declarations, and concurrent statements; non-procedural constructs such as continuous assignments (assign) and module instantiations represent structurally separable combinational or interconnect topology and are preserved directly as standalone ALBs. At the micro level, procedural blocks (such as always and initial blocks) are recursively peeled: the outer procedural wrapper is stripped, and the execution body is split strictly along top-level control-flow divergence points (if-else branches and case items). Crucially, enclosing Boolean predicates are conjoined downwards into nested branches (\(C_{\text{outer}} \wedge C_{\text{inner}}\)) so that each execution path functions as an independent hardware logic unit with an explicit trigger condition \(\tau\), control context \(\chi\), and local assignment logic \(\psi\). Lightweight markers are inserted into the code without mutating Verilog semantics, establishing an exact bidirectional mapping between token indices and ALBs.

2. Semantics-Aware Signal-Level Feedback: Protocol-Driven Simulation for Dense Reward Extraction

To eliminate the sparse and misleading reward signals caused by naive random testbench stimulation, CirCA automatically synthesizes semantics-aware stimuli by statically analyzing the reference design. The generator extracts top-level interfaces and parameter bounds, identifies clock nets from edge-triggered sensitivity lists, resolves active-high versus active-low reset polarities from conditional predicates, and infers enable signals via structural naming conventions. For sequential behavior, it identifies state registers and state encodings to reconstruct finite state machine (FSM) transition graphs, while matching interface patterns for communication protocols like UART to drive transaction-compliant serial streams. Unconstrained data inputs are systematically swept across valid ranges. During simulation, co-executing the reference and generated designs yields cycle-level mismatches across all \(M\) output signals \(Y = \{y_1, \dots, y_M\}\). For each output signal \(y_m\), the cycle error rate over simulation duration \(T\) is computed as:

\[e_m = \frac{1}{T} \sum_{t=1}^T \mathbb{I}\left[\hat{y}_m^{(t)} \neq y_m^{(t)}\right]\]

A sample-level macroscopic reward \(r = \frac{1}{M} \sum_{m=1}^M (1 - e_m)\) is supplied to GRPO, while the set of failing signals (\(e_m > 0\)) is forwarded to downstream causal cone matching.

3. Fingerprint-Guided Cone Matching: Syntax-Invariant Causal Dependency Cone Isomorphism

To trace erroneous output signals back to their originating source ALBs, CirCA constructs a directed signal dependency cone \(C(y) = (V_y, E_y)\) for each target output signal \(y\) in both reference and generated circuits. Vertices represent signals or signal-update events, and directed edges reflect data dependencies flowing from right-hand side (RHS) operands to left-hand side (LHS) targets. To prevent cosmetic syntactic variations (such as variable renaming, constant base formatting, or Boolean term reordering) from confusing structural verification, every node \(s \in V_y\) is augmented with a canonicalized logic fingerprint \(\phi(s) = \langle\tau(s), \chi(s), \psi(s)\rangle\). Fault localization proceeds coarse-to-fine: an ALB-level hash derived from the sorted multiset of internal event fingerprints quickly prunes structurally verified, functionally identical ALBs. For the remaining unverified cones, recursive isomorphism \(\text{Iso}(u, v)\) navigates upstream from the output port, validating port bit-widths, pairing driving events via fingerprint similarity in a bipartite graph, and recursively verifying dependent antecedent signals. Any event mismatch or unpaired update immediately flags the corresponding generated ALB as a responsible faulty block, populating error block set \(B_m\).

4. Asymmetric Advantage Correction: Localized Dampening without Global Gradient Collapse

To translate localized error attributions into stable policy optimization, CirCA applies an asymmetric advantage correction scheme that penalizes faulty ALBs without destabilizing global exploration. When an ALB \(b\) is implicated by multiple failing output signals, it is assigned the maximum penalty rate across those signals: \(\gamma(b) = \max_{m: b \in B_m} e_m\). Given the global sample-level advantage \(\hat{A}\) and the set of tokens \(\mathcal{T}(b)\) belonging to block \(b\), the corrected token advantage \(\tilde{A}_t\) is defined as:

\[\tilde{A}_t = \begin{cases} (1 - \gamma(b)) \hat{A}, & \text{if } \hat{A} > 0 \text{ and } t \in \mathcal{T}(b) \\ \hat{A}, & \text{otherwise} \end{cases}\]

This asymmetric formulation resolves a critical vulnerability in relative policy optimization. When a generated candidate achieves a globally favored outcome (\(\hat{A} > 0\)), uniformly rewarding all tokens reinforces localized bugs; dampening the advantage of responsible ALBs by \((1 - \gamma(b))\) suppresses faulty constructs while rewarding surrounding correct logic. Conversely, when a trajectory is globally unfavored (\(\hat{A} \le 0\)), the sample is already suppressed. Artificially magnifying negative advantages on individual sub-blocks injects destructive gradient variance and induces policy collapse. Preserving the negative advantage intact safeguards training stability while achieving pinpoint attribution.

Loss & Training

CirCA is initialized from a Qwen2.5-Coder-7B-Instruct backbone and trained across two sequential phases: 1. Supervised Fine-Tuning (SFT): The model is trained for 2 epochs on 165k filtered Description-Verilog pairs curated from CRUX and CodeV-R1. Training uses the Adam optimizer with a learning rate of \(1.0 \times 10^{-5}\), linear warmup ratio of 0.1, sequence length of 4096, taking 32 hours across 4 NVIDIA A100 (80GB) GPUs. 2. GRPO Reinforcement Learning: After pruning tasks completely solved by the SFT model, 135k prompts are trained for 1 epoch under GRPO. For each prompt, \(G = 5\) candidate rollouts are generated. The AdamW optimizer is employed with a learning rate of \(1.0 \times 10^{-6}\), temperature 1.0, top-\(p\) 0.99, KL penalty coefficient \(\beta = 0.01\), and clipping threshold \(\epsilon = 0.2\). The token-level objective evaluates:

\[J_{\text{CirCA}}(\theta) = \mathbb{E}_{q, \{o_i\}} \left[ \frac{1}{G} \sum_{i=1}^G \frac{1}{|o_i|} \sum_{t=1}^{|o_i|} \min \left( \rho_{i,t} \tilde{A}_{i,t}, \text{clip}(\rho_{i,t}, 1-\epsilon, 1+\epsilon) \tilde{A}_{i,t} \right) - \beta D_{\text{KL}}(\pi_\theta \parallel \pi_{\text{ref}}) \right]\]

The entire RL phase requires approximately 40 hours on 4 A100 GPUs, where CirCA's online ALB decomposition and cone matching introduce a minimal computational overhead of only ~32.49 ms per sample compared to standard GRPO.

Key Experimental Results

Main Results

CirCA is comprehensively evaluated against foundation LLMs, specialized hardware models, and cutting-edge reasoning models across four widely recognized benchmarks: VerilogEval-v1 (Machine and Human subsets), VerilogEval-v2 (Code Completion CC and Spec-to-RTL SR), RTLLM-v1, and RTLLM-v2. The primary metric reported is pass@1 under standard low-temperature and cross-temperature sampling regimes:

Category Model VE-v1-Machine (p@1) VE-v1-Human (p@1) VE-v2-SR Cross (p@1) RTLLM-v1 (p@1) RTLLM-v2 (p@1)
Foundation General GPT-4o 67.7 60.1 64.1 41.7 56.5
Foundation General DeepSeek-V3-671B 77.6 70.7 62.4 60.9 59.1
Foundation Reasoning DeepSeek-R1-671B 81.0 81.5 77.5 64.8 64.7
Verilog Reasoning CodeV-R1-7B 86.1 69.9 68.8 72.9 68.0
Non-Reasoning Base Qwen2.5-Coder-7B 50.1 22.9 31.3 32.2 36.1
Specialized Non-Reasoning QiMeng-CRUX-7B 82.9 65.2 64.6 62.8 63.8
Proposed Method CirCA-7B (Ours) 86.0 65.1 69.1 70.5 69.0

When analyzing RL post-training efficacy via SFT-to-RL gains, CirCA achieves an absolute pass@1 improvement of +17.7 (relative gain of 33.52%) on RTLLM-v1 and +12.7 (relative gain of 22.56%) on RTLLM-v2, substantially outpacing both the previous best non-reasoning model QiMeng-CRUX (+13.8 and +11.5) and the leading reasoning model CodeV-R1-7B (+15.5 and +10.8).

Ablation Study

Controlled ablations investigate the isolated contributions of semantics-aware testbenches (SA-TB), attribution granularity (ALB vs. Concurrency vs. Line), and the directionality of advantage correction:

Configuration VE-v1-Mach (p@1) VE-v1-Hum (p@1) VE-v2-CC Cross (p@1) VE-v2-SR Cross (p@1) RTLLM-v2 (p@1) Description
SFT Baseline 74.6 60.4 61.9 61.3 56.3 Initial supervised checkpoint
w/o SA-TB (Random TB) 80.6 (+6.0) 62.5 (+2.1) 64.9 (+3.0) 63.7 (+2.4) 61.4 (+5.1) Standard GRPO with random inputs
w/ SA-TB (Semantic TB) 82.6 (+8.0) 63.1 (+2.7) 66.2 (+4.3) 65.8 (+4.5) 63.7 (+7.4) Naive GRPO with semantics-aware testbench
Concurrency-Level Attribution 78.8 (+4.2) 58.3 (-2.1) 61.2 (-0.7) 56.6 (-4.7) 54.9 (-1.4) Coarse macro-block level attribution
Line-Level Attribution 79.2 (+4.6) 60.7 (+0.3) 63.6 (+1.7) 60.9 (-0.4) 50.7 (-5.6) Fragmented line-level attribution
Symmetric Credit Assignment 83.2 (+8.6) 64.4 (+4.0) 67.4 (+5.5) 67.5 (+6.2) 65.1 (+8.8) Bidirectional penalty on positive & negative \(\hat{A}\)
CirCA Full (Asymmetric ALB) 86.0 (+11.4) 65.1 (+4.7) 67.9 (+6.0) 69.1 (+7.8) 69.0 (+12.7) Complete proposed framework

Key Findings

  • Criticality of Attribution Granularity: Deviating from the ALB abstraction severely harms performance. Concurrency-level attribution penalizes entire multi-branch procedural blocks indiscriminately, degrading RTLLM-v2 performance to 54.9% (below the 56.3% SFT baseline). Line-level attribution is even worse, collapsing RTLLM-v2 to 50.7% as fragmented token attribution injects destructive gradient variance into the policy network.
  • Diagnostic Mutation Localization: In a controlled diagnostic study where 50 atomic semantic mutations were injected into RTLLM-v2 designs, ALB-level cone matching correctly identified the faulty code region in 45 out of 50 cases (90.0% accuracy), compared to only 33 out of 50 cases (66.0% accuracy) for line-level matching.
  • Superiority of Asymmetric Correction: Symmetric advantage penalization degrades pass@1 by 3.9% on RTLLM-v2 and 1.6% on Spec-to-RTL relative to Asymmetric CA. Amplifying negative advantages in globally rejected samples (\(\hat{A} < 0\)) corrupts policy gradients without providing constructive guidance, validating the necessity of one-sided damping.
  • Surpassing Massive Reasoning Models: On complex sequential designs in RTLLM-v2, the 7B non-reasoning CirCA reaches 69.0% pass@1, outperforming the 671B DeepSeek-R1 (64.7%) and CodeV-R1-7B (68.0%). This demonstrates that aligning RL credit assignment with circuit semantics provides a compute-efficient alternative to inference-heavy chain-of-thought exploration.

Highlights & Insights

  • Hardware-Native Credit Alignment: CirCA identifies the fundamental mismatch between software-centric RL token attribution and concurrent circuit execution, pioneering Atomic Logic Blocks as the proper inductive bias for hardware post-training.
  • Fingerprint-Guided Graph Isomorphism in the Loop: By leveraging syntax-invariant hardware fingerprints to canonicalize Boolean relations and trigger conditions, the framework achieves robust causal fault localization in ~30 ms, proving that formal circuit analysis concepts can be seamlessly embedded into online RL loops.
  • Taming Relative Policy Optimization via Asymmetric Damping: Rather than attempting full bidirectional redistribution of advantages, CirCA selectively filters credit only when trajectories are globally successful, avoiding policy collapse while sharpening localized code synthesis.

Limitations & Future Work

  • Static Analysis Scalability in Deep Sequential Designs: ALB decomposition and dependency cone construction rely on static AST parsing and data-flow traversal, which can suffer latency bottlenecks or path-explosion when scaled to massive industrial SoCs with cross-module hierarchical interconnects.
  • Corner-Case Testbench Coverage: Although semantics-aware stimuli successfully activate state machines and standard buses, rare temporal states requiring intricate multi-cycle handshake protocols (such as odd-frequency clock divider phases or complex FIFO buffer underflow states) remain challenging to trigger within fixed simulation step bounds.
  • Future Directions: Extending the ALB paradigm to SystemVerilog object-oriented testbench verification and High-Level Synthesis (HLS), as well as integrating lightweight formal property verification (FPV) or bounded model checking to augment simulation-based error detection.
  • vs QiMeng-CRUX (Huang et al., 2026): While CRUX improves Verilog generation by structuring natural language task descriptions into formal diagrams and Karnaugh maps, its RL phase relies on naive GRPO with sequence-level attribution. CirCA addresses the algorithmic bottleneck of RL credit assignment directly, yielding an absolute +4.5% boost on Spec-to-RTL.
  • vs QiMeng-SAL-V (Zhang et al., 2024): SAL-V introduces signal-level feedback within an offline Direct Preference Optimization (DPO) framework. CirCA advances this paradigm to online reinforcement learning via GRPO, providing continuous policy exploration guided by causal cone matching.
  • vs Long-CoT Reasoning Models (DeepSeek-R1 & CodeV-R1): Reasoning models achieve strong accuracy by expanding lengthy natural language thought chains during inference, at the expense of severe computational latency and decoding fragility under low temperature. CirCA establishes that circuit-aligned credit assignment enables compact 7B non-reasoning models to match or exceed reasoning models on complex RTL tasks without inference overhead.

Rating

  • Novelty: โญโญโญโญโญ Pioneering introduction of Atomic Logic Blocks (ALBs) and fingerprint-guided cone matching to align RL credit assignment with physical circuit semantics.
  • Experimental Thoroughness: โญโญโญโญโญ Rigorous validation across 4 primary benchmarks, systematic ablations across granularities, and a controlled 50-sample mutation diagnostic study.
  • Writing Quality: โญโญโญโญโญ Exceptionally coherent narrative, formal algorithmic definitions, and precise technical exposition.
  • Value: โญโญโญโญโญ Provides an effective and practical foundation for automated EDA and hardware-software co-design using post-trained language models.