Circuit-MLLM: Topological Logic-Guided Latent-Space Visual Reasoning for Circuit Schematic Understanding

Conference: ECCV2026
Paper: Official paper page · PDF
Code: https://github.com/IC-Yuan/Circuit-MLLM
Area: VLM Reasoning (Circuit Schematic Understanding)
Keywords: circuit topology, latent visual reasoning, multi-expert distillation, topological sequencing, fine-grained wire recognition

TL;DR

Circuit-MLLM organizes training-time circuit knowledge from multiple vision experts into latent-vector supervision ordered by connectivity, enabling Qwen2.5-VL-7B to trace connections without invoking those external experts at inference and raising its average score from 48.28 to 76.20 on a custom benchmark, although exact match on hard connection identification remains only 28.52%.

Background & Motivation

A circuit schematic is not simply a collection of objects: recognizing resistors, capacitors, and transistors establishes what is present, but not how those components connect. Connectivity is often encoded in thin, extended wires, junctions, and device terminals, where a small visual error can change the network. A general-purpose MLLM may classify symbols correctly yet fail to enumerate their connections. In the paper's main table, Qwen2.5-VL-7B reaches 83.55% element-classification accuracy but only 15.20% connection-identification exact match, illustrating this gap between recognition and topology understanding.

One approach uses detectors or parsers to produce structured descriptions before language-model reasoning; another directly fine-tunes on images and answers. The former turns upstream omissions into fixed downstream inputs, while the latter lacks sufficiently detailed supervision explaining why components connect. Explicit visual chain-of-thought cropping is not automatically suitable either: rectangular regions help inspect local objects but can sever wires crossing region boundaries or retain extensive irrelevant background. Latent reasoning avoids having to decode intermediate states into words or crop boxes. Yet generic patch features may still miss thin wires, and arranging those features from top left to bottom right imposes a reading order unrelated to electrical connectivity.

The authors therefore distinguish what latent vectors should contain from the order in which they should appear. Their attention observations suggest a useful starting point: a model sometimes anchors on the queried device and follows wires toward connected targets instead of scanning the whole image. Core idea: provide fine-grained circuit knowledge through fused expert features, then order the supervision targets by query-related topology so that both wire perception and progression along connections are learned in latent space.

Method

Overall Architecture

The input is a schematic and a question; the output is a textual answer such as a count, a device category, a connectivity judgment, or a set of connected devices. Training first uses offline component, text, junction, and wire extraction to construct query-related pixel-level topology annotations. Latent-Space Circuit Knowledge Mining, Topology-Guided Sequencing, and Text-Latent Joint Supervision then train latent vectors placed between preceding and subsequent text. At inference, the model generates latent vectors and an answer from the image and question without invoking the HAWP, DeepLSD, and DINOv2 experts used to construct training targets.

Here, the absence of external tools applies to inference, not to supervision production. YOLO11, OCR, pixel tracing, and human verification all contribute to the training data. Pixel masks and fused expert features belong to the teacher side; they should not be interpreted as mandatory online operations for every question.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Data["Training images and questions"] --> Expert["Latent-Space Circuit Knowledge Mining<br/>HAWP / DeepLSD / DINOv2"]
    Data --> Annotation["Offline annotation and verification<br/>Query-related pixel masks"]
    Expert --> Sequence["Topology-Guided Sequencing<br/>Filter, sort, and compress"]
    Annotation --> Sequence
    Sequence --> Joint["Text-Latent Joint Supervision<br/>Latent alignment and text prediction"]
    Joint --> Model["Trained Circuit-MLLM"]
    Query["Inference image and question"] --> Model
    Model --> Answer["Latent reasoning and textual answer<br/>No teacher-expert calls"]

Key Designs

1. Latent-Space Circuit Knowledge Mining: encode wires and junctions, not just device appearance

A generic visual encoder supplies semantic foundations but may not emphasize the thin lines and intersections essential to circuits. The authors introduce three complementary teachers: HAWP captures holistic wireframes and junctions, DeepLSD supplies finer line-segment structure, and DINOv2 provides patch-level semantic representations. Their feature maps differ in resolution and channel count, so bilinear interpolation first maps them onto the common grid determined by the model's original visual encoder. Channel concatenation, flattening, and a learnable projection produce expert vectors with the model's latent dimensionality. These are concatenated with the original visual tokens and passed through a fusion layer to form a knowledge-enriched feature pool.

This does not insert wire detections as textual prompts or allow a detector to decide the final answer online. Each pool location remains a continuous feature that retains the original encoder's semantics while incorporating wireframe and line-segment priors; subsequent latent vectors learn to approach combinations of these locations. Expert capabilities therefore enter through a training loss rather than repeated inference-time tool calls. Retaining the original visual tokens also preserves the original model's semantic source instead of simply replacing its encoder. However, the cache does not specify whether the experts are frozen, how projection layers are initialized, or the complete gradient flow, so those implementation details cannot be supplied here.

2. Topology-Guided Sequencing: turn spatial locations into query-related traversal targets

Detailed features alone do not determine which locations matter or which should come first. The authors collect 12,000 analog, digital, and mixed-signal circuit schematics from public datasets. They manually annotate a subset of devices, text, and junctions, train YOLO11 to extend detection, use OCR to recognize text and associate device attributes, reconstruct topology through pixel tracing, and involve human experts in verification. The resulting annotations include netlist-style descriptions, visual grounding, and pixel-level masks. For a connectivity question, supervision distinguishes the root device, relevant wires, connected targets, and background rather than assigning every question the same global attention map.

The Topological Logic Mask encodes these regions as a sequencing signal. The text describes a near-to-far principle based on breadth-first search (BFS) distance from the root along wires, with continuous random values distinguishing connected devices. A reproducibility boundary matters here: Equation (4) is severely corrupted in the cache. The surviving root assignment, normalized wire distances, and random target-device values do not fully establish how the subsequent ascending sort guarantees the intended root-to-wire-to-target order. This note therefore explains the intended behavior and readable operations without reconstructing the piecewise formula or treating the intended traversal as a proven algorithmic property.

To map pixel annotations onto visual patches, the authors use adaptive max pooling. A thin wire may occupy only a small fraction of a patch; average pooling could dilute it, whereas max pooling aims to preserve such localized structures. Patches with zero mask values are removed as background, and the remaining fused features are sorted by increasing mask value, replacing fixed raster order. A final one-dimensional adaptive average pooling compresses the ordered sequence into \(k\) target latent vectors. The two pooling operations solve different problems: spatial max pooling preserves thin structures, while one-dimensional average pooling accommodates the latent-vector budget rather than performing another background decision.

After compression, one latent vector can summarize several adjacent positions in the sorted sequence. Consequently, \(k=4\) does not mean the model can recognize only 4 devices, nor does each token necessarily correspond to one device. The paper also observes simultaneous attention to multiple branches from a node. Sequencing primarily constrains progression across latent stages, not a strictly serial visit to one branch at every step. The additional contribution is therefore structure in the supervision sequence, not merely stronger visual features.

3. Text-Latent Joint Supervision: constrain intermediate evidence while teaching answer generation

The output contains text preceding the latent vectors, the latent-vector sequence, and subsequent text. Training only on answer text might reward device co-occurrence or linguistic shortcuts without teaching wire tracing; matching visual features alone would not ensure a correctly expressed connection set. The authors optimize both: at each latent position, the final-layer hidden state is aligned with its corresponding ordered teacher vector using cosine distance, while preceding and subsequent text positions receive autoregressive cross-entropy supervision. The subsequent text is conditioned on the generated latent vectors.

The following is a normalized restatement of the core losses from the readable descriptions in Sections 3.2 and 3.3. Equations (7) through (9) contain extraction damage in the cache, so this is not a literal transcription of their corrupted characters:

\[ \mathcal{L}_{\mathrm{topo}}=\frac{1}{k}\sum_{m=1}^{k}\left[1-\cos(h_m,E_m^*)\right], \qquad \mathcal{L}_{\mathrm{total}}=\lambda\mathcal{L}_{\mathrm{topo}}+\mathcal{L}_{\mathrm{text}}. \]

Here, \(h_m\) is the final-layer hidden state at latent position \(m\), \(E_m^*\) is the corresponding sorted and compressed teacher vector, and \(\mathcal{L}_{\mathrm{text}}\) covers text before and after the latent vectors. Position-wise matching matters: a loss that merely matched the overall collection of latent vectors to the overall feature collection might remain similar after swapping their order. Matching corresponding positions instead transfers the teacher-side sequence into the training objective. This is nevertheless a soft constraint, not an inference-time BFS execution or a symbolic checker enforcing circuit rules.

A Worked Example

Consider the question, "List the devices connected to the specified resistor." This is a mechanism illustration, not an additional test example or measurement reported by the paper. Offline annotations identify the resistor, wires extending from its terminals, and connected devices. The three experts provide wireframe, fine-line, and semantic features on a common grid. A query-related mask removes irrelevant background, reorders retained locations using the topological sequencing signal, and compresses them into the default \(k=4\) supervision targets.

During training, these targets constrain the four latent positions to progress toward the intended root, connection-path, and target regions, while the answer text learns the correct device set. At inference there is no ground-truth mask or teacher output, so the model must rely on its learned correspondence to perform this implicit process. If it identifies most connections but omits one device, F1 may remain high, whereas exact match ratio (EMR) counts the response as incorrect. This explains how a high F1 and a low Hard EMR can coexist.

Loss & Training

The backbone is Qwen2.5-VL-7B. Training uses PyTorch on 8 NVIDIA A100 GPUs for 15 epochs with batch size 8, an initial learning rate of \(10^{-5}\), and cosine learning-rate scheduling. The default latent size is \(k=4\), and the loss-balancing coefficient is 0.6. The method section denotes that coefficient by \(\lambda\), whereas the implementation paragraph and ablation tables use \(\gamma\); this note preserves the inconsistency rather than interpreting them as two independent hyperparameters. The implementation paragraph also mentions "both stages," but the cache does not fully specify separate stage configurations.

Instruction tuning uses 5,000 instances: 30% connection identification, 25% connection judgment, and 15% each for element classification, total counting, and type counting. Training and evaluation images are strictly disjoint. This is an image-level separation statement, not evidence of source-level, template-level, or topology-level deduplication. Mirage and ILVR are fine-tuned using the same latent-space data and latent size 4; SFT and SFT+GRPO measure direct training without the latent-space mechanism.

Key Experimental Results

Main Results

Evaluation uses the custom Circuit-MLLM-Bench. Total counting, type counting, element classification, and connection judgment use accuracy. Connection identification is a multiple-response task evaluated with F1 and EMR, the latter requiring the predicted set to match the ground-truth set exactly. Scores below follow the paper's 0–100 scale. Avg is the mean of the reported metric columns, not a separate measurement. The cache does not specify the precise aggregation convention for connection-identification F1.

Table 1 excerpts the paper's Table 1, retaining models that cover the main comparison routes.

Model	Total Count Acc.	Type Count Acc.	Element Class Acc.	Connection Judgment Acc.	Connection Identification F1	Connection Identification EMR	Avg
Qwen2.5-VL-7B	21.06	48.40	83.55	55.92	65.53	15.20	48.28
Masala-CHAI-7B	27.72	48.10	93.10	51.00	66.35	19.10	50.88
GPT-5.1	43.60	60.62	96.49	61.11	76.08	30.20	61.35
GPT-4o	42.01	60.08	97.40	58.50	78.68	36.30	62.16
GLM-4.5V-106B	58.62	59.50	90.08	58.59	79.37	40.00	64.36
Circuit-MLLM-7B	75.87	65.10	97.40	72.60	88.90	57.30	76.20

Compared with GPT-5.1, the average rises by 14.85 points, or approximately 24.21% relative to its tabulated score; the abstract's "25%" is an approximate headline claim. Compared with the backbone, the gain is 27.92 points, or approximately 57.83%. These are results after domain-specific training on a custom benchmark, not evidence of generally surpassing general-purpose models. Relative percentages must also not be described as accuracy percentage-point gains.

Table 2 excerpts key columns from the paper's Table 2. Easy, Medium, and Hard are defined using the number of connected devices and the total number of devices in the schematic. The retained Avg still comes from all 9 metric columns in the original table; it must not be recomputed using only the 4 task metrics displayed below.

Method (7B)	Judgment Medium Acc.	Judgment Hard Acc.	Identification Hard F1	Identification Hard EMR	Original Avg
Backbone, no SFT	51.56	41.52	61.54	3.78	45.09
SFT	63.28	56.68	80.40	26.12	67.74
SFT+GRPO	62.76	57.40	80.53	26.80	67.75
Mirage	66.15	64.98	81.88	26.80	69.82
ILVR	63.28	63.90	79.47	22.68	68.37
Circuit-MLLM	70.31	64.98	83.54	28.52	72.26

Circuit-MLLM leads Mirage by 2.44 average points and ILVR by 3.89. It ties Mirage at 64.98 on Hard connection judgment, so it is not strictly better in every cell. Hard EMR improves over ILVR by 5.84 percentage points, approximately 25.75% relatively, but the absolute value of 28.52 still means most difficult examples are not completely resolved.

Ablation Study

Table 3 excerpts the paper's Table 3. Seq denotes Topology-Guided Sequencing, and Expert denotes Latent-Space Circuit Knowledge Mining. Avg again comes from all 9 topology metric columns in the original table. The baseline latent model scores 69.85 here, whereas Mirage scores 69.82 in Table 2; these should not be forcibly treated as the same experimental row.

Seq	Expert	Judgment Medium Acc.	Judgment Hard Acc.	Identification Medium EMR	Identification Hard EMR	Original Avg
No	No	65.10	62.45	61.87	25.46	69.85
Yes	No	66.15	64.26	62.40	25.77	70.61
No	Yes	65.62	62.50	63.47	27.15	70.64
Yes	Yes	70.31	64.98	62.67	28.52	72.26

Key Findings

• Sequencing alone adds 0.76 average points, expert knowledge alone adds 0.79, and combining them adds 2.41, supporting their complementarity. However, the full model's Medium EMR of 62.67 is below the expert-only score of 63.47, so not every submetric improves monotonically.
• In the paper's Table 4, average scores for \(k=2,4,6,8\) are 74.42, 76.20, 74.19, and 75.43. More latent vectors do not yield consistent gains. A budget of 4 is best on this data, not necessarily sufficient for every complex circuit.
• Table 5 reports average scores of 75.16, 74.95, 75.23, and 76.20 for DINOv2, HAWP, DINOv2+HAWP, and all three experts, respectively. There is no DeepLSD-only row, so its full independent contribution cannot be quantified from that table.
• Table 5 states \(k=6\), but its all-expert row reuses the \(k=4\) results from Table 4, whose \(k=6\) average is 74.19. The main table also reports connection-identification F1 as 88.90, while later tables report 88.92. This note does not silently reconcile those conflicts; cross-table configuration comparisons require checking the definitive paper or code.

Highlights & Insights

• Experts move from online decision makers to training teachers. Wireframe and fine-line knowledge can guide intermediate representations through continuous targets rather than being passed as a discrete netlist. This reduces inference-time dependence on a tool chain without eliminating teacher annotation errors.
• Ordering is part of supervision. Models aligned to the same set of patches can learn different intermediate trajectories if those patches are presented in different orders. This paper adds topology to target construction instead of merely adding input information.
• The pooling choices address different constraints. Max pooling addresses the loss of sparse thin wires, whereas one-dimensional average pooling addresses a limited latent budget. The transferable principle is structure-aware compression, not unconditional reuse of the chosen hyperparameters.

Limitations & Future Work

• Difficult topology remains far from reliable automation. Hard connection-identification F1 is 83.54, but EMR is only 28.52; partially correct connections do not make the entire predicted set trustworthy. EDA deployment would still need netlist consistency checks, error localization, or abstention mechanisms.
• Data separation and evaluation coverage need further disclosure. The cache claims image-disjoint training and testing but does not provide the complete evaluation size, source grouping, or near-duplicate topology checks. Additional Amsbench results are referenced only through unresolved cross-references, without verifiable tables, limiting conclusions about distributional generalization.
• Sequencing cannot be fully reproduced from this cache. Equation (4) is corrupted, the way random target-device values guarantee strict ordering is unclear, and coefficient notation and expert-ablation configurations are inconsistent. These require clarification in the definitive algorithm or code rather than being hidden behind a smooth explanation.
• Attribution maps are not causal verification. Attention heatmaps show movement among roots, wires, and targets, but do not prove execution of a graph-traversal algorithm. Interventions such as removing a critical wire or replacing a junction would provide stronger evidence.
• Efficiency and stability evidence is limited. Avoiding external teachers does not imply zero overhead relative to ordinary direct answering, because generating latent vectors still costs computation. The cache supplies no latency, throughput, memory comparison, or multi-seed error bars. These are the note's evidence-based concerns; the paper does not include a dedicated systematic limitations section.

Related Work & Insights

• Compared with Masala-CHAI / MAPs: These represent external visual-tool assistance and schematic-to-netlist conversion followed by text-model reasoning, respectively. Circuit-MLLM turns structural knowledge into training targets, avoiding dependence on that intermediate conversion for every inference, at the cost of latent outputs being less directly inspectable than explicit netlists.
• Compared with Mirage / ILVR: All use latent visual reasoning, but Circuit-MLLM adds circuit-specific experts and topologically ordered targets. Comparisons using the same data and latent budget are more informative about these mechanisms than comparisons against general-purpose models without the same domain adaptation.
• Compared with explicit visual CoT: Crops are useful for inspecting local evidence, whereas circuit connectivity requires continuity across regions. For flowcharts or pipeline diagrams, reliable structural annotations could similarly be used to construct ordered targets; this is a transfer hypothesis, not an application tested in the paper.
• Resource boundary: The code URL is taken from the paper's abstract. This note uses only the locally cached full text, does not verify the repository online, and does not treat unavailable supplementary experiments as established results.

Rating

• Novelty: 4/5. Combining circuit-expert distillation with topological sequence supervision is well targeted, although latent-space alignment builds on existing methods.
• Experimental Thoroughness: 3/5. General-purpose models, direct fine-tuning, latent baselines, and component ablations are covered, but external generalization, efficiency, and statistical stability remain insufficiently documented.
• Writing Quality: 3/5. The main argument is clear, but damaged equations, unresolved references, and cross-table configuration conflicts in the current cache impede reproduction.
• Value: 4/5. The supervision-construction approach is useful for structured visual tasks, but exact correctness on difficult connections is not yet adequate for engineering use without verification.