From Macro to Micro: Benchmarking Microscopic Spatial Intelligence on Molecules via Vision-Language Models¶

Conference: ECCV2026
Paper: ECCV Virtual
Area: Multimodal VLM
Keywords: microscopic spatial intelligence, vision-language model, molecular spatial reasoning, benchmark, hydrogen bond recognition

TL;DR¶

This paper introduces MiSI-Bench: it decomposes "microscopic spatial intelligence" into four elementary operations (translation, rotation, zooming, interaction) and renders roughly 4,000 PDBbind crystal complexes into 163K QA pairs and 588K orthographic-projection images across nine unit and composite tasks; state-of-the-art VLMs average only 16-34 (humans: 81.18), while a 7B model fine-tuned on this training split reaches 62.96 and surpasses humans on rotation tasks, yet still lags far behind on knowledge-grounded tasks such as hydrogen-bond recognition.

Background & Motivation¶

Spatial intelligence is one of the hottest directions for vision-language models, and a family of benchmarks has already mapped out whether models can read layouts, occlusions, and viewpoints: VSI-Bench / Thinking in Space probes spatial memory and reasoning over video and multi-image input, MuirBench tests robustness of multi-image understanding, LEGO-Puzzles examines multi-step spatial reasoning in a synthetic block world, and SAT takes the spatial-ability training route. All of these share one implicit premise — the objects under test are macroscopic and visible: cups on a table, furniture in a room, a shifting street scene.

The real battleground of scientific discovery, however, lies at the invisible microscopic end. Structural biologists and medicinal chemists open protein-ligand complexes in tools such as PyMOL and ChimeraX every day, reading a three-dimensional structure back out of orthographic views, judging geometric complementarity, and inferring physical relationships such as hydrogen bonds. This paper calls the ability to reconstruct 3D structures from 2D projections and then translate abstract scientific concepts (binding, interaction) into precise geometric constraints Microscopic Spatial Intelligence (MiSI). It differs from macroscopic spatial tasks in three substantive ways: visual elements must be mapped onto hierarchical scientific entities (atoms, residues) rather than onto "cup" or "chair"; words like "binding" have no everyday visual counterpart and must be grounded in the geometry of specific atom pairs; and whereas macroscopic tasks tolerate spatial imprecision, microscopic ones demand a tight coupling between visual perception and domain language. Nobody had measured it systematically before, partly because the interface did not exist — conventional 3D molecular modelling (force fields, 3D convolutions, graph neural networks, equivariant graph networks and geometric transformers) all operate in Cartesian coordinate space and cannot consume images at all, while VLMs, which handle images and text natively, had no paired "molecular rendering + domain question" data to work with. Both enablers are now in place: ChimeraX can batch-render orthographic projections, and PDBbind supplies roughly 4,000 experimentally solved crystal structures.

This paper therefore builds a ruler rather than a model. It adopts the orthographic projections experts already use (canonical front / top / left views) as the 2D representation, and condenses expert skill into four elementary operations; four unit tasks isolate one operation each, and five composite tasks chain several operations together. Every QA pair is instantiated from a fixed template, continuous quantities are discretized into bins, and multiple-choice distractors are produced by three targeted perturbation schemes. Core idea: decompose "microscopic spatial intelligence" into an enumerable set of elementary operations and their compositions, use orthographic projections of one shared pool of crystal structures to mass-produce QAs of controllable difficulty, and separate approximately-correct answers from exhaustive enumeration with an axis-gated, error-based partial-credit scheme plus two anti-cheating rules — thereby quantifying, for the first time, the real gap in VLMs' molecular spatial reasoning.

Method¶

Overall Architecture¶

The input is a protein-ligand complex: crystal structures from PDBbind refined, filtered for visualization problems down to 3,503 complexes for supervised fine-tuning and 490 for testing, all with experimentally solved structures. Each complex is rendered in ChimeraX as 1160 × 803 orthographic projections, supplying three or six of the front / left / top / back / right / bottom views depending on the task. Nine tasks prise microscopic spatial intelligence apart: four unit tasks isolate translation, rotation, zooming, and residue-ligand interaction, while five composite tasks chain operations and test spatial transformation reasoning (translation + rotation, rotation + rotation), local relational reasoning (interaction location), and global relational reasoning (docking, pocket-ligand interaction). Questions are instantiated from fixed templates in two formats — cloze and multiple-choice — yielding 150,597 training QA plus 12,917 test QA (163,514 in total), corresponding to 538,015 training images and 49,960 test images. Beyond the full test set, the authors sample 50 QA pairs per task into MiSI-Bench(tiny) for cost-sensitive evaluation of closed-source models. The dataset is public at https://huggingface.co/datasets/zongzhao/MiSI-bench.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["PDBbind crystal complexes<br/>3,503 train + 490 test"] --> B["Capability decomposition<br/>4 operations → 9 tasks"]
    A --> C["Orthographic rendering<br/>ChimeraX 3/6 views, 1160×803"]
    B --> D["Template-based QA generation<br/>bin discretization + 3 perturbations"]
    C --> D
    D --> E["Layered scoring protocol<br/>partial credit + anti-cheating"]
    E --> F["MiSI-Bench<br/>163,514 QA / 587,975 images"]

Key Designs¶

1. Capability decomposition: splitting vague "molecular understanding" into four elementary operations and two task tiers

The most intuitive way to judge whether a VLM "understands" molecules is to ask it chemistry questions, but such a score cannot localize the failure. This paper takes a different route: first observe what experts actually do. Working with molecules in visualization software, experts repeatedly use four atom-level operations — translation along the x / y axes, rotation about the x / y / z axes, zooming along the z axis perpendicular to the screen (which reads as magnification), and interaction, judging whether a given residue interacts with the ligand and which atom pairs are involved. These four operations become four unit tasks, each varying only one factor: the translation task moves the complex between −4 and 4 Å along x or y and asks the model to fill in move x 3; the rotation task rotates about one coordinate axis by an angle drawn from [−90°, 90°] and asks for roll x 15; the zooming task moves the complex along z between 40 and 60 Å (the magnification range best suited to viewing pocket-ligand interactions near the image centre; the distribution is in Appendix A); and the interaction task requires a yes/no decision plus the list of atom pairs, e.g. ARG NH2, O22; ARG N, O23.

Above the unit tasks, five composite tasks chain multiple operations, probing the higher-order abilities experts need in practice, and these split into two semantic categories. Spatial transformation reasoning provides the three orthographic views of two complexes \(c_1\) and \(c_2\) together with the front view of \(f_2(f_1(c_1))\) — the result of applying \(f_1\) and then \(f_2\) to \(c_1\) — and asks the model to pick the front view of \(f_2(f_1(c_2))\) out of four candidates: that is, "read one transformation and transfer it to another molecule". The two tasks sample \(f_1\) from translations with \(f_2\) from rotations (translation + rotation), and both \(f_1\) and \(f_2\) from rotations (rotation + rotation, with the two rotations constrained to different axes to prevent trivial correlations). Relational reasoning splits into local and global. Interaction location gives the six views of a complex plus a hydrogen-bond atom pair (e.g. ARG 45 NH2 A, O1B) and asks for the transformation command that moves that interaction to the centre of the view. Docking gives the ligand alone, the pocket alone, and one displaced complex view, and asks for the operation sequence that restores the native conformation (e.g. roll y 45, move x -12). Pocket-ligand interaction scales the residue-level task up to the whole binding pocket and asks for every hydrogen bond between ligand and pocket, each entry a five-tuple of residue type, residue index, residue atom, chain, and ligand atom. In scale, unit tasks account for 48.29% of the data and composite tasks for 51.71%; translation has 14,012 / 980 training / test samples, rotation 21,018 / 1,470, zooming 14,012 / 980, residue-ligand interaction 11,572 positive plus 12,125 negative training samples and 1,499 positive plus 1,603 negative test samples, translation + rotation and rotation + rotation 21,018 / 1,470 each, docking 21,018 / 1,470, and interaction location and pocket-ligand interaction 7.52% and 2.33% of the total respectively. The benefit of this cut is attribution: when a model collapses on rotation tasks, that failure no longer blends into "it does not recognize what a protein looks like".

2. Orthographic rendering: how a 3D complex becomes expert-style 2D multi-view input

VLMs cannot consume coordinates, and feeding point clouds or voxels directly is neither their native input format nor the way experts actually work. The representation chosen here is the orthographic projection: parallel light rays cast the 3D structure onto a 2D plane, taking canonical front / top / left views — the same convention humans use when reconstructing a solid object from two plan views and a section. The advantage over perspective projection is the absence of foreshortening: the same Ångström distance maps to a consistent pixel scale across views and depths, which is exactly what allows a model (or a human) to infer "how many Ångströms did it move" from a pixel shift.

The number of views is decided per task type, and this is a grounded choice rather than an arbitrary one. Translation, rotation, and zooming tasks receive three views (top / front / left) plus the post-transformation front view, because continuous transformations need a stable scale reference. Residue-ligand interaction receives all six views, on the stated grounds that atom-name labels overlap and occlude each other in dense regions, so additional viewpoints substantially reduce ambiguity. Rendering is batched through ChimeraX at a uniform 1160 × 803, keeping the image distribution consistent across complexes and tasks. On the data side only one filtering step is applied: complexes with visualization problems (missing ligand, abnormal views) are removed, leaving 3,503 + 490 complexes. This design also explains an interesting empirical finding discussed later — under orthographic projection depth change can only appear as subtle shifts in interatomic spacing and occlusion, which is precisely why even humans score only 30.00 on zooming.

3. Template-based QA generation: bin discretization and three directed perturbations that create hard distractors

Answers in spatial tasks are naturally continuous (angles, distances) or even variable-length lists, so free-form generation can be neither graded nor compared across models. The paper therefore templatizes the questions and discretizes the answers: translation distances into 1.0 Å bins, rotation angles into 15° bins, and zooming depths into 1.0 Å bins, so the model only has to fill in an enumerable value. The cost is sub-bin precision, but the benefit is deterministic evaluation — and the scoring stage does not demand exact matches anyway, since it grants partial credit for "correct axis plus how large the deviation is" (see design 4), so discretization does not kill approximately-correct answers.

Multiple-choice distractors are the part of this benchmark that requires the most design care, because spatial multiple-choice questions are easily broken by elimination or language priors. For spatial transformation tasks (translation + rotation, rotation + rotation), wrong options come from three directed perturbations: ① altering the magnitude of both \(f_1\) and \(f_2\) (testing "by how much"); ② flipping the sign of \(f_1\) (clockwise ↔ counter-clockwise) and adjusting the magnitude of \(f_2\) (testing "which way"); ③ changing the axis of \(f_1\) and modifying the magnitude of \(f_2\) (testing "about which axis"). The three perturbations target the three canonical error types — magnitude, direction, and axis — so shirking on any single dimension gets caught by one of the options. Interaction location perturbs only the sign and magnitude of the ground-truth translation, since the task itself contains no rotation and the distractors must stay within the same capability dimension to be fair. Docking is initialized more carefully still: rotation angles are sampled uniformly from [−90°, 90°], while the translation distance is not sampled from a fixed range but chosen adaptively per complex so that it is clearly visible yet does not make the displaced ligand badly interpenetrate the pocket (details in Appendix A). Otherwise the model could reverse-engineer the answer from the prior clue "this pose is physically impossible" instead of genuinely reasoning about spatial relations.

4. Layered scoring protocol: partial credit for continuous answers, enumeration-proofing for list answers

Exact matching throughout would collapse the continuous cloze tasks to zero and destroy the benchmark's resolution; hit-rate throughout would let list answers be gamed by brute force. The metrics are therefore layered by answer structure. Multiple-choice tasks and the zooming task follow convention and use accuracy (ACC), the proportion of answers exactly matching the ground truth. Cloze tasks switch to a weighted composite score: for spatial transformations the axis is checked first, and only when the predicted axis is correct is credit assigned from the normalized absolute error between predicted and ground-truth magnitudes, \(|\hat d - d|\) (the paper only says "normalized" and never states the normalization constant — ⚠️ refer to the original paper for the exact range). In composite tasks each component operation (e.g. move and roll) contributes equally, summing to 1.0; in docking, because the move axis is fixed to x, credit is given only when the sign of the prediction matches the ground truth.

Interaction tasks are scored as the ratio of correctly predicted entries to ground-truth entries, with two anti-cheating constraints added: answers that list every correct interaction but also fabricate irrelevant ones are capped at 0.5; and if the number of hydrogen bonds in a response exceeds twice the ground-truth count, the response is treated as an attempt to score by exhaustive enumeration and receives 0. These two rules target the classic loophole of multi-entry extraction answers — in residue-ligand interaction and pocket-ligand interaction, the space of enumerable atom pairs is finite, and without a penalty a model could score highly simply by listing every possible pair. The authors also report exact-match results for the cloze tasks in Appendix B, for comparisons at a different strictness level.

A Worked Example¶

Take a "translation + rotation" composite question (the values fall inside the sampling ranges stated in the paper; ⚠️ refer to Appendix C of the original for a full example). Complex \(c_1\) and a randomly paired \(c_2\) are each shown in three views; \(f_1\) is a +2 Å translation along x and \(f_2\) is a 30° rotation about y. The question provides the front view of \(f_2(f_1(c_1))\) as the reference and asks which of A-D is the front view of \(f_2(f_1(c_2))\). The distractors are built as follows: option A alters both magnitudes (+3 Å and 45°), option B flips the sign of \(f_1\) to −2 Å and adjusts \(f_2\) to 20°, and option C changes the axis of \(f_1\) from x to y. A model answering "move x 2, roll y 30" scores full marks; answering "move x -2, roll y 30" gets the rotation right but the translation sign wrong, and with the two sub-operations weighted equally this yields 0.5; answering "move y 2, roll y 30" gets the translation axis wrong, so that component scores 0. For a residue-ligand interaction question with ground truth ARG 221 NH2 A, O22; ARG 221 N A, O23, a model that returns both correct entries but adds a non-existent ARG 221 NE A, O22 triggers the anti-cheating rule and is capped at 0.5; if it dumps ten entries, more than twice the ground truth, it scores 0 outright.

Loss & Training¶

The method side introduces no new training objective; the only trained model is Qwen2.5VL-7B-SFT, fine-tuned on the 150,597 training QA pairs of MiSI-Bench and evaluated on the 12,917 test QA pairs, as a probe for whether VLMs already possess microscopic spatial intelligence or merely lack adaptation. The paper does not report the learning rate, batch size, whether LoRA was used, or the number of epochs (⚠️ refer to the original paper), so the relative contributions of data scale and training configuration to the gain cannot be separated. On the evaluation side, all closed-source and open-source MoE models are evaluated on MiSI-Bench(tiny) (50 QA pairs per task) under few-shot settings for cost reasons, and the human baseline and the SFT model are evaluated under that setting or on the full test set; this matters for cross-model comparisons (see Limitations & Future Work).

Key Experimental Results¶

Main Results¶

Table 1 summarizes ten mainstream VLMs, human evaluators, and the authors' SFT model across the nine tasks. The average follows the original convention of excluding the "translation + rotation" and "rotation + rotation" columns (where almost all models are at chance among four options); rows are sorted by average score.

Method	Avg.	Translation	Rotation	Zooming	Res-Lig (Pos.)	Res-Lig (Neg.)	Trans-Rot.	Rot-Rot.	Docking	Inter. Location	Poc-Lig Inter.
Human	81.18	100.00	70.18	30.00	100.00	100.00	32.00	26.00	74.54	92.00	82.78
Qwen2.5VL-7B-SFT	62.96	99.84	99.71	27.14	63.46	89.52	88.44	89.59	57.79	88.37	10.72
Claude Sonnet4.5	34.37	45.71	44.18	6.00	22.33	84.00	28.00	26.00	34.12	38.00	0.60
o3	33.65	52.29	43.82	2.00	18.67	94.00	22.00	22.44	20.69	36.00	1.71
Claude Opus4	33.13	57.43	24.73	6.00	33.67	74.00	12.00	26.00	34.39	34.00	0.77
Claude Sonnet3.5	31.23	47.14	37.11	10.00	27.50	70.00	18.00	32.00	27.52	28.00	2.55
Gemini-2.5-pro	29.94	50.15	38.88	0.00	28.67	52.00	30.61	21.62	30.38	38.00	1.44
GPT-4.1	29.20	29.71	37.45	2.00	7.33	80.00	33.33	29.26	32.90	36.00	0.20
o4-mini	28.55	39.71	36.36	2.08	12.67	76.00	40.00	20.00	31.51	30.00	0.00
GPT-5-mini	27.71	47.71	30.55	4.00	29.33	34.00	28.00	22.44	27.24	47.82	1.01
Qwen3-vl-235b-a22b-thinking	23.34	46.36	25.21	6.00	17.03	25.00	20.40	22.00	29.32	38.00	0.00
Gemini-2.5-flash-lite	16.00	36.29	22.55	4.00	6.67	0.00	30.00	25.00	32.25	26.00	0.25

Numbers are transcribed from Table 1 of the original; recomputing column-wise averages reproduces most rows, but the GPT-4.1 and Qwen2.5VL-7B-SFT rows differ slightly from the reported averages (⚠️ refer to the original paper).

Ablation Study¶

The first ablation tests whether the model is genuinely reading the images or leaning on textual priors: instruction templates are paraphrased in bulk with GPT-5.2 (⚠️ model name as in the original) into semantically equivalent but syntactically richer versions, and the SFT model is re-evaluated; the second setting keeps the original text instructions but replaces the molecular images with random ones. T1-T9 denote translation, rotation, zooming, residue-ligand interaction, translation + rotation, rotation + rotation, docking, interaction location, and pocket-ligand interaction (T4 in the original table uses the merged residue-ligand protocol).

Config	T1	T2	T3	T4	T5	T6	T7	T8	T9
Qwen2.5VL-7B-SFT (original templates)	99.8	99.7	27.1	78.1	88.4	89.5	57.8	88.3	10.7
Paraphrased instructions (GPT-5.2 rewrite)	99.7	99.6	29.4	76.9	89.2	87.8	53.9	87.4	10.2
Random images (text kept, images replaced)	33.5	16.1	5.0	0.0	26.0	27.4	13.9	25.2	0.0

A second analysis examines the effect of the number of views, hydrogen-bond complexity, and the zooming range.

Analysis	Setting	Score
Number of views (rotation)	three views / single view	99.71 / 97.51
Number of views (pocket-ligand interaction)	six views / 27 views	10.72 / 23.77
Hydrogen-bond count vs. accuracy (residue-ligand positive)	0 / 1 / 2 / 3 bonds	89.52% / 64.72% / 28.70% / 20.51%
Zooming error rate (40-60 Å range)	mean / highest / lowest	72.5% / 86.5% / 42.9%

Per-bin sample counts for the hydrogen-bond analysis are roughly 1,746 / 1,094 / 223 / 39 (⚠️ read off Figure 4; refer to the original paper).

Key Findings¶

Every frontier VLM falls far below human level: the best closed-source model, Claude Sonnet4.5, averages 34.37 against the human 81.18, and GPT-4.1, o4-mini, GPT-5-mini, and Gemini-2.5-flash-lite all land in the 16-30 band. The widest gap is pocket-ligand interaction — humans score 82.78 while every model sits between 0.00 and 2.55, effectively a total failure.
Models systematically do better on "distance-like" tasks than on "angle / axis-like" tasks: translation generally beats rotation (o3: 52.29 vs 43.82) and interaction location beats rotation + rotation (GPT-5-mini: 47.82 vs 22.44). The authors attribute this to current VLMs being trained predominantly on 2D data, where distance is a 2D attribute and transfers more readily, whereas rotation about an axis demands genuine 3D mental simulation.
Negative samples are far easier than positive ones: on residue-ligand negatives (no hydrogen bond) o3 reaches 94.00 and Claude Sonnet4.5 84.00, against 18.67 and 22.33 on positives. In negative samples the residue and ligand are often simply far apart, so spatial proximity alone suffices and no chemistry is needed.
The fine-tuned 7B model beats every commercial VLM and surpasses humans on rotation-class tasks: average 62.96, rotation 99.71 (humans 70.18), rotation + rotation 89.59 (humans 26.00, close to the four-option chance level). The authors conclude that advanced VLMs already possess spatial cognitive potential and that their earlier low scores stem from a domain adaptation barrier — they lack visual priors for specialized structures such as proteins, and suitable fine-tuning can build the cross-domain mapping. Yet positive hydrogen-bond recognition (63.46 vs human 100.00) and pocket-ligand interaction (10.72 vs 82.78) still lag, showing that domain priors missing from base training cannot be supplied by this dataset alone.
Paraphrased instructions barely dent performance, but removing the images is catastrophic: after rewriting the templates, every task stays within about 2 points of the original (T1 99.7 vs 99.8, T5 89.2 vs 88.4), so the high scores are not template memorization; replacing images with random ones drops the model to near chance, with T4 and T9 hitting zero, confirming that textual priors are wholly insufficient and visual input is necessary. The paper also notes that a purely image-only setting is infeasible: several distinct tasks share the same initial visual state, so without text instructions the model cannot know which operation is being asked for.
The number of views matters enormously for complex tasks: cutting three views to one costs almost nothing on elementary tasks (rotation 99.71 → 97.51), showing robust monocular depth perception; but for pocket-ligand interaction, going from 6 to 27 views lifts the score from 10.72 to 23.77, nearly doubling it — occlusion is the main bottleneck on complex molecular interactions, and dense viewpoints restore the missing depth cues.
More hydrogen bonds means harder: accuracy on residue-ligand positives falls from 89.52% (0 bonds, i.e. correctly judging "no hydrogen bond") to 20.51% (3 bonds), so the model cannot find all interactions in a complex hydrogen-bond network. The zooming error rate rises and then falls with depth (mean 72.5%, peak 86.5%, minimum 42.9%); the authors hypothesize that the peak corresponds to a visually critical scale containing the least discriminative structural information, where the attention mechanism parses least efficiently.
Human failure modes differ from model failure modes: humans are strong on unit tasks (translation 100.00, residue-ligand positive 100.00) but struggle with continuous and large transformations — large rotations require maintained spatial continuity and multi-atom tracking, zooming lacks clear reference points so judgements rest on boundary shifts and atomic-density intuition, and consecutive composite operations collapse to near chance through error accumulation and frequent reference-frame shifts (rotation + rotation 26.00). The human bottleneck is working memory and the stability of spatial mental simulation; the model's bottleneck is the coupling between domain knowledge and 3D geometry.
Case study (rotation task, Claude Sonnet 4.5): the model's approach is sound — it treats residues conserved across the structural change as anchors and infers the rotation axis and angle from them. But in the key region it identifies the rearrangement of residues 221 and 225 correctly and still misreads it as "moved slightly backward", when the positional shift of residue 221 alone would indicate a rotation about the y axis. Frontier VLMs thus still lack a reliable spatial reasoning mechanism and need more effective ways to elicit it.

Highlights & Insights¶

Operationalizing "molecular understanding" into four elementary operations across two task tiers makes errors attributable: a collapse on rotation tasks is fully decoupled from "not recognizing what a protein looks like", which is what allows the paper to conclude that the spatial potential exists but the domain knowledge is missing. The same recipe transfers to any continuous operation an expert performs in software — window/level adjustment in medical imaging, image registration in remote sensing, CAD assembly — all of which can be turned into diagnostic benchmarks through the same "elementary operations + unit/composite tasks" decomposition.
Axis gating plus magnitude error replaces exact matching: on continuous-answer tasks exact matching would flatten every model to zero; gating on the axis first and then awarding partial credit by magnitude error preserves resolution and mirrors a real ability hierarchy ("right direction, imprecise estimate"). This metric design is almost directly reusable for any continuous-action evaluation.
Two anti-cheating rules target enumeration exploits on list answers: responses with extra irrelevant interactions are capped at 0.5, and responses with more than twice the ground-truth count score 0. Extraction benchmarks generally lack this line of defence, and it is especially necessary for molecular tasks, where the space of enumerable atom pairs is finite.
View count is set per task type: three views for continuous transformations (preserving scale consistency) and six for interaction tasks (combating overlapping atom labels and occlusion). This is better grounded than a uniform six views, and the later view-count ablation (10.72 → 23.77) delimits exactly where that judgement holds.
A counter-intuitive finding: humans are not a reliable upper bound on continuous spatial transformation. Rotation 70.18 vs the model's 99.71, rotation + rotation 26.00 vs 89.59 — human-level performance is itself low on 3D mental simulation, so using human scores as a ceiling underestimates model potential and suggests such tasks are better judged against geometric ground truth than against human voting.
Idea leads: pocket-ligand interaction (models 10.72 / humans 82.78) is the largest blank, and the failure is explicitly attributed to missing domain priors. The natural extension is to model hydrogen-bond geometry explicitly — either as a reward signal that teaches the model to locate donor/acceptor atoms before judging distance and angle, or by tool use, letting the VLM handle localization and semantics while a geometric module makes the precise call, which plugs directly into the mature coordinate-level capability of equivariant graph networks.

Limitations & Future Work¶

Interactions cover hydrogen bonds only: the authors explicitly describe this as a proof-of-concept choice; hydrophobic contacts, π-π stacking, salt bridges, and halogen bonds — equally decisive for binding modes — are excluded, so the "interaction" capability dimension is incompletely covered.
Positive and negative samples are grossly asymmetric in difficulty, inflating averages: almost every model scores 70-94 on residue-ligand negatives but only 7-34 on positives, so a merged average hides the real collapse on positives and small cross-model differences should not be over-interpreted.
Evaluation protocols differ, so cross-model comparisons need care: closed-source and MoE models are evaluated only on MiSI-Bench(tiny) (50 QA pairs per task), the human baseline answers that same subset, while the SFT model's score comes from the full test set; with only 50 samples per task the sampling variance is itself substantial, and the gap between the SFT model and closed-source models mixes in the difference of evaluation sets.
SFT training details are missing: the learning rate, batch size, whether LoRA was used, and the number of epochs are not reported, so how much of the 62.96 average depends on training configuration is unknown and the model cannot be reproduced.
No chirality / mirror-image tasks: the rotation tasks cover only proper rotations, yet the most demanding spatial ability in molecular science is precisely chirality — deciding whether two structures are mirror images or superposable. Adding reflection or configuration-judgement tasks would fill a conspicuous gap in this benchmark.
Depth cues are inherently degraded under orthographic projection: on zooming the model scores 27.14 and humans 30.00, both close to incapable. Orthographic projection has no perspective scaling, so motion along z shows up only as subtle changes in interatomic spacing and occlusion; this may mean the task measures a shared limit of humans and models rather than a VLM-specific weakness, and conclusions could differ with perspective views or depth hints.
Improvement directions: combine equivariant geometric models with VLMs (the VLM produces a coarse answer and semantic judgement, the geometric module refines it precisely); replace the fixed 6 / 27 views with a learnable view-selection policy that lets the model decide which angles to inspect; and inject hydrogen-bond geometric criteria during training explicitly, to test whether domain-knowledge injection can lift the 10.72 of pocket-ligand interaction.

vs macroscopic spatial benchmarks (VSI-Bench / Thinking in Space, MuirBench, LEGO-Puzzles, SAT): these test layout, occlusion, and viewpoint reasoning over everyday objects and video, within a human-scale 3D world; MiSI-Bench tests reconstructing atom-level structure from 2D orthographic projections and inferring physical interactions. The difference is not just a change of scene but a change of requirements: visual elements must land on hierarchical scientific entities, abstract concepts (binding) must be translated into precise geometric constraints, and spatial precision is not negotiable.
vs 3D molecular modelling (force-field methods, 3D convolutions, GNNs, equivariant graph networks and geometric transformers): these take Cartesian coordinates as input and perform representation learning or numerical optimization in coordinate space — accurate, but unable to "look at an image and talk about it", and unable to bring in semantic knowledge from the literature; what this paper measures is the reasoning channel whose interface is images plus natural language, which is naturally complementary to coordinate models. That also explains why the authors name explicit knowledge injection into pre-training as the necessary path toward scientific AGI.
vs molecular software and tool-augmented LLMs (docking tools such as AutoDock Vina, chemistry-tool-augmented LLMs): these excel at numerical optimization or tool invocation on demand but lack visual perception and semantic interpretation of the structure itself; MiSI-Bench's value is quantifying the gap in that channel — for the same "molecular spatial understanding", a different interface yields a completely different ability profile (automated tools far exceed humans at docking, where VLMs score only 10-58).

Rating¶

Novelty: ⭐⭐⭐⭐ First systematic definition and quantification of microscopic spatial intelligence, with solid capability decomposition, three-way perturbation distractors, and anti-cheating metrics; however, no new model or algorithm at the method level, and the SFT probe is routine.
Experimental Thoroughness: ⭐⭐⭐⭐ Ten mainstream VLMs plus a human baseline, three ablation groups (modality, instructions, view count), factor analysis, and a case study; marked down because closed-source models are evaluated only on a small subset, SFT hyper-parameters are undisclosed, and interactions cover hydrogen bonds only.
Writing Quality: ⭐⭐⭐⭐ Concept definitions (orthographic projection, expert skill taxonomy), task templates, and scoring rules are described in enough detail to reproduce; but a few table numbers disagree with the reported averaging convention, and the claim that humans perform worst on docking contradicts the human docking score of 74.54 in the same table.
Value: ⭐⭐⭐⭐⭐ The dataset (163K QA / 588K images) is public and provides a reproducible diagnostic instrument for "can VLMs do science", while clearly pointing to domain-knowledge injection as the next direction.