LatentChem: From Textual CoT to Latent Thinking in Chemical Reasoning

Conference: ICML 2026
arXiv: 2602.07075
Code: Yes (GitHub + HuggingFace public as declared in paper, though link not in footnote)
Area: LLM Reasoning / Scientific Computing (Chemical LLM)
Keywords: Chemical reasoning, latent thinking, continuous thought vectors, GRPO, modality mismatch

TL;DR

LatentChem replaces "explicit textual CoT chains" with "continuous latent thought vectors + dynamic molecular perception updates" in chemical LLMs. Under pure outcome-based GRPO rewards, the model spontaneously abandons textual CoT in favor of latent reasoning, achieving a 59.88% non-tie win rate over the explicit CoT baseline on ChemCoTBench, with a 10.84x reduction in reasoning steps and a 5.96x wall-clock speedup.

Background & Motivation

Background: Current chemical LLMs (e.g., domain-tuned models based on Qwen-3 + SMI-TED encoders) almost exclusively follow the "explicit CoT" route—forcing continuous physicochemical intuitions like electron delocalization, steric hindrance, and functional group modifications into linearized natural language token chains before producing an answer. Benchmarks like Mol-Instructions and ChemCoTBench also adopt textual CoT as the default reasoning interface.

Limitations of Prior Work: The inherent landscape of chemical properties is a high-dimensional continuous manifold (where properties like LogP or QED change smoothly with structure), whereas textual tokens are discrete, low-bandwidth symbols. Translating a structural operation like "adding a methyl group"—which could be represented in one step—into dozens of tokens analyzing binding sites and valence bonds creates what the authors call a "jagged staircase" trajectory. This is both slow (explosion in reasoning steps) and prone to hallucinations over long chains.

Key Challenge: There is a natural modality mismatch between continuous physicochemical dynamics and discrete linguistic symbol channels. Textual CoT is neither the native carrier of chemical logic nor necessarily the optimal computational medium. Although existing "latent reasoning" works (like Coconut) move reasoning into the latent space, they treat molecular embeddings as static contexts, failing to refocus on different substructures during the reasoning process.

Goal: To construct a specialized latent reasoning interface for chemistry to answer two questions: (1) When allowed to reason in a continuous latent space without being forced to output text, which modality will the model spontaneously choose? (2) Is this choice a "shortcut" or a truly superior computational strategy?

Key Insight: Decoupling reasoning from generation. Natural language serves only as the I/O interface, while intermediate reasoning follows a cycle of "continuous thought vectors + dynamic perception refresh." Using GRPO to reward only format validity, structural effectiveness, and answer correctness—without penalizing brevity or incentivizing the omission of CoT—allows the model to decide whether to use text autonomously.

Core Idea: Constructing a "perception-reasoning" dual-pathway via a ChemUpdater for dynamic re-querying of molecular representations and a latent projector for closed-loop hidden state feedback. Under outcome-oriented RL, the model spontaneously internalizes CoT into the latent space.

Method

Overall Architecture

LatentChem attaches three chemistry-specific modules to a general LLM backbone (default Qwen-3-8B): (1) A Chemical Adapter compresses variable-length molecular features from the SMI-TED encoder into fixed-length "ChemTokens" soft prompts; (2) A latent thinking loop inserts several steps of continuous hidden states after the ChemTokens and text instructions, feeding them back into the model without decoding into text; (3) A ChemUpdater uses the latest thought vector to cross-attend to historical sequences and refresh ChemTokens, allowing the model to "take another look at the molecule" during reasoning. The process is trained via a 4-stage progressive schedule (Alignment → SFT CoT → Latent Activation → GRPO Outcome Reward). In the final stage, latent modules are frozen while the backbone is optimized to treat the learned latent dynamics as an "internal simulator."

Key Designs

1. Chemical Adapter (Perceiver Resampler-style Molecule-Language Alignment):

   • Function: Compresses variable-length dense molecular features \(\mathbf{H}_{mol}\in\mathbb{R}^{L\times d_{enc}}\) from SMI-TED into \(N\) fixed-length "ChemTokens" injected as soft prompts.
   • Mechanism: Learns \(N\) latent queries \(\mathbf{Q}\) to perform cross-attention on molecular features: \(\mathbf{H}_{chem}=W(\text{LN}(\mathbf{Q}+\text{MHA}(\mathbf{Q},\mathbf{H}_{mol},\mathbf{H}_{mol})))\), where each query acts as a "semantic anchor" for specific chemical attributes. Stage 1 uses "answer-only" supervision with a counterfactual alignment loss \(\mathcal{L}_{CF}\) (hinge loss maximizing the likelihood difference between "clean" and "perturbed" molecules) to force the adapter to compress necessary properties into ChemTokens.
   • Design Motivation: Unlike traditional prefix tuning which just "plugs in an embedding," the authors require a molecular representation container that can be dynamically re-queried during latent thinking. It must be fixed-length and faithfully reflect structure for the subsequent ChemUpdater to refocus based on reasoning states.

2. ChemUpdater (Dynamic Perception Refresh):

   • Function: Re-queries and refreshes ChemTokens after each latent reasoning step based on the accumulated thought history, shifting the model's focus on the molecule as reasoning progresses.
   • Mechanism: Uses current ChemTokens \(\mathbf{H}_{chem}^{(t)}\) as queries and all historical thoughts \(\mathbf{Z}_{1:t}\) as keys/values for cross-attention: \(\mathbf{H}_{chem}^{(t+1)}=\text{LN}(\mathbf{H}_{chem}^{(t)}+\text{CrossAttn}(\mathbf{H}_{chem}^{(t)},\mathbf{Z}_{1:t},\mathbf{Z}_{1:t}))\). This upgrades the encoder from a "one-time static extractor" to a "differentiable perceiver scheduled by reasoning states."
   • Design Motivation: This is the core difference from Coconut. Coconut treats molecular embeddings as invariant context, causing the latent chain to drift further from the original structure as it grows. ChemUpdater lets the model "look at the molecule again, but at different parts," which is crucial for tasks like molecular optimization requiring repeated sub-structure localization (ablation shows a 12% drop in SR without it).

3. Latent Projector + GRPO Outcome-Only Reward (Trigger for Spontaneous Internalization):

   • Function: Maps the raw hidden state \(\mathbf{z}_t\) back to the input embedding space as \(\mathbf{h}_{t+1}=\mathbf{z}_t+\text{FFN}(\text{LN}(\mathbf{z}_t))\), bypassing the tokenization bottleneck. Reasoning continues until <end_latent> is emitted or the budget \(T_{max}\) is reached. Stage 4 uses GRPO rewards for format, validity, and correctness.
   • Mechanism: The projector is a lightweight residual FFN. Crucially, the reward contains no terms for brevity or CoT omission. Despite this, LatentChem spontaneously abandons textual CoT, transitioning to pure latent reasoning (outputting only a "." or ":" transition token before generating the XML answer).
   • Design Motivation: The authors treat latent reasoning as an observable experimental instrument. By allowing free choice of modality and using only outcome rewards, they verify the "modality mismatch" hypothesis—if continuous latents are more compatible with chemical logic, the model will choose them. Causal ablation (replacing the first \(k\) latents with Gaussian noise) proves these "silent steps" perform substantive computation.

Loss & Training

Four-stage progressive training: Stage 1 (Adapter+LLM training, latent disabled, answer-only + counterfactual alignment \(\mathcal{L}_{total}^{(1)}=\mathcal{L}_{clean}+\lambda\mathcal{L}_{CF}\)); Stage 2 (Same modules, adding explicit CoT \(\mathbf{y}_{full}=[\mathbf{y}_{cot},\mathbf{y}_{ans}]\)); Stage 3 (Freeze adapter+LLM, train ChemUpdater + Projector to generate thought vectors compatible with the fixed semantic space); Stage 4 (Freeze latent modules, optimize backbone with GRPO using a composite reward = format + validity + correctness). Training data uses a 14k CoT sample snapshot (2025-11) from ChemCoTDataset covering molecular understanding, editing, optimization, and reaction prediction.

Key Experimental Results

Main Results

Task	Metric	LatentChem	Stage 1+2+4 (Explicit CoT)	Coconut-Chem	Δ vs CoT
Mol Optim. LogP	SR%	96	77	44	+19
Mol Optim. GSK3-β	SR%	82	67	47	+15
Mol Optim. Solubility	SR%	89	86	58	+3
ChEBI-20 Desc.	METEOR	0.15	0.05	0.07	+0.10
ChemLLMBench Desc.	METEOR	0.16	0.07	0.04	+0.09
ChemCoTBench All	Win Rate \(\mathcal{R}^*_{win}\)	59.88%	— (Baseline)	32.11%	Significant
ChemLLMBench All	\(\mathcal{R}^*_{win}\)	55.58%	—	44.82%	Significant
ChEBI-20 Open	\(\mathcal{R}^*_{win}\)	85.26%	—	65.58%	Significant
Mol-Instructions All	\(\mathcal{R}^*_{win}\)	49.88	—	40.57	Near Tie

LatentChem (8B) outperforms the Claude 3.7 Sonnet SOTA reported by ChemCoTBench in molecular optimization. Reasoning steps decreased by 10.84x on average (up to 29.9x for reaction tasks), with a 5.96x wall-clock speedup.

Ablation Study

Configuration	Mol Optim. SR% ↑	Mol Description METEOR ↑
LatentChem (Full)	80.67	0.143
w/o ChemUpdater	68.67 (−12.0)	0.068 (−0.075)
w/o Latent Projector	69.83 (−10.8)	0.087 (−0.056)
w/o Latent Thinking	71.00 (−9.67)	0.052 (−0.091)

Scale-matched tests: Re-implementing LatentChem with 4B and 14B backbones yielded 52.42% and 51.80% win rates on ChemCoTBench All against explicit CoT of same scale, confirming the advantage is not due to parameter size.

Key Findings

• Spontaneous CoT Internalization: Despite Stage 1-3 supervision via explicit CoT, once Stage 4 introduces outcome-only rewards, the model stops generating intermediate text, emitting only a transition token before the XML answer. This "internalization" is strategy migration driven by rewards, not loss-based enforcement.
• Causal Necessity: Replacing the first \(k\) latent steps with Gaussian noise leads to significant performance drops, proving these steps perform necessary encoding rather than being redundant.
• "Hydraulic Trade-off" in Budget Stress Tests: When the latent budget \(T \geq 6\), the model generates almost no textual CoT. When \(T < 6\), the model reactivates textual CoT to compensate for the lack of reasoning depth—showing dynamic scheduling between implicit and explicit modalities.
• Functional Partitioning of the Latent Manifold: t-SNE shows that initial representations are entangled but rapidly split into task-specific clusters within 2 steps, stabilizing by step 10. RSA indicates that latent geometry maintains stable Spearman correlation with Tanimoto chemical topology, suggesting latent updates are orthogonal to structural encoding and do not destroy physical fidelity.

Highlights & Insights

• Latent Reasoning as an Experimental Instrument: The authors do not assume latents are better; they construct an interface allowing free choice and observe behavior under GRPO. This "evidence from model choice" is highly persuasive.
• Transferability of Dynamic Perception: Upgrading the encoder into a "differentiable perceiver queried by reasoning states" is a paradigm likely applicable to proteins, 3D geometry, or code ASTs.
• Internalization Without Brevity Penalty: Triggering internalization via outcome-only rewards is a counter-example to "reward hacking = performance drop." The model chooses shorter outputs because latents are more native than text for these tasks.
• Architectural Hint for Cognitive Systems: The fallback to textual CoT under tight latent budgets aligns with the Kahneman System 1/2 framework, positioning this as a prototype for future "hybrid cognitive architectures."

Limitations & Future Work

• Interpretability Collapse: Latent reasoning is opaque to humans. Post-hoc analysis confirms physical grounding, but intermediate molecular modification steps cannot be audited—a hurdle for scientific applications requiring verifiable reasoning.
• Mol-Instructions Tie: The win rate of 49.88% indicates that latent advantages are task-dependent and not a universal "silver bullet."
• Training Stability: GRPO stability and reward weight robustness across tasks were not fully explored.
• Architectural Dependency: Experiments rely heavily on the Perceiver Resampler and SMI-TED; transferability to other encoders (e.g., Uni-Mol) remains unproven.
• Future Direction: Hybrid System 1/2 architectures—using latents for structural computation and decoding to language when justifications are needed. Reverse distillation of latent thoughts into human-readable CoT is another potential extension.

Related Work & Insights

• vs. Coconut (Hao et al., 2025c): Coconut uses latent reasoning with static molecular embeddings; LatentChem uses ChemUpdater for dynamic refresh. On ChemCoTBench, Coconut-Chem reaches only 32.11% win rate vs. LatentChem's 59.88% on the same backbone.
• vs. Explicit Chemical CoT: In a head-to-head with the same GRPO process but forced textual CoT, LatentChem's +9.88pp gain is purely attributable to the latent interface.
• Insight: Using outcome-only GRPO as a "modality selection probe" is a generalizable methodology—it can verify which modality is more suitable for any task \(T\) by letting the model choose under reward.

Rating

• Novelty: ⭐⭐⭐⭐⭐ Spontaneous internalization + ChemUpdater dynamic perception is a first in chemical LLMs.
• Experimental Thoroughness: ⭐⭐⭐⭐ Includes 4 benchmarks, 3 baselines, scale-matched tests, and causal ablations. One star off for lacking direct comparison with general latent reasoning baselines like Geiping recursive unrolling.
• Writing Quality: ⭐⭐⭐⭐⭐ Clear logic from hypothesis to verification. Intuitive conceptual diagrams (jagged staircase vs. smooth manifold) and stress test visualizations are excellent.
• Value: ⭐⭐⭐⭐ Paradigmatic contribution to chemical LLMs with 5.96x acceleration. Practical value is slightly limited by interpretability issues, pending hybrid System 1/2 follow-ups.

Related Papers

• [NeurIPS 2025] Beyond Chemical QA: Evaluating LLM's Chemical Reasoning with Modular Chemical Operations
• [ACL 2026] Render-of-Thought: Rendering Textual Chain-of-Thought as Images for Visual Latent Reasoning
• [ICML 2026] Efficient Reasoning with Hidden Thinking
• [ICML 2026] Conformal Thinking: Risk Control for Reasoning on a Compute Budget
• [ICML 2026] Prioritize the Process, Not Just the Outcome: Rewarding Latent Thought Trajectories Improves Reasoning in Looped Language Models