Skip to content

Adaptive Knowledge Routing for Complex Reasoning in LLMs

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/yanqiyao1/Propo-MI
Area: LLM Reasoning
Keywords: Logical Reasoning, Mechanistic Interpretability, Causal Intervention, Attention Head Specialization, Propositional Logic

TL;DR

This paper conducts systematic causal intervention and probing across four LLMs, revealing that propositional logical reasoning is not an opaque pattern-matching heuristic, but rather an internally organized system of four interlocking mechanisms: staged computation, boundary token aggregation, fact retrospection, and specialized attention heads.

Background & Motivation

Recent iterations of large language models have achieved impressive breakthroughs across formal reasoning and complex deduction benchmarks, with frontier models reaching 92% accuracy on GPQA-Diamond and 76% on SimpleBench. However, the exact neural computational mechanisms driving these empirical successes remain fundamentally opaque. This opacity fuels an ongoing debate within the community: are LLMs merely sophisticated stochastic parrots exploiting shallow token co-occurrences and lexical heuristics, or have they internalized robust, algorithmic deduction procedures that mirror formal logic? The emerging field of Mechanistic Interpretability (MI) has reverse-engineered transformer subgraphs in arithmetic addition, syllogistic deduction, and entity-attribute binding, yet an overarching mechanistic picture of propositional reasoning has remained elusive.

Propositional logic, governing boolean operators, premise manipulation, and multi-step rule composition, constitutes the foundational bedrock of all higher-order deductive systems. Behavioral evaluations consistently show that model accuracy degrades sharply as proof depth expands, yet where and why internal representations break down remains unexplained. Pioneering mechanistic work by Hong et al. (2024) discovered specialized attention heads in implication reasoning, but their analysis was strictly restricted to single-hop implication chains, bypassing broader boolean rules like De Morgan's laws, distributive laws, and multi-step composition. Furthermore, traditional circuit discovery focuses primarily on component-level necessity within isolated task instances, lacking a strategy-level perspective that explains how components coordinate network-wide across architectures, rules, and inference depths.

To overcome this fundamental gap, this work shifts analytical perspective from identifying task-specific circuits to uncovering generalizable, strategy-level computational mechanisms across four models (Qwen3-8B/14B, Llama-3.1-8B-Instruct, Mistral-7B-Instruct). The authors design PropLogic-MI, a controlled benchmark spanning 11 propositional rule categories across one-hop and two-hop deduction evaluated under both premise-first (\(\pi_{\text{FF}}\)) and expression-first (\(\pi_{\text{EF}}\)) prompt layouts. Core idea: Pretrained LLMs solve propositional logical reasoning via four interlocking mechanisms: Staged Computation across network depth, Information Transmission converging at semantic boundary tokens, Fact Retrospection maintaining active premise lookup under heavy loads, and Specialized Attention Heads structurally executing these discrete operations.

Method

Overall Architecture

The framework applies a macroscopic-to-microscopic causal intervention methodology on the residual stream, MLP layers, and multi-head attention blocks. Starting from pairs of clean prompts (\(x_{\text{clean}}\)) and minimally corrupted prompts (\(x_{\text{corr}}\)) where a factual truth assignment is flipped to alter the output label, the authors deploy mean ablation and activation patching to track when, where, and how logical deduction is computed across layers and token positions.

The overall reasoning pipeline internally processes inputs through four coordinated phases: partitioning input clauses, routing token representations to boundary tokens, dynamically binding variables to truth values while retroactively querying facts during hard reasoning, and integrating truth values at the final query token.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    In["Input Propositional Problem<br/>F: Fact premises, E: Logical expressions, Q: Query"] --> Stage1["Staged Computation<br/>Early: Fact encoding -> Middle: Expression evaluation -> Late: Query integration"]
    Stage1 --> Stage2["Information Transmission<br/>Semantic aggregation at boundary tokens: fact_value, expr_last, query_last"]
    Stage2 --> Stage3["Fact Retrospection<br/>Middle-layer binding flow + sustained late-layer fact retrieval under hard loads"]
    Stage3 --> Stage4["Specialized Attention Heads<br/>Splitting heads + Transmission heads + Fact-Retrieval heads"]
    Stage4 --> Out["Output Answer Token<br/>True / False"]

Key Designs

1. Staged Computation: Temporal Functional Division Across Network Depth Component-level circuit studies often isolate individual layers without revealing how the network schedules deduction over its full depth. To uncover global scheduling, the paper applies mean ablation to residual streams, MLP outputs, and attention outputs across three coarse logical blocks: Fact Block, Expression Block, and Query Block. By tracking the probability difference shift (\(d\text{PD}\)), the experiments demonstrate clear functional staging: early layers encode factual premises, middle layers execute boolean expression calculations, and late layers synthesize query information for final answer selection. Strikingly, this temporal progression persists under prompt order reversal (\(\pi_{\text{EF}}\), where expressions precede premises), proving that the staged execution reflects internal algorithmic deduction rather than a trivial artifact of surface token order.

2. Information Transmission: Semantic Aggregation at Discrete Boundary Tokens Rather than disseminating reasoning states uniformly across all input positions, efficient transformer deduction requires compact routing bottlenecks. Using fine-grained per-token residual stream activation patching across fine structural categories, the authors discover that causal influence sharply condenses onto semantic boundary tokens: fact_value tokens dominate early layers, expr_last tokens dominate middle layers, and query_last tokens dominate late layers, while interior variable names and operators remain near zero causal influence. Boundary tokens act as discrete semantic hubs that summarize intermediate state for each clause before dispatching it forward to subsequent computational stages.

3. Fact Retrospection: Dual-Mode Premise Access and Capacity Coupling Standard intuition assumes that once premises are encoded into hidden representations in early layers, input fact tokens become causally inert. This paper uncovers "Fact Retrospection", where fact tokens remain actively queried across middle and late depths through two complementary modes: - Middle-layer binding flow: In intermediate layers, causal influence shifts dynamically from fact_value to fact_var_ref, reflecting attention heads using variable references as queries to copy bound truth assignments into expression evaluation. - Sustained late-layer access: Under demanding reasoning loads (such as two-hop deduction or difficult rules), models cannot compress all factual state into a single forward pipeline; consequently, fact_value maintains a statistically significant causal effect into the latest layers. Quantitatively, Late Fact Restoration (LFR) strongly correlates with task difficulty (\(\bar{r} = 0.449, p = 0.0004\)).

4. Specialized Attention Heads: Tripartite Operator Specialization To identify the microscopic operators realizing the macroscopic patterns, the authors classify attention heads by geometric attention patterns and validate them using top-\(k\) mean ablation: - Splitting Heads: Concentrated in early layers (e.g., Layers 0–1 in Qwen3-8B), these heads attend sharply to delimiter tokens (such as commas), structurally demarcating clause boundaries to instantiate Staged Computation. - Transmission Heads: Distributed across middle layers (Layers 7–9), these heads exhibit lower-triangular intra-clause attention maps that route information within facts and expressions toward their respective closing boundary tokens. - Fact-Retrieval Heads: Spanning middle to late layers (Layers 26–31), these heads attend from query tokens or expression variable references back to factual truth values (True/False), physically executing variable binding and late-layer retrospection.

Benchmark & Evaluation Protocol

The empirical investigation is conducted on PropLogic-MI, an author-curated corpus of 20,000 clean/corrupted problem pairs balanced across 11 propositional logic rules (Identity, Domination, Idempotent, Double Negation, Excluded Middle, Contradiction, Commutative, Associative, Distributive, De Morgan, Absorption) and two reasoning depths (9,523 one-hop and 10,477 two-hop instances). Clean and corrupted prompts share logical structure but flip a single factual assignment, allowing clean metric formulation.

Evaluations measure the shift in Probability Difference (\(d\text{PD}\)) over answer tokens \(w_y \in \{\text{True}, \text{False}\}\): $\(\text{PD}(x, y) = P(w_y \mid x) - P(w_{\neg y} \mid x)\)$ $\(d\text{PD} = \text{PD}(\tilde{x}, y) - \text{PD}(x_{\text{baseline}}, y)\)$ To eliminate spurious guessing, all mechanistic analyses are strictly computed on the Dual-Correct pool (samples where the unintervened model correctly solves both the clean and corrupted instances).

Key Experimental Results

Main Results: Zero-Shot Performance and Dual-Correct Rates

Zero-shot behavioral performance on PropLogic-MI across prompt orders (\(\pi_{\text{FF}}\) vs. \(\pi_{\text{EF}}\)) defining the candidate pools for mechanistic investigation:

Model Prompt Order Clean Acc (%) Corrupted Acc (%) Dual-Correct Rate One-Hop Acc (%) Two-Hop Acc (%)
Qwen3-8B \(\pi_{\text{FF}}\) 86.98 88.17 75.44% (15,087/20,000) 86.35 87.55
Qwen3-14B \(\pi_{\text{FF}}\) 87.42 89.11 76.55% (15,310/20,000) 92.64 82.68
Qwen3-8B \(\pi_{\text{EF}}\) 74.22 79.25 53.89% (10,777/20,000) 79.09 69.79
Qwen3-14B \(\pi_{\text{EF}}\) 74.39 77.91 52.80% (10,561/20,000) 88.34 61.72
Llama-3.1-8B \(\pi_{\text{FF}}\) 63.9 - 38.4% (7,674/20,000) - -
Mistral-7B \(\pi_{\text{FF}}\) 65.1 - 35.1% (7,022/20,000) - -

Ablation Study: Causal Impact of Targeted Attention Head Ablation

Evaluating post-ablation top-1 accuracy (Acc) and mean probability difference shift (\(d\text{PD}\)) on clean held-out sets when ablating top-\(k\) heads from each specialized role against random unassigned heads (Random-Outside):

Model & Setting Baseline Acc \(k=8\) FR Acc / \(d\text{PD}\) \(k=8\) TR Acc / \(d\text{PD}\) \(k=8\) SP Acc / \(d\text{PD}\) \(k=8\) Rand-Out Acc / \(d\text{PD}\) \(k=32\) FR Acc / \(d\text{PD}\) \(k=32\) Rand-Out Acc / \(d\text{PD}\)
Qwen3-8B (\(\pi_{\text{FF}}\)) 96.6% 74.6% / -0.40 80.4% / -0.29 84.6% / -0.21 97.1% / +0.01 57.0% / -0.74 97.2% / +0.02
Qwen3-8B (\(\pi_{\text{EF}}\)) 92.8% 77.4% / -0.26 78.0% / -0.26 81.2% / -0.19 94.7% / +0.02 63.2% / -0.54 92.5% / -0.02
Qwen3-14B (\(\pi_{\text{FF}}\)) 94.8% 83.2% / -0.24 76.8% / -0.34 90.4% / -0.09 96.5% / +0.02 63.8% / -0.58 98.3% / +0.05
Qwen3-14B (\(\pi_{\text{EF}}\)) 97.6% 86.0% / -0.23 79.8% / -0.33 93.4% / -0.09 98.0% / +0.00 77.2% / -0.42 98.1% / +0.01
Llama-3.1-8B (\(\pi_{\text{FF}}\)) 60.2% 55.2% / -0.16 55.2% / -0.17 55.2% / -0.18 62.4% / +0.03 55.2% / -0.16 72.0% / +0.12
Mistral-7B (\(\pi_{\text{FF}}\)) 99.4% 41.6% / -0.21 45.2% / -0.14 47.0% / -0.11 95.9% / +0.02 41.2% / -0.28 88.4% / +0.07

Note: FR = Fact-Retrieval Heads, TR = Transmission Heads, SP = Splitting Heads, Rand-Out = Random-Outside Heads.

Key Findings

  • High functional specificity of head roles: In Qwen3 models, ablating as few as 8 or 32 Fact-Retrieval or Transmission heads collapses accuracy dramatically (e.g., Qwen3-8B FR drops accuracy from 96.6% to 57.0% at \(k=32\)), whereas ablating 32 non-specialized Random-Outside heads produces zero degradation (97.2%), verifying that these circuits are causal bottlenecks.
  • Hierarchical importance of head roles: Across all configurations, the severity of degradation follows Fact-Retrieval \(\ge\) Transmission \(>\) Splitting, showing that truth-value routing is the most critical link in the deductive chain.
  • Cross-family modularity variation: Qwen3 exhibits sharp role modularity and clean separation from baseline controls. In contrast, Mistral-7B shows global sensitivity where even random ablations degrade performance, and Llama-3.1-8B saturates at low \(k\), indicating that circuit modularity scales with internal reasoning capacity.

Highlights & Insights

  • Invariance to physical prompt layout: Reversing the prompt layout (\(\pi_{\text{EF}}\)) retains the identical layer-wise staging (early Facts, middle Expressions, late Query). This proves that transformers execute deduction via an internal computational schedule rather than passively reflecting the left-to-right reading order.
  • Fact retrospection under capacity limits: Discovering that late-layer representations re-engage with input premise tokens under multi-hop complexity provides a mechanistic explanation for why transformers fail on deeper chains—working memory capacity saturates when intermediate facts cannot be fully compressed into forward residual streams.
  • Strategy-level interpretability paradigm: By connecting macroscopic layer ablations, mesoscopic boundary routing, and microscopic head specializations, the authors offer a unified blueprint for reverse-engineering complex deduction in LLMs.

Limitations & Future Work

  • Opaque MLP transformations: While the paper localizes when MLP blocks are causally vital, the exact algebraic transformations computed inside feed-forward neurons (e.g., boolean truth-table lookups or feature rotations) remain uncharacterized.
  • Controlled synthetic scope: Experiments use synthetic propositional logic instances with 1-2 inference hops. Validating whether these four mechanisms govern multi-step natural language deduction, predicate logic, or real-world mathematical reasoning remains an open direction.
  • Exclusion of failure trajectories: The causal analyses are restricted to the Dual-Correct pool to ensure fidelity to successful computation. Consequently, the exact mechanistic breakdowns that trigger deduction errors in failed runs require dedicated follow-up study.
  • vs. Hong et al. (2024): Hong et al. discovered four attention head families in Mistral/Gemma restricted to single-hop implication chains; this work scales to 11 boolean rule categories, multi-hop deduction, and constructs a holistic four-mechanism strategy framework.
  • vs. Dutta et al. (2024): Dutta et al. studied how Chain-of-Thought (CoT) verbalizations anchor reasoning steps; this paper shows that even in zero-shot direct generation without an explicit CoT scaffold, models internally execute a highly organized multi-stage deductive algorithm.
  • vs. Feng & Steinhardt (2024): Feng and Steinhardt demonstrated entity-attribute binding via ID vectors; this paper discovers the logical deduction counterpart where variable references actively retrieve bound truth assignments in intermediate layers.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Establishes a comprehensive four-mechanism strategy framework for propositional deduction.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous causal interventions spanning four models, 11 logic rules, 1-2 hops, and bidirectional prompt controls.
  • Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally structured narrative with precise mathematical formulations and insightful conceptual abstractions.
  • Value: ⭐⭐⭐⭐⭐ Highly influential for understanding formal reasoning inside transformers and mitigating hallucination in logical execution.