Skip to content

Simulating Society Requires Simulating Thought

Conference: NeurIPS 2025 arXiv: 2506.06958 Code: None Area: Interpretability Keywords: social simulation, cognitive modeling, causal reasoning, belief graphs, reasoning fidelity

TL;DR

This paper proposes a paradigm shift from "behaviorism" to "cognitive modeling" in LLM-based social simulation. The GenMinds framework models the internal reasoning processes of LLM agents via causal belief graphs, and the RECAP benchmark evaluates reasoning fidelity along three dimensions: traceability, demographic sensitivity, and intervention consistency.

Background & Motivation

Background: LLMs are increasingly employed for social simulation. The dominant approach relies on persona prompting or RLHF to elicit human-like responses from agents.

Limitations of Prior Work: Existing methods remain confined to a behavioristic paradigm. Although agent outputs are fluent, they lack internal causal reasoning, belief traceability, and counterfactual reasoning capabilities. Three core failure modes are identified: untraceable reasoning, insensitivity to counterfactuals, and consensus hallucination.

Key Challenge: Superficial alignment at the output level does not imply structural alignment at the reasoning level. Autoregressive architectures optimize next-token likelihood rather than belief-state transitions.

Goal: To equip LLM agents in social simulation with structured, revisable, and traceable belief reasoning capabilities.

Key Insight: Drawing from cognitive science, human reasoning exhibits three core properties: causality, compositionality, and revisability.

Core Idea: Replace token-level generation with cognitively inspired causal belief graphs, enabling agents to simulate thought rather than merely simulate language.

Method

Overall Architecture

The paper proposes GenMinds (a modeling framework) and RECAP (an evaluation framework). The pipeline proceeds as follows: natural language interviews → LLM parsing → cognitive motif extraction → causal Bayesian network construction → neuro-symbolic hybrid inference → belief propagation and intervention simulation.

Key Designs

  1. Cognitive Motifs:

  2. Function: Extract minimal causal reasoning units from natural language.

  3. Mechanism: LLM-assisted parsing of interviews to identify concept nodes and directed causal relations. Cross-individual aggregation forms a shared cognitive motif topology.

  4. Design Motivation: Modular, reusable reasoning units support cross-domain generalization.

  5. Causal Belief Networks (CBN):

  6. Function: Compile cognitive motifs into directed acyclic graphs.

  7. Mechanism: Nodes encode concepts; edges encode directional causal influence. Supports do-calculus for counterfactual intervention simulation.

  8. Design Motivation: Explicit causal graphs render reasoning paths traceable and interpretable.

  9. Neuro-Symbolic Hybrid Reasoning:

  10. Function: Combine the flexibility of LLMs with the rigor of symbolic reasoning.

  11. Mechanism: LLMs select interventions and assemble motifs into the CBN; probabilistic updates simulate belief dynamics.

  12. RECAP Evaluation Framework:

  13. Three dimensions: motif alignment, belief consistency, and counterfactual robustness.

Key Experimental Results

Proof of Concept

Scenario Intervention Variable Pre-intervention Post-intervention
Urban surveillance do(Transparency=high) Privacy Concern 0.7 0.3
Urban surveillance do(Transparency=high) Opposition 0.7 0.2

Paradigm Comparison

Dimension Existing Paradigm GenMinds
Reasoning Format Token-level generation Structured belief graphs
Belief Dynamics Static or reset Causally updatable and revisable
Evaluation Perspective Output fluency Reasoning fidelity
Social Representation Averaged and flattened Diverse, positional cognition

Key Findings

  • CoT performance collapses under distributional shift.
  • Behavioral convergence in multi-agent systems produces consensus hallucination.
  • Demographic conditioning leads to identity flattening.

Highlights & Insights

  • The paradigm shift from behaviorism to cognitivism is a perspective transferable to many LLM evaluation settings.
  • The concept of cognitive motifs elegantly decomposes reasoning into minimal, reusable causal units.
  • The distinction between "structured inconsistency" and "unstructured inconsistency" is a valuable analytical contribution.

Limitations & Future Work

  • The work lacks empirical validation; it remains at the level of conceptual design and illustrative examples.
  • Causal graph construction faces challenges such as conceptual ambiguity.
  • Human reasoning encompasses not only causal inference but also associative and analogical reasoning.
  • Acquiring semi-structured interview data entails high annotation costs.
  • vs. Generative Agents: Still output-oriented; this paper critiques such systems for producing averaged group behavior.
  • vs. CoT/ReAct: The paper argues these constitute post-hoc rationalization rather than genuine reasoning.
  • vs. RLHF: Optimizes behavioral alignment rather than cognitive alignment.

Rating

  • Novelty: ⭐⭐⭐⭐ The cognitive science perspective demonstrates genuine originality.
  • Experimental Thoroughness: ⭐⭐ No quantitative experiments are provided.
  • Writing Quality: ⭐⭐⭐⭐⭐ Argumentation is clear and well-structured.
  • Value: ⭐⭐⭐⭐ The paradigm shift carries long-term impact.