HACHIMI: Scalable and Controllable Student Persona Generation via Orchestrated Agents

Conference: ACL 2026
arXiv: 2603.04855
Code: https://github.com/ZeroLoss-Lab/HACHIMI
Area: LLM Evaluation / Education / Multi-Agent Systems
Keywords: Student Persona, Multi-Agent, Neuro-Symbolic Verification, Stratified Sampling, Group Consistency

TL;DR

HACHIMI formalizes "student persona generation" as the TAD-PG (Theory-Aligned and Distribution-Controllable) task. By employing a "propose-validate-revise" multi-agent framework integrated with a neuro-symbolic validator and stratified sampling, the authors generated 1 million synthetic student personas for grades 1–12. Group-level evaluations on CEPS/PISA 2022 reveal a distinct "fidelity gradient": math and curiosity constructs show high alignment, whereas well-being and family dynamics constructs exhibit only weak alignment.

Background & Motivation

Background: Educational Large Language Models (LLMs) for personalized tutoring, virtual classrooms, and teacher training increasingly rely on large-scale "synthetic students" for dialogue simulation and effectiveness evaluation. Traditional methods rely on interviews, questionnaires, or observations to manually construct a few representative personas (HCI personas), which are detailed but not scalable. Recent approaches use LLM "role-playing + one-shot generation" to batch-produce personas, which are scalable but suffer from quality degradation.

Limitations of Prior Work: Pure prompt-based LLM student personas exhibit three systematic flaws: (1) Intra-profile contradictions: Descriptions conflict across long contexts; (2) Lack of theoretical anchoring: Generated "motivations/personalities" rarely correspond to established pedagogical or developmental psychology theories (e.g., Piaget, Erikson, OECD Learning Compass); (3) Uncontrollable group distribution: Proportions of high/low achievers, gender, or psychological risk levels are random, failing to meet the requirement for "evaluation according to real demographic structures." Existing RAG or memory frameworks only mitigate consistency issues without addressing the latter two.

Key Challenge: Synthetic students in education require three hard constraints: theory alignment, group quotas, and intra-individual consistency. These constraints often conflict: strong consistency leads to mode collapse; strong diversity disrupts theoretical constraints; and strict quotas can dilute rare groups. One-shot prompting cannot satisfy all three simultaneously.

Goal: (1) Formally define the Theory-Aligned and Distribution-Controllable Persona Generation (TAD-PG) task; (2) Design a framework that allows LLMs to strictly satisfy educational theories and quotas while maintaining diversity; (3) Perform group-level external validation using large-scale real-world surveys (China's CEPS and international PISA 2022).

Key Insight: The generation process is decomposed into multiple agents responsible for different dimensions of the schema, using a shared whiteboard to synchronize intermediate states and prevent contradictions. Pedagogical theories are hard-coded as executable logical predicates for a "Symbolic Validator" within a "propose-validate-revise" loop. Stratified sampling combined with LSH (Locality-Sensitive Hashing) deduplication is used to combat mode collapse.

Core Idea: Transform the "soft constraints" of prompt engineering into hard constraints via a "propose-validate-revise" loop and neuro-symbolic predicates, treating "quota scheduling" as an external scheduler rather than internal prompt guidance.

Method

Overall Architecture

The HACHIMI pipeline consists of: (1) Target Distribution Input: Specifying quotas for grade, gender, and academic level; (2) Theory-Anchored Schema: Dividing the persona into five components based on the OECD Learning Compass (Demographics & Development, Academic Profile, Personality & Values, Social Relations & Creativity, Mental Health & Well-being); (3) Modular Multi-agent Generation: Each component is authored by an independent agent conditioned on a shared whiteboard; (4) Neuro-Symbolic Validator: Checking for violations against executable rules (R1–R15, e.g., mapping grades to Piaget/Erikson stages) and providing structured feedback for revisions; (5) Stratified Sampling + LSH Deduplication: Ensuring 250,000 samples for each of the four academic levels and removing semantic duplicates using SimHash. The result is HACHIMI-1M (1 million personas generated using Qwen2.5-72B, requiring ~3200 H100-hours).

Key Designs

1. Modular Generation via Shared Whiteboard (Mechanism I):

   • Function: Enables multiple agents to collaborate on a single persona while maintaining consistency across the five components.
   • Mechanism: Personas are split into five components (Demographic / Academic / Personality-Value / Social-Creativity / Mental-Health), each generated by a specialized agent. All agents share a "whiteboard" context; subsequent agents must condition their output on the intermediate products of previous agents. This replaces "one-shot long-context writing" with "incremental accumulation," preventing intra-profile contradictions and allowing agents to specialize.
   • Design Motivation: Self-contradictions occur when LLMs "forget" previous context. An explicit whiteboard externalizes memory, enforcing strong alignment constraints on subsequent agents.

2. Neuro-Symbolic Constraint Satisfaction (Mechanism II, Propose-Validate-Revise):

   • Function: Converts theoretical alignment from subjective LLM judgment into hard, executable rule checks.
   • Mechanism: Developmental and pedagogical axioms are formalized into R1–R15 logical predicates (e.g., Grade 2 must map to Piaget’s "Concrete Operational" stage; moral stages must be a subset of Kohlberg's 6 stages). The Symbolic Validator runs these rules; if violated, a structured error signal (specifying the rule, the field, and the expected value) is returned to the agent for revision. This is a "neuro-creation + symbolic-judge" hardened version of the self-refine approach.
   • Design Motivation: LLMs have high creativity but low theoretical consistency; symbolic systems are rigorous but cannot write narratives. Their combination ensures LLMs write while symbolic systems act as "red-line checkers."

3. Stratified Sampling + LSH Semantic Deduplication (Mechanism III):

   • Function: Prevents LLMs from converging on "average personas" during massive batch generation and ensures rare groups appear according to target quotas.
   • Mechanism: An external stratified sampler draws samples across orthogonal factors (4 academic levels × 12 grades × 2 genders). The "academic level" is propagated as a conditional variable affecting downstream attributes like self-efficacy. Post-generation, SimHash \(h(x) = \text{sign}(W\phi(x))\) maps long narratives to a binary hash space, and items within a Hamming distance threshold are discarded to ensure semantic diversity.
   • Design Motivation: Random sampling naturally over-samples frequent personas due to LLM bias; stratified sampling is a classic statistical tool against bias. LSH is used because traditional n-gram overlap fails to detect semantically identical but lexically varied narratives.

Loss & Training

This framework does not involve training new models. It utilizes Qwen2.5-72B as the generation agents and DeepSeek-V3 as the "student agents" for shadow surveys. The focus is on the inference-time Propose-Validate-Revise loop and the scheduler; thus, there is no loss function, only "constraint satisfaction" as the termination condition.

Key Experimental Results

Main Results: CEPS Grade 8 Group-Level Consistency

HACHIMI personas were instantiated as student agents to perform a shadow survey based on the China Education Panel Survey (CEPS) Grade 8 data. Comparisons were made across 16 cohorts (4 academic levels × 2 genders × 2 psychological risk levels) using 16-dimensional mean vectors.

CEPS Target Construct	Pearson \(r\)	Spearman \(\rho\)	Rating
Educational aspirations (w2b18)	\(\ge 0.86\)	\(\ge 0.90\)	High
Parental achievement expectation (w2a27)	\(\ge 0.86\)	\(\ge 0.90\)	High
Perceived difficulty in Math/English	0.86 / 0.85	0.81 / 0.80	High
Teacher attention (Aggregated)	\(\approx 0.86\)	\(\approx 0.90\)	High
Mother-child relationship (w2a23)	0.73	0.66	Med
Prosocial behavior	—	\(\approx 0.63\)	Med
Misbehavior / Parental pressure	—	Med	Med
School bonding / Depressive symptoms	Weak/Neg	Weak/Neg	Low
Parental strictness	Weak/Neg	Weak/Neg	Low

Generality was verified using PISA 2022 across 5 regions × 16 cohorts: MATHEFF showed \(r > 0.95\) in all regions, CURIOAGR \(r \gtrsim 0.85\), while school climate/belonging were moderate, and mental health/workload stayed near 0 or even flipped signs across regions.

Ablation Study: vs. One-Shot Baseline (10K samples, same protocol)

Metric	One-shot baseline	HACHIMI	\(\Delta\)
Hard error rate ↓	12.03%	0.00%	−12.03
Warning rate ↓	25.33%	0.82%	−24.51
Distinct-1 ↑	0.2328	0.3285	+0.0957
Distinct-2 ↑	0.4589	0.7893	+0.3304
Near-duplicate logs ↓	157	0	−157
CEPS teacher-attention \(\rho\)	base	+0.132	+0.132
PISA MATHEASE \(r\)	0.45–0.63	+0.27–0.29	+0.27

Key Findings

• Fidelity Gradient: Across both CEPS and PISA, "school-facing and observable" constructs (math efficacy, teacher attention, learning interest) show extremely high alignment. "Latent and private/family" constructs (depression, parental strictness, well-being) show weak or even negative correlation. This suggests that inferring psychological latent variables from static personas is inherently more difficult.
• Multi-agent + Neuro-symbolic Validation = Zero Hard Errors: Hard errors dropped from 12% to 0% without using post-processing filters; the "propose-validate-revise" loop forces the agent to correct itself, proving superior to simple RAG or prompt engineering.
• Significant Diversity Gain: Distinct-2 increased from 0.46 to 0.79. Stratified sampling and LSH deduplication nearly doubled phrase-level diversity, proving that default LLM sampling suffers from severe mode collapse.
• Stable Consistency Across Datasets: The ranking of strengths and weaknesses on CEPS was replicated across five PISA regions, indicating the fidelity gradient is a property of synthetic personas rather than a dataset artifact.

Highlights & Insights

• Formalizing theoretical alignment as executable predicates (R1–R15): This makes pedagogical compliance a machine-verifiable and debuggable attribute rather than a subjective feeling. This approach of hard-coding domain knowledge into validators is applicable to LLM data generation in medicine or law.
• Shared Whiteboard as a Lightweight Anti-contradiction Tool: Instead of training specialized consistency models, allowing multiple agents to write sequentially on a shared "scratchpad" with visibility into previous steps reduces intra-profile contradictions to near zero.
• Identification of the Fidelity Gradient: This is a standalone contribution that informs the community on what synthetic students can reliably evaluate (math efficacy, academic expectations) and what is risky (depression, well-being). It establishes a "red line" for claims made using synthetic students.

Limitations & Future Work

• Static vs. Dynamic Students: HACHIMI personas are static states rather than learners evolving over time, missing long-term learning trajectories and micro-scale classroom interaction causality.
• Single Base Model: Agents were primarily based on Qwen2.5-72B and DeepSeek-V3. Changing base models or decoding strategies might alter alignment.
• Simplified Theoretical Schema: Folding mental health and family relationships into limited labels and narratives inevitably loses continuous, spectrum-like differences, which may be the root cause of the fidelity gradient's low performance in these areas.
• Future Directions: Incorporating dynamic learning trajectory modeling (episodic agent state) and multi-base-model ensembles; utilizing "real data augmentation" for low-fidelity constructs.

Related Work & Insights

• vs. MathDial / Book2Dial: Previous works focused on dialogue data with personas as a byproduct; HACHIMI treats the persona as a first-class citizen with explicit quotas and theoretical constraints.
• vs. Generative Agents (Park 2023): While Park et al. used memory and reflection for long-term consistency, HACHIMI uses a shared whiteboard and symbolic critic for batch-generation intra-profile consistency—complementary rather than alternative.
• vs. PPLM / GeDi: Controllable decoding also aims for attribute control but struggles to scale across 5D complex student personas. HACHIMI moves "control" to the agent scheduling layer for better explainability and scalability.

Rating

• Novelty: ⭐⭐⭐⭐ TAD-PG task formalization + first systematic neuro-symbolic validator for persona generation.
• Experimental Thoroughness: ⭐⭐⭐⭐ Dual-layer external validation (CEPS + PISA) + intrinsic schema tests + controlled baseline.
• Writing Quality: ⭐⭐⭐⭐ Mechanisms are clearly explained, and the fidelity gradient conclusion is compelling.
• Value: ⭐⭐⭐⭐ 1 million personas + evaluation framework serves as public infrastructure for the educational LLM community.

Related Papers

• [ACL 2026] SILO-BENCH: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systems
• [ACL 2026] Latent Agents: A Post-Training Procedure for Internalized Multi-Agent Debate
• [ICML 2026] When Cloud Agents Meet Device Agents: Lessons from Hybrid Multi-Agent Systems
• [ACL 2026] ATLAS: Adaptive Trading with LLM AgentS Through Dynamic Prompt Optimization and Multi-Agent Coordination
• [ACL 2026] Memory-Augmented LLM-based Multi-Agent System for Automated Feature Generation on Tabular Data