Skip to content

PsyAgent: Constructing Human-like Agents Based on Psychological Modeling and Contextual Interaction

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/HKUST-FintechLab/PsyAgent
Area: LLM (Other)
Keywords: Large Language Models, Human-like Agents, Big Five Personality, Multi-Scenario Interaction, Controllable Generation

TL;DR

PsyAgent introduces a schema-first framework that couples a Big Five Individual Structure (IS) prior with Multi-Scenario Contexting (MSC) frames across eight arenas, utilizing parameter-efficient SFT and DPO to deliver stable trait fidelity and norm-aware contextual behavior in compact language models.

Background & Motivation

Developing socially competent, human-like autonomous agents requires them to consistently manifest stable dispositional traits while seamlessly adapting to varied interpersonal roles, social relationships, and communicative norms. While the Big Five personality framework provides a robust psychological foundation for describing enduring behavioural tendencies, prevailing persona-conditioned language agents frequently suffer from prompt brittleness, persona drift, or complete collapse into an unexpressive, generic assistant stance over extended interactive horizons and domain transitions.

This tension between trait stability and contextual adaptivity stems from fundamental architectural limitations in existing paradigms. Prior approaches either oversimplify persona conditioning into shallow demographic labels or free-form descriptive promptsโ€”which heavily risk reinforcing social stereotypesโ€”or completely lack explicit socio-structural scaffolding to govern situation-specific etiquette, authority hierarchies, and conversational stakes. Furthermore, compact language models face severe data scarcity regarding context-rich, psychologically grounded demonstrations, leaving them persistently trailing behind massive general-purpose instruction-tuned models on persona fidelity benchmarks.

To resolve the divergence between stable personality priors and dynamic situational demands, PsyAgent adopts a schema-first approach that formalizes the trait-context interface into two auditable, machine-usable representations: Individual Structure (IS), which encodes biographical trajectories and cultural-economic capital, and Multi-Scenario Contexting (MSC), which catalogs role-relationship-norm frames spanning eight everyday arenas. The core idea is to explicitly couple Big Five trait priors with high-dimensional individual life-course structures and structured multi-arena social norms, utilizing synthesized supervision and two-stage SFT-DPO alignment to enable compact models to maintain robust personality fidelity while strictly adhering to local social affordances.

Method

Overall Architecture

The PsyAgent pipeline comprises three tightly coordinated modules: Individual Structure (IS) profile generation, Multi-Scenario Contexting (MSC) frame retrieval across eight arenas, and a two-stage parameter-efficient fine-tuning (PEFT via LoRA/QLoRA with SFT and DPO) alignment regimen. Given a target Big Five percentile vector, the framework first constructs an expansive, privacy-preserving biographical background via IS, couples it with relevant relational frames from MSC, formulates unified persona prompts with dedicated control tokens, and optimizes compact backbones to produce contextually appropriate, trait-consistent first-person behavior.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Target Big Five Vector<br/>OCEAN percentile specification"] --> B["Individual Structure Profile<br/>Disentangled multi-domain background"]
    A --> C["Multi-Scenario Contexting<br/>Eight arena role and norm catalog"]
    B --> D["Unified Persona Prompt<br/>Bound with control tokens and instructions"]
    C --> D
    D --> E["Two-Stage Adapter Tuning<br/>LoRA SFT and DPO preference alignment"]
    E --> F["Controlled Generation and Evaluation<br/>Role-play and IPIP-NEO-120 assessment"]

Key Designs

1. Individual Structure (IS): De-stereotyped Biographical and Cultural Profile Scaffolding

Addressing the fragility and stereotyping risks associated with flat demographic prompting, IS operationalizes persona background through a structured four-domain schema: (1) Educational Trajectory, detailing institutional stages, specialization, pedagogy, and critical turning points; (2) Life Experience, capturing developmental origins, milestone psychological events, travel exposure, and enduring hobbies; (3) Socioeconomic Context, recording family dynamics, social network capital, and intergenerational mobility trajectories; and (4) Cultural Capital, partitioned into embodied dispositions, objectified aesthetic tastes, and institutional qualifications. IS translates numeric Big Five priors into fine-grained autobiographical memory cues without leaking numeric labels or superficial demographic stereotypes, thereby grounding long-horizon stability.

2. Multi-Scenario Contexting (MSC): Role-Relationship-Norm Scaffolding Across Eight Arenas

To prevent behavioral drift and inappropriate responses when switching between disparate social settings, MSC establishes a comprehensive library of interaction frames covering eight everyday arenas: Working Interactions, Family Interactions, Friendship & Informal Socialization, Interactions with Strangers, Solitary Reflection, Romantic & Intimate Communication, Learning & Intellectual Engagement, and Public Communication. Each frame is formally instantiated as a typed tuple: $\(\langle\text{arena}, r, c, n, s, \text{subskills}, \text{template}\rangle\)$ where \(r\) denotes social roles, \(c\) captures counterpart power and affiliation structures, \(n\) specifies salient norms (e.g., deference, reciprocity, confidentiality), \(s\) represents relational and task stakes, and subskills provide canonical communicative strategies. MSC explicitly equips the agent with situational affordances, ensuring the same psychological profile manifests context-appropriate behaviors across diverse settings.

3. Two-Stage PEFT and Hard-Negative Preference Optimization (SFT & DPO): Dual Alignment for Style and Norms

To verify learnability beyond simple prompt engineering, PsyAgent constructs an extensive synthetic supervision corpus by taking the Cartesian product of curated IS profiles and MSC frames (\(I \times M\)). Parameter-efficient adapters (LoRA/QLoRA) are attached to frozen compact backbones. In the supervised fine-tuning (SFT) phase, the model minimizes the length-normalized cross-entropy over response tokens following the <RESP> delimiter: $\(\mathcal{L}_{\text{SFT}}(\theta) = -\frac{1}{|\tilde{y}|}\sum_{t=1}^{|\tilde{y}|}\log p_\theta(\tilde{y}_t \mid x, \tilde{y}_{<t})\)$ In the subsequent Direct Preference Optimization (DPO) stage, the model is trained on chosen-rejected pairs derived from identical prompt contexts. While chosen completions \(y^+\) correspond to high-fidelity SFT responses, rejected instances \(y^-\) are sampled across five targeted hard-negative recipes (score leakage with bullet lists, meta-assistant disclaimers, scene misalignments, trait-tone flips, and under-informative brevity): $\(\mathcal{L}_{\text{DPO}} = -\log \sigma\left(\beta\left(\ell_\theta(y^+ \mid x) - \ell_\theta(y^- \mid x)\right) - \beta\Delta_{\text{ref}}\right)\)$ This alignment systematically suppresses assistant-like hedging, eliminates explicit trait regurgitation, and sharpens persona-grounded decision consistency.

Loss & Training

During SFT, all prompt tokens and metadata prefixes are masked out, ensuring loss computation is exclusively applied to response tokens. In DPO, length-normalized log-likelihood \(\ell_\theta(y \mid x)\) mitigates verbosity bias, with \(\beta\) modulating preference margins against the frozen reference model. At evaluation time, deterministic greedy decoding (do_sample=False) is enforced across all stages. Evaluation follows a two-stage protocol: generating a first-person persona self-description from a target OCEAN vector, followed by completing the 120-item IPIP-NEO-120 inventory under a fixed system prompt embedding the generated description. Raw scores are mapped onto a uniform percentile space \([0, 100]^5\).

Key Experimental Results

Main Results

Under matched inference controls and identical scoring pipelines, PsyAgent is evaluated against standard instruction-tuned backbones using Mean Absolute Error (MAE5), Root Mean Squared Error (RMSE5), Profile Accuracy (ProfileAcc = 100 - MAE5), and Cosine Similarity \(\cos(p, t)\) in percentile space.

Model Method / Condition RMSE5 โ†“ MAE5 โ†“ ProfileAcc โ†‘ \(\cos(p, t)\) โ†‘
Llama-3.2-1B Baseline (w/o PsyAgent) 58.72 49.88 50.12 0.84
Llama-3.2-1B PsyAgent (with) 45.22 38.19 61.81 0.81
Llama-3.2-3B Baseline (w/o PsyAgent) 33.52 28.75 71.25 0.84
Llama-3.2-3B PsyAgent (with) 31.68 27.82 72.18 0.85
Vicuna-7B Baseline (w/o PsyAgent) 46.27 38.99 61.01 0.83
Vicuna-13B Baseline (w/o PsyAgent) 32.12 27.35 72.65 0.86
Qwen3-4B Baseline (w/o PsyAgent) 39.84 32.91 67.09 0.75
Gemma-4B Baseline (w/o PsyAgent) 34.94 29.48 70.52 0.83
OLMo-32B Baseline (w/o PsyAgent) 39.73 32.92 67.08 0.82

Ablation Study

Component ablations and negative controls conducted on Llama-3.2-3B equipped with PsyAgent:

Block Removed Component / Control Condition ProfileAcc Note
IS+MSC Full Configuration (no removal) 72.18 Benchmark reference
IS Ablation โˆ’Educational Trajectory 65.12 Significant drop (-7.06)
IS Ablation โˆ’Life Experience 64.20 Drops by 7.98 points
IS Ablation โˆ’Socioeconomic Context 57.87 Largest single drop in IS (-14.31)
IS Ablation โˆ’Cultural Capital 65.77 Drops by 6.41 points
MSC Ablation โˆ’Working Interactions 68.37 Loss of professional hierarchy norms
MSC Ablation โˆ’Romantic and Intimate Communication 63.71 Impairs delicate emotional calibration
MSC Ablation โˆ’Learning and Intellectual Engagement 62.32 Largest drop among MSC arenas (-9.86)
Control Length-matched profile-misaligned donor IS 55.73 Confirms semantic grounding over prompt length

Key Findings

  • Substantial Gains on Compact Backbones: Incorporating PsyAgent produces an 11.69 point increase in ProfileAcc on Llama-3.2-1B (rising from 50.12 to 61.81), outperforming larger untreated baselines such as Vicuna-7B and OLMo-1B. On the 3B model, performance reaches 72.18, rivaling 13B models.
  • Reverse Scaling in Baseline Prompting: On the common baseline prompt surface, larger models like Qwen3-30B and Gemma-27B underperform their smaller variants due to stronger safety/neutrality priors that induce generic, hedged assistant stances under ambiguous role specifications.
  • Criticality of Socioeconomic Context: Eliminating Socioeconomic Context results in the steepest decline among all IS domains (-14.31 ProfileAcc points), highlighting that family background and social trajectory form the bedrock of realistic trait expression.

Highlights & Insights

  • Dual Socio-Psychological Grounding: Decoupling invariant personality dispositions (IS) from situational norms (MSC) provides a principled resolution to persona drift and stereotype propagation.
  • Rigorous Length-Matched Donor Control: By demonstrating that a prompt-length-matched donor with conflicting semantic content yields a depressed score of 55.73 (comparable to full IS removal), the authors conclusively prove that semantic alignment, rather than token padding, drives performance.
  • Modular Post-Training Pipeline: The two-stage SFT+DPO protocol utilizing hard-negative filtering recipes establishes a reusable, parameter-efficient pipeline for role-playing and conversational alignment in edge deployment.

Limitations & Future Work

  • Reliance on Synthetic Supervision: The ISร—MSC dataset is synthesized via curated prompting and rules, which may carry linguistic artifacts and lack the organic spontaneity, subtext, and colloquial nuances of human interactions.
  • Proxy Metric Dependency: Psychometric readout relies on simulated IPIP-NEO-120 survey completion, which partially captures formatting and instruction-following fidelity rather than direct real-world psychological constructs.
  • Cultural Specificity of Norm Catalog: The current MSC schema mirrors predominantly Western social conventions, necessitating targeted re-authoring when porting to distinct cultural frameworks.
  • vs Big5-Chat (Li et al., 2024): Big5-Chat controls isolated personality traits at high/low binary poles; PsyAgent achieves joint 5-dimensional continuous percentile conditioning across 8 social arenas, achieving +1.12 higher target-trait ProfileAcc in head-to-head evaluations.
  • vs Character-LLM (Shao et al., 2023): Character-LLM focuses on memorizing idiosyncratic historical or fictional figures; PsyAgent introduces a generalized, reusable psychological schema that does not require dedicated fine-tuning for each new persona.

Rating

  • Novelty: โญโญโญโญโญ Pioneering schema-first architecture decoupling Big Five dispositional priors from multi-arena social norms.
  • Experimental Thoroughness: โญโญโญโญโญ Comprehensive evaluation across 1,000 profile configurations, complete ablations, length-matched negative controls, PersonaGym, and blinded human studies.
  • Writing Quality: โญโญโญโญโญ Exceptional clarity, rigorous statistical framing, and complete disclosure of prompt surfaces and hyperparameters.
  • Value: โญโญโญโญโญ Establishes a highly effective, data-efficient engineering and theoretical foundation for persona-consistent, norm-compliant autonomous agents.