Skip to content

Persona-CAPS: Controllable and Auditable LLM Personality via Retrieval over Situation–Behavior Contingencies

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/caps-research/persona-caps
Area: LLM Reasoning
Keywords: LLM personality control, Cognitive-Affective Personality System (CAPS), situation-behavior contingencies, Experience Sampling Method (ESM), auditable and repairable persona

TL;DR

Persona-CAPS departs from monolithic trait prompts and opaque activation vectors by grounding LLM personality in Mischel and Shoda's Cognitive-Affective Personality System (CAPS): it represents a persona as a corpus of human Experience Sampling Method (ESM) if-then contingencies retrieved per situation, delivering continuous trait control via mixture weights without trait labels, while supporting per-response audit traces and atom-level repairs at ~1/3 the edit footprint of paragraph baselines.

Background & Motivation

Current LLM persona interfaces divide sharply into two paradigms: persona-as-string and persona-as-vector. The string-based family—spanning trait labels, character cards, Simple-Prompt, and paragraph descriptions like P2—offers broad model-agnostic usability and strong behavioral steering, yet it provides no per-response trace from generated text back to specific persona constituents, no atom-level edit surface, and no situation-conditional structure. Conversely, the vector-based family—exemplified by internal activation steering (NPTI, persona vectors) and situated neuron steering (IRiS)—anchors personality mechanistically in latent representations, but completely sacrifices human-readability and closed-model commercial deployment. Both paradigms achieve continuous strength modulation, but neither provides an interface suited for enterprise agent governance where persona behaviors must be inspected, audited, and surgically rectified post-deployment.

This representational bottleneck produces severe operational failures in mission-critical deployments such as digital companions, social simulation agents, and customer-facing enterprise assistants. When an agent exhibits aberrant or non-compliant persona behaviors, practitioners using prompt-based methods are forced to regenerate or append complex monolithic prompts, risking uncalibrated semantic drift across unrelated personality facets. Crucially, paragraph-based representations exhibit an insidious "dilution masking effect": when conflicting behaviors are injected into a long prompt paragraph, the intact high-level trait declaration dilutes the contradictory cues, allowing corrupted personas to pass standard psychometric sanity checks undetected. Meanwhile, internal activation steering remains completely opaque to human operators who cannot identify or modify individual behavioral rules in closed-source proprietary backbones.

To resolve this impasse, this work turns to classical personality psychology. In 1995, Mischel and Shoda established the Cognitive-Affective Personality System (CAPS), demonstrating that human personality invariance does not reside in unconditional, static trait averages, but in cross-situational behavioral signatures: "if situation \(S\), then behavior \(B\)." Core idea: decompose LLM persona into a retrievable corpus of if-then contingencies mined from empirical Experience Sampling Method (ESM) data, condition inference on situation-matched behavioral atoms via deployer-controlled mixture weights \(\pi_d\) without explicit trait labels, and leverage per-response fired-atom logs to establish a closed-loop, atom-level audit-and-repair interface.

Method

Overall Architecture

Persona-CAPS models an agent's persona as an empirical corpus \(\mathcal{C} = \{a_1, \dots, a_n\}\) of if-then atoms. Each atom \(a = (\text{situation}, \text{behavior}, [\text{affect}], [\text{goal}], [\text{dim}], [\text{pol}])\) is structured as two natural-language sentences accompanied by optional metadata, binned into 10 distinct (dimension, polarity) cells \(\{C_z\}_{z=(d, p)}\) across the Big Five factors. At test time, given a runtime prompt query \(q\), a dense retriever encodes only the query and each candidate atom's situation field using a lightweight encoder (all-MiniLM-L6-v2, 384-dim, L2-normalized) and retrieves the top-\(k\) (default \(k=4\)) most relevant atoms via cosine similarity. The agent renders these retrieved atoms into a fixed system prompt template that deliberately contains zero trait labels, inducing personality purely through concrete behavioral precedents. Every activated atom is persistently recorded into a fired-atom log alongside its empirical ESM observation provenance, producing a fully auditable and editable execution trace.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Runtime Situation query<br/>User interaction or evaluation prompt"] --> B["ESM Contingency Mining & Label-Free Retrieval<br/>all-MiniLM-L6-v2 encodes situation field only for Top-k"]
    B --> C["Mixture-Weight Continuous Strength Modulation<br/>Deployer weights π regulate active cell sampling"]
    C --> D["Atom-Level Audit & Surgical Repair Loop<br/>Fired-atom log trace · corruption localization · single-row edit"]
    D --> E["LLM Generates Response y<br/>Logged with empirical ESM provenance metadata"]

Key Designs

1. ESM Contingency Mining & Label-Free Retrieval: Eliminating Circular Leakage and Monolithic Black Boxes

Conventional trait prompting relies on explicit abstract descriptors (e.g., "You are an extremely extraverted person"), prompting LLMs through surface-level instruction-following rather than authentic behavioral grounding. Persona-CAPS constructs its 166-atom benchmark corpus from the openESM dataset 0060_beck (Beck and Jackson, 2022). The mining pipeline stratifies participants by trait quartiles, intersects the Situational Eight DIAMONDS perception taxonomy with 18 objective context flags against specific behavioral items, retains contingency pairs displaying significant effect-size separation (\(|d_{\text{high}} - d_{\text{low}}| \ge 0.3\)), and verbalizes them via MiniMax-M2.7 into first-person if-then atoms. Crucially, the retriever indexes only the situation field, forcing the model to perform a cross-target match between the query and the situational trigger, strictly preventing circular behavior-to-behavior retrieval. The system prompt contains no Big-Five keywords; the persona emerges spontaneously from concrete behavioral exemplars, maintaining zero lexical contamination with evaluation inventories (max trigram-Jaccard \(\le 0.134\)).

2. Mixture-Weight Continuous Strength Modulation: Enabling Fine-Grained Trait Intensity Without Fine-Tuning

Human personality exists on a continuous spectrum rather than in binary poles. Persona-CAPS operationalizes continuous intensity control by defining the generation distribution as a hierarchical mixture over sampling pipelines rather than an internal likelihood. For a targeted dimension \(d\), the deployer defines mixture weights \(\pi_d = [w_{\text{high}}, 1 - w_{\text{high}}]\). On each query \(q\), the active polarity cell is drawn according to \(z_d \sim \text{Categorical}(\pi_d)\), followed by deterministic top-\(k\) retrieval \(R \sim P_\tau(R \mid q, C_{z_d})\) and model sampling \(y \sim P_\theta(y \mid q, \sigma(R))\):

\[P_\theta(y \mid q, \mathcal{C}, \pi_d) = \sum_{p \in \{\text{high}, \text{low}\}} \pi_{d, p} \cdot P_\theta\big(y \mid q, \sigma(\operatorname{TopK}_k(q, C_{(d, p)}))\big)\]

This design anchors continuous control along the frequency axis of behavioral expressions across interactions, precisely realizing CAPS theory's formulation of personality as stable cross-situational behavioral signatures. In joint 5-dimensional composition mode, the agent independently retrieves \(k_{\text{per\_dim}}\) atoms from each specified trait cell and concatenates them, achieving robust multi-dimensional profile steering without inter-trait semantic interference.

3. Atom-Level Audit & Surgical Repair Loop: Resolving Dilution Masking and High-Footprint Drift

When an LLM exhibits inappropriate behavioral bias due to verbalization misinterpretations (e.g., misreading the codebook phrase "Others charge" as "Others attack me" rather than "Others lead", silently inverting the polarity of two high-extraversion atoms), monolithic paragraph methods are structurally incapable of localized debugging. Persona-CAPS introduces two complementary audit workflows: CAPS-A (Incident-driven, per-response triage) flags candidate atoms by matching the situation of fired atoms against specific test items exhibiting directional disagreement; CAPS-B (Scheduled corpus-wide sweep) employs an auxiliary judge to audit the stored polarity metadata of every atom in batch. Once corrupted atoms are localized, operators execute a surgical atom-level edit by modifying only the behavior string of the flagged JSONL rows, restoring intended behavior with minimal footprint and zero collateral drift on orthogonal dimensions.

Benchmark & Evaluation Protocol

The framework is rigorously benchmarked across 6 distinct language models: 3 commercial closed-weight backbones (MiniMax-M2.7-highspeed, GPT-4o-2024-08-06, Gemini-2.5-Flash) and 3 open-weight models (Llama-3-8B-Instruct Q4_K_M, Qwen3.5-9B Q4_K_M, DeepSeek-R1-Distill-Qwen-14B Q4_K_M). Evaluation includes: 1. 20-item Mini-IPIP Inventory: 5-point Likert scale across Big Five factors, measuring target dimension spread (\(M_{\text{high}} - M_{\text{low}}\)); 2. PersonalityBench (NPTI): 459 open-ended scenario questions scored 1–5 (scaled to /10) by a held-constant uniform GPT-4o judge across 4,500 total responses; 3. Controlled Corruption and Repair Audit: Controlled injection of \(K \in \{5, 10, 15\}\) polarity-inverted atoms into the corpus, evaluating localization precision (\(P\)), recall (\(R\)), post-fix recovery rate (Fix), replaced character count (Footprint), and off-target dimension drift.

Key Experimental Results

Main Results: PersonalityBench 3×3 Head-to-Head Matrix

Under a held-constant uniform GPT-4o judge (\(T=0\)), Persona-CAPS is compared head-to-head against Deng et al. (2025)'s verbatim Simple-Prompt (SP) and P2 paragraph-prompt baselines across 3 backbones and 5 traits (50 questions per cell):

Backbone Trait Dimension CAPS /10 Simple-Prompt (SP) /10 P2 Paragraph /10 CAPS − SP CAPS − P2
MiniMax-M2.7 Agreeableness (A) 9.64 9.60 9.00 +0.04 +0.64
Conscientiousness (C) 9.00 7.76 8.53 +1.24 +0.47
Extraversion (E) 9.72 9.30 9.43 +0.42 +0.29
Neuroticism (N) 9.22 8.68 9.56 +0.54 -0.34
Openness (O) 8.78 6.74 9.28 +2.04 -0.50
Average 9.27 8.42 9.16 +0.86 +0.11
Gemini-2.5-Flash Agreeableness (A) 9.50 9.66 9.64 -0.16 -0.14
Conscientiousness (C) 9.26 8.14 8.82 +1.12 +0.44
Extraversion (E) 9.76 9.52 9.56 +0.24 +0.20
Neuroticism (N) 9.36 8.84 9.78 +0.52 -0.42
Openness (O) 9.34 7.52 9.42 +1.82 -0.08
Average 9.44 8.74 9.44 +0.71 0.00
Llama-3-8B Q4 Agreeableness (A) 9.38 9.64 9.54 -0.26 -0.16
Conscientiousness (C) 9.38 8.86 9.32 +0.52 +0.06
Extraversion (E) 9.80 9.32 9.62 +0.48 +0.18
Neuroticism (N) 8.36 8.30 9.64 +0.06 -1.28
Openness (O) 8.88 7.20 9.54 +1.68 -0.66
Average 9.16 8.66 9.53 +0.50 -0.37

Controlled Corruption, Localization, and Repair Audit

On MiniMax-M2.7-highspeed, controlled polarity corruption (\(K \in \{5, 10, 15\}\)) is injected into 4 sanity-passing cells, evaluating CAPS-A, CAPS-B, and P2 repair workflows:

Cell (n) Audit Workflow Precision P Recall R Fix Success Rate Replaced Characters (Footprint) Off-target Drift
high-E (n=6) CAPS-A (Sparse Triage) 1.00 0.33 5 / 6 378 (4 atoms) 0.19
CAPS-B (Exhaustive Sweep) 0.92 1.00 6 / 6 1449 (12.5 atoms) 0.23
P2-Oracle (Ground Truth Bound) 1.00 1.00 6 / 6 ~320 words (Full Paragraph) 0.48
P2-Blind (Full Regeneration) — — 6 / 6 ~320 words (Full Paragraph) 0.48
P2-Detect (Sentence Diagnosis) 1.00 1.00 4 / 6 ~320 words (Full Paragraph) 0.29
high-N (n=3) CAPS-A 1.00 0.40 1 / 3 544 (6 atoms) 0.35
CAPS-B 1.00 1.00 2 / 3 1328 (15 atoms) 0.48
P2-Oracle 1.00 1.00 1 / 1 ~320 words 0.31
P2-Blind — — 1 / 1 ~320 words 0.31
P2-Detect 1.00 1.00 1 / 1 ~320 words 0.38
low-A (n=2) CAPS-A 0.42 0.15 0 / 2 326 (2 atoms) 0.50
CAPS-B 0.92 1.00 1 / 2 1549 (12.5 atoms) 0.12
P2-Oracle / Blind / Detect 1.00 1.00 0 / 2 ~320 words 0.56 / 0.56 / 0.81
low-O (n=9) CAPS-A 0.80 0.44 6 / 9 538 (4 atoms) 0.35
CAPS-B 0.68 1.00 4 / 9 1563 (10 atoms) 0.40
P2 Baselines (Dilution Masking) — — 0 / 9 (Undetected) Gate Failed (0/9 Sanity Pass) —

Key Findings

  • Universal Superiority over Simple-Prompt: Across all 3 backbones, Persona-CAPS consistently beats Simple-Prompt by an average of +0.69/10, with massive gains on Conscientiousness (+0.96) and Openness (+1.85)—precisely the two dimensions that prior literature notes simple trait labels struggle to express.
  • Parity with Paragraph Baselines on Commercial LLMs: On commercial backbones, CAPS matches P2 within 0.11 points (0.00 gap on Gemini, +0.11 on MiniMax). The aggregate deficit on Llama-3-8B (-0.37) is driven by Q4 quantization inducing verbatim copying of (felt: ...) metadata; stripping affect metadata (--strip-affect-metadata) recovers 33% of the gap, elevating L8B score to 9.28.
  • Dilution Masking in Paragraph Representations: In low-openness (low-O) settings, P2 suffers an absolute sanity-gate failure (0/9 pass rate). Even when \(K=15\) inverted sentences are stitched into the paragraph, the surrounding trait label dilutes the perturbation, rendering the corruption completely invisible to psychometric screening. In contrast, CAPS exposes the corruption clearly through situational retrieval.
  • 1/3 Edit Footprint with Minimal Collateral Damage: CAPS-A restores target persona performance by editing only ~378 characters (roughly 1/3 of P2's footprint) with an off-target drift of just 0.19 (vs. 0.48 for P2-Blind, a 60% reduction in collateral drift). On a cross-vendor replication with GPT-4o, the flagged atom sets and repaired character counts are bit-for-bit identical across 13/13 paired seeds.

Highlights & Insights

  • Engineering Mischel's CAPS Theory into RAG Architecture: Rather than treating personality as a monolithic vector or rigid label, grounding personality in empirical situation-behavior contingencies enables authentic, label-free personality emergence.
  • Strictly Monotonic Continuous Trait Tuning: Sweeping the mixture weight \(\pi_d \in [0, 1]\) achieves strictly monotonic control (\(\text{Spearman } \rho = +1.000\)) across all 15 commercial vendor-by-dimension cells, providing a well-behaved operational knob for personality strength.
  • From Prompt Black Magic to Auditable Data Governance: By linking every response to its underlying ESM observation via the fired-atom log, prompt debugging is transformed from guesswork into a principled localize-intervene-verify engineering pipeline.

Limitations & Future Work

  • Performance Drop on Adversarial All-Low (LLLLL) Profiles: In the extreme setting where all five traits are set to low polarities (directly opposing the model's RLHF prosocial prior), CAPS-k=4 direction match lags P2 by 20 percentage points due to the absence of explicit heavy-handed trait directives; adaptive anti-prior retrieval budgets represent a promising remedy.
  • Systematic Response-Length Differences: Due to concise prompt instructions, CAPS responses are shorter than baseline responses (e.g., 33 words vs. 116 for SP on Gemini); although partial correlation and matched-length bootstrap analyses confirm that length does not explain CAPS's gains, future work should incorporate length-matched decoding.
  • Demographic Scope of Empirical Corpus: The 166 atoms were mined from a US undergraduate ESM cohort; the underlying situational triggers and behavioral reactions reflect specific cultural assumptions, requiring localized re-mining for cross-cultural deployments.
  • vs. Monolithic Trait Prompting (Simple-Prompt, PersonaAgent, DPRF, P2): Prompt baselines rely on static character cards or trait descriptions, providing no per-response trace, failing under dilution masking, and requiring full-paragraph rewrites. Persona-CAPS offers situation-contingent retrieval, label-free prompts, and surgical atom-level repair.
  • vs. Activation-Steering Approaches (Persona Vectors, NPTI, IRiS): Activation methods manipulate model-internal activations, which are incompatible with proprietary closed-source APIs and uninterpretable to end-users. Persona-CAPS operates at the textual API surface, combining closed-model compatibility with complete human-readability.
  • vs. Conventional Personalization RAG (PersonaRAG, PersonaAI): Prior personalization RAG systems retrieve user-specific historical facts or conversational logs to mimic an individual. Persona-CAPS inverts the unit (retrieving psychological if-then contingencies) and target (instantiating general Big-Five trait dispositions), establishing an auditable framework for behavioral steering.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ (Pioneering integration of CAPS psychological theory with retrieval architectures, establishing the first auditable and repairable LLM personality framework)
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ (Exemplary 6-backbone evaluation spanning diagnostic inventories, open benchmarks, controlled corruption loops, and rigorous cross-family judge replications)
  • Writing Quality: ⭐⭐⭐⭐⭐ (Clear conceptual framing, rigorous mathematical formalization of the mixture surface, and transparent analysis of failure modes and null results)
  • Value: ⭐⭐⭐⭐⭐ (Provides an immediate, production-ready blueprint for compliance-critical enterprise agent governance, companion simulation, and safety alignment)