Personas Differ from Native-Language Generation: Language Pathways Shape LLM Interpersonal Advice¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: Social Computing
Keywords: LLM Personas, Language Pathways, Interpersonal Advice, Behavioral Scaffolding, Cross-Lingual Pragmatics
TL;DR¶
Evaluating cross-lingual advice elicitation across 600 interpersonal dilemmas, 13 languages, and eight leading LLMs reveals that English native-speaker persona prompting (NP) systematically diverges from native-language generation followed by translation (NL), inflating superficial prosocial markers while depleting actionable behavioral scaffolding and shifting choices toward confrontation.
Background & Motivation¶
Large language models are increasingly deployed to advise users on sensitive, socially situated dilemmas, ranging from de-escalating workplace disagreements and asserting boundaries with peers to managing romantic friction and familial obligations. In these interpersonal contexts, language models do not merely produce informational text; they actively shape behavioral choices and provide social scripts. This role is especially pivotal in cross-lingual or culturally mismatched settings where users understand their communicative intent but struggle with target-language pragmatic norms—such as nuanced levels of directness, deference, politeness hierarchies, emotional restraint, and face-saving mechanisms. Yet despite this real-world application, standard multilingual benchmarks remain heavily preoccupied with objective tasks such as translation, question answering, mathematical reasoning, and instruction following, where a single ground truth exists and interpersonal pragmatics can be largely ignored.
To elicit culturally or linguistically contextualized behaviors from LLMs without generating in non-English languages, researchers and practitioners routinely rely on a popular shortcut: prompting the model in English with an explicit native-speaker persona instruction (e.g., "You are a native Japanese speaker..."). This practice operates on the untested methodological assumption that role-play persona prompting in English serves as a functional proxy for processing social dilemmas within the native linguistic pathway of the target language. However, recent social simulation studies suggest that persona prompting often triggers superficial tropes or flat demographic profiles stored within dominant English pretraining distributions, rather than reproducing the authentic pragmatic trade-offs and interpersonal stances embedded in native discourse communities.
This divergence creates a critical methodological tension in computational social science and multilingual conversational systems: does prompting an LLM with a native-speaker persona genuinely access native interpersonal decision-making, or does it merely apply a stylistic mask over an English-centric behavioral model? To resolve this question, this work investigates whether native-language generation followed by back-translation into English (NL) and native-speaker persona prompting directly in English (NP) produce equivalent interpersonal advice. Core idea: cross-lingual elicitation strategy is not a neutral implementation detail; persona prompting systematically inflates surface lexical social markers while stripping actionable behavioral scaffolding and biasing recommendations toward confrontation compared to native-language generation.
Method¶
Overall Architecture¶
The framework systematically disentangles how different language pathways shape LLM advice across 600 socially situated dilemmas sourced from a public interpersonal advice community, spanning four primary domains: workplace interactions, friendship dynamics, romantic relationships, and family conflicts. To guarantee that cross-lingual variations stem from generative pathways rather than translation artifacts, the original English prompts are converted into 12 non-English target languages via an automated semantic preservation pipeline that optimizes back-translation consistency. The study evaluates five distinct language pathways across eight state-of-the-art models (comprising both proprietary APIs and open-weight architectures), analyzing the resulting responses through four complementary lenses: dictionary-based lexical analysis, LLM-based holistic pragmatic scoring, Behavior Change Technique (BCT) scaffolding evaluation, and a forced-choice action categorization task.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
In["Interpersonal Advice Dilemmas<br/>600 situated real-world dilemmas"] --> Des1["Semantically Controlled Multilingual Corpus<br/>NLLB sampling and back-translation variance filtering"]
Des1 --> Des2["Language Pathway Disentanglement<br/>comparing native generation back-translation and persona prompting"]
Des2 --> Des3["Multi-Layer Pragmatic and Scaffolding Evaluation<br/>LIWC lexical, LLM pragmatic, BCT scaffolding, and forced-choice"]
Des3 --> Out["Empirical Findings<br/>NP inflates surface warmth, reduces scaffolding, and favors confrontation"]
Key Designs¶
1. Semantically Controlled Multilingual Corpus: Translation Semantic Drift -> Multi-Candidate NLLB Sampling with Back-Translation Variance Minimization -> Robust Semantic Equivalence Across Language Families
Rigorous cross-lingual comparison requires that all prompts present identical situational stakes, emotional subtleties, and communicative dilemmas across languages without semantic corruption. The authors select 150 top-voted advice questions across four distinct categories (work-environment, friends, relationships, family) from the Interpersonal Skills Stack Exchange, excluding geographically tagged prompts to maintain generalizable interpersonal dilemmas. For each of the 12 target languages (spanning East Asian, Middle Eastern/Turkic, Germanic, and Romance families), 32 translation candidates are generated using facebook/nllb-200-distilled-1.3B at temperature 0.3. Each candidate is back-translated into English 4 times at temperature 0.7, scoring semantic fidelity via all-MiniLM-L6-v2 cosine similarity against the original prompt. The pipeline retains the top-5 candidates by mean similarity and selects the single candidate with the minimum back-translation variance. This yields an overall mean semantic similarity of 0.928 across all languages (ranging from 0.876 in Japanese to 0.976 in Spanish), ensuring that observed behavioral shifts reflect language pathways rather than translation errors.
2. Language Pathway Disentanglement: Conflation of Persona Mimicry and Native Cognition -> Systematic Comparison of NL and NP with Intermediate Controls -> Precise Isolation of Input versus Output Cross-Lingual Pathways
To determine whether persona prompting accurately mirrors native-language processing, the authors systematically formulate and contrast multiple execution pathways across 600 dilemmas. In Native-Language Generation followed by Translation (NL), the model receives the prompt in the target language, generates its complete open-ended advice in that language, and is subsequently translated into English via a standardized second step for unified evaluation. In Native-Speaker Persona Prompting (NP), the model receives the original English dilemma prepended with an explicit persona instruction ("You are a native
3. Multi-Layer Pragmatic and Scaffolding Evaluation: Traditional NLP Benchmarks Inability to Measure Interpersonal Guidance -> Multi-Tier Lexical, Pragmatic, Behavioral Scaffolding, and Forced-Choice Metrics -> Revealing the Disconnect Between Surface Tone and Concrete Action Because interpersonal guidance entails both social positioning and practical execution, evaluation spans four distinct analytical tiers. First, LIWC-22 dictionary analysis measures 12 surface psychological categories, capturing affiliation, positive tone, prosocial language, social references, and power. Second, an LLM-as-a-judge (GPT-4o) rates five holistic pragmatic dimensions on 5-point ordinal scales: directness, formality, emotional expressiveness, social attunement, and concreteness. Third, to measure actionable behavioral utility rather than mere stylistic tone, the authors adapt eight core categories from the Behavior Change Technique (BCT) Taxonomy (goal setting, action planning, social support, information consequences, self-monitoring, problem solving, cognitive reframing, and behavioral feedback). Finally, a forced-choice action classification presents the identical dilemma under each pathway and requires the model to choose exactly one action family: Confrontation (directly engaging the issue or authority), Disengagement (delaying, withdrawing, or creating space), or Redirection (reframing, switching communication channels, seeking mediation, or compromising).
Evaluation Setup¶
The benchmark assesses eight frontier LLMs: five open-weight architectures (google/gemma-4-E4B-it, google/gemma-4-26B-A4B-it, google/gemma-4-31B-it, Qwen/Qwen3.6-27B, Qwen/Qwen3.6-35B-A3B) and three leading proprietary models (claude-opus-4.6, gpt-4o, gpt-5.5). All continuous features are standardized relative to the English baseline for each matched prompt and model. Significance testing utilizes paired-difference t-tests for continuous measures and Pearson chi-square tests for categorical action distributions, with false discovery rates strictly controlled via the Benjamini–Hochberg procedure (\(q < 0.05\)).
Key Experimental Results¶
Main Results¶
Aggregated across 600 dilemmas, 12 non-English languages, and eight language models, NP and NL diverge systematically across pragmatic framing, behavioral scaffolding, and strategic recommendations. The table below presents standardized feature differences (treatment minus English baseline) and direct NP–NL contrasts.
| Feature / Metric | TTG-Eng | NL-Eng | OSP-Eng | NP-Eng | NP-NL Gap | Empirical Implication |
|---|---|---|---|---|---|---|
| Social Attunement | -0.07* | -0.38* | -0.12* | -0.59* | -0.21* | NP shows significantly less relational and empathetic awareness |
| Emotional Expressiveness | -0.03* | -0.30* | -0.07* | -0.48* | -0.18* | NP suppresses emotional nuance and context relative to NL |
| Tentativeness | +0.01 | -0.21* | -0.03* | -0.19* | +0.01 | No significant difference in tentativeness |
| Positive Tone | +0.01* | -0.03* | +0.08* | +0.18* | +0.21* | NP exhibits inflated surface-level positive and prosocial words |
| Composite BCT Scaffolding | -0.09* | -0.54* | -0.32* | -0.98* | -0.44* | NP provides substantially less actionable procedural guidance |
In the forced-choice action classification, language pathways systematically alter the concrete behavioral recommendations prescribed to users. The table below shows the distribution of action choices overall and across broad language families.
| Target Group / Family | Pathway Strategy | Confrontation (%) | Redirection (%) | Disengagement (%) | Key Action Distribution Trend |
|---|---|---|---|---|---|
| Overall (12 Languages) | English Baseline | 34.1 | 52.0 | 13.9 | Default preference for redirection over confrontation |
| Overall (12 Languages) | Translate-then-Generate (TTG) | 35.4 | 50.3 | 14.3 | Closely aligns with English baseline |
| Overall (12 Languages) | Native-Language Gen. (NL) | 28.8* | 57.6* | 13.5* | Strongest shift toward collaborative redirection |
| Overall (12 Languages) | One-Stage English (OSP) | 30.1* | 54.6* | 15.3* | Intermediate between NL and English |
| Overall (12 Languages) | Native Persona (NP) | 32.3 | 53.4 | 14.3 | Significant rebound toward direct confrontation |
| East Asian | Native-Language Gen. (NL) | 23.0* | 61.2* | 15.8* | Highest redirection rate (61.2%), minimal confrontation |
| East Asian | Native Persona (NP) | 28.0* | 57.0* | 15.0* | Confrontation rate surges by +5.0 percentage points |
| Middle Eastern / Turkic | Native-Language Gen. (NL) | 27.5* | 58.7* | 13.8* | Markedly lower confrontation than English baseline |
| Middle Eastern / Turkic | Native Persona (NP) | 31.5* | 55.3* | 13.2* | Shifts back toward confrontation (+4.0 pp) |
| Germanic | Native-Language Gen. (NL) | 34.1* | 54.0* | 11.9* | Higher baseline confrontation than East Asian languages |
| Germanic | Native Persona (NP) | 37.0* | 49.6* | 13.5* | Peak confrontation rate across all groups (37.0%) |
| Romance | Native-Language Gen. (NL) | 30.7* | 56.7* | 12.7* | Moderate confrontation, elevated redirection |
| Romance | Native Persona (NP) | 33.4 | 50.8 | 15.8 | Redirection decreases by nearly 6 percentage points |
Ablation Study & Robustness Checks¶
The authors conducted extensive sensitivity analyses to confirm that the observed NP–NL divergence represents a fundamental property of cross-lingual elicitation:
| Robustness Check | Scope & Condition | Key Findings & Metric Stability |
|---|---|---|
| Persona Wording Sensitivity | 100 prompts, 12 languages, 2 models testing Original NP, "Grew up using", and "Currently uses" | 14 of 17 linguistic features retain identical NP–NL direction; all 8 BCT dimensions remain strictly negative; confrontation rates consistently exceed NL across all formulations. |
| Translation Quality Filter | 4,015 prompt-language pairs with back-translation similarity \(\ge 0.95\) (55.8% retention) | 59 of 60 LLM-annotated contrasts retain their direction (mean absolute change \(\approx 0.03\) z-score); all 96 BCT contrasts remain negative; confrontation and redirection shifts persist in all 12 languages. |
| Evaluator Judge Stability | Independent evaluation using claude-sonnet-5 on 100 prompts across 12 languages and 3 models |
Complete directional agreement on all 5 linguistic dimensions (\(r = 0.89\)); agreement on 7 of 8 BCT dimensions; 12 of 13 dimensions agree even when GPT-4o evaluates its own generated texts (\(r = 0.77\)). |
Key Findings¶
- The Politeness–Scaffolding Paradox: Persona prompting selectively inflates surface-level prosocial markers and positive sentiment (+0.21), but severely degrades actionable guidance (composite BCT drops by -0.44). While NP advice sounds cordial, polite, and supportive on the surface, it lacks concrete implementation details, step-by-step sequencing, and consequence reasoning.
- Systematic Action Shift Toward Confrontation: Across all tested language families and models, native-language generation substantially decreases confrontation (dropping to 28.8%) and elevates redirection (rising to 57.6%). In contrast, persona prompting pushes the model back toward Anglo-centric directness, raising confrontation to 32.3%.
- Cross-Lingual Heterogeneity: The gap between NP and NL is not a uniform shift across language families. While East Asian languages exhibit the largest buffering effect under NL (61.2% redirection), persona prompting flattens and distorts these cultural norms, causing disproportionate shifts depending on the target language.
Highlights & Insights¶
- Methodological Wake-Up Call: Demonstrates conclusively that prompt-based persona simulation is not a faithful substitute for native-language cognitive generation in computational social science or cross-cultural modeling.
- Bridging Pragmatics and Behavioral Decision-Making: Connects linguistic tone (LIWC), discourse pragmatics, and empirical psychology (BCT Taxonomy) to reveal that "how advice sounds" can completely disguise "whether the advice helps the user act."
- Reusable Experimental Pipeline: The multi-sample NLLB filtering framework combined with the four-way pathway control (English, TTG, OSP, NL, NP) establishes a gold-standard protocol for future studies of multilingual LLM alignment.
Limitations & Future Work¶
- Western Source Bias: Dilemmas are drawn from an English-language Stack Exchange forum; while explicit country tags were removed, the core relational premises may still reflect Western interpersonal norms.
- Coarse Persona Granularity: Broad prompts such as "You are a native Korean speaker" flatten internal demographic complexity across age, gender, social class, and geographic dialect.
- Absence of Native Human Evaluation: The study measures model elicitation differences using automated annotators; future research must validate whether NL or NP aligns more closely with real-world advice provided by native human speakers.
Related Work & Insights¶
- vs. XTREME / MEGA / Aya Multilingual Benchmarks: While existing benchmarks measure objective cross-lingual transfer in classification, reasoning, and QA, this paper focuses on subjective, socially situated advice where no single correct answer exists.
- vs. Persona and Demographic Simulation Studies (Argyle et al., Hu & Collier): Prior persona studies established that role-playing prompts alter stance on political surveys; this work proves that in multilingual interpersonal guidance, persona prompting produces an illusion of cultural warmth while degrading actionable support.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Pioneering empirical disentanglement of language pathways in interpersonal advice, exposing deep flaws in persona-based cultural simulation.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ 600 dilemmas, 13 languages, 8 state-of-the-art LLMs, and extensive robustness checks across prompt variations, translation filters, and multi-judge audits.
- Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally structured, precise in statistical reasoning, and clear in delineating methodological boundaries.
- Value: ⭐⭐⭐⭐⭐ Essential reading for researchers and engineers developing conversational agents for cross-cultural communication, mental health support, and social simulation.