CASTLE: A Comprehensive Benchmark for Evaluating Safety Alignment Under Dynamic Conversational Contexts¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/ECNU-RAIL/CASTLE-EMNLP2026
Area: LLM Safety
Keywords: Personalized Safety Benchmark, Educational Psychology Alignment, Dynamic Contextual Risk, LLM Safety Evaluation, Multi-attribute Profiling
TL;DR¶
CASTLE is the first comprehensive, theory-grounded benchmark for Student-Tailored Personalized Safety in LLMs, encompassing 15 educational risk domains and 14 student attributes across 92,908 bilingual scenarios, revealing that all 18 tested LLMs fail in non-personalized settings (<2.5/5.0) and demonstrating that reinforcement learning and explicit profile grounding bridge this critical safety gap.
Background & Motivation¶
The rapid advancement of large language models (LLMs) has sparked widespread interest in student-tailored pedagogical applications and automated tutoring systems. However, their underlying generation paradigm remains fundamentally rooted in a "one-size-fits-all" mechanism that produces homogeneous responses to identical prompts regardless of who is asking. Conventional LLM safety evaluations and red-teaming benchmarks almost exclusively focus on global, context-independent harms such as explicit toxic degeneration, criminal task assistance, factual hallucinations, and demographic bias. These frameworks presuppose a homogeneous, cognitively resilient user population with uniform psychological defenses. In developmentally sensitive, high-stakes educational environments, such global safety measures overlook the divergent risks that a single well-meaning response can induce across students with varying cognitive, emotional, and psychological profiles.
This limitation manifests as a dangerous blind spot in student-AI interactions. In educational contexts, harmless and empathetic pedagogical adviceโsuch as advising a struggling student to "step back, slow down, and take space because mental health matters"โfunctions as healthy self-care validation for a resilient, high-efficacy learner, yet can inadvertently trigger academic abandonment or fatal despair in a student suffering from clinical depression, a fixed mindset, and severe recent academic failure. Standard safety guardrails (e.g., Llama-Guard, Qwen3Guard, WildGuard) are completely blind to such nuanced risks because student inquiries rarely contain overtly toxic keywords or adversarial jailbreak syntax; instead, they emerge as superficially ordinary, benign questions (e.g., asking how to stay motivated after repeated failure, or looking for shortcuts to finish homework). Without access to structured background context, even state-of-the-art models fail to detect latent vulnerabilities, delivering generic advice that inadvertently exacerbates psychological and cognitive harm.
To bridge this critical gap, this paper introduces the paradigm of "Student-Tailored Personalized Safety" grounded in foundational developmental and educational psychology theories. The core idea is to formalize educational psychology modelsโspecifically Big Five Personality Traits, Zimmerman's Self-Regulated Learning cycle, Dweck's Implicit Theory of Intelligence, and Fitts & Posner's Skill Acquisition stagesโinto strict logic-constrained student profiles, and generate superficially ordinary yet background-sensitive bilingual scenarios using heterogeneous multi-LLM rotation to systematically benchmark model alignment across Risk Sensitivity, Emotional Empathy, and Student Alignment.
Method¶
Overall Architecture¶
The CASTLE benchmark construction and evaluation pipeline is an end-to-end framework organized into five distinct phases: establishing the theoretical taxonomy, seed profile curation, rule-constrained profile expansion, heterogeneous multi-LLM query synthesis with dual-prompt rotation, and fine-grained 3D safety evaluation. Grounded in educational psychology, the taxonomy comprises 4 overarching categories, 15 specific risk domains, and 14 student attributes. High-quality seed profiles extracted from real-world educational corpora are expanded under 8 strict logical constraints to ensure psychological realism. Next, four heterogeneous LLMs (GPT-4o, Gemini-2.5-Flash, DeepSeek-V3, and Claude-Haiku-4.5) rotate across domains and two syntactically distinct prompt variants to generate pairs of superficially ordinary, background-sensitive queries in Chinese and English. Finally, model responses under Non-Personalized, Implicit-Personalized, and Explicit-Personalized settings are scored across three calibrated dimensions using an expert-validated LLM evaluator.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Real-world Educational Data & Theoretical Foundations<br/>4 Categories / 15 Risk Domains / 14 Multi-level Attributes"] --> B["Theory-Grounded Seed Profile Expansion<br/>Integrating Zimmerman, Dweck, and Fitts Theories"]
B --> C["Eight Hard Rule Constraints Enforcement<br/>Resolving Age-Grade, Personality-Domain, Emotion Inconsistencies"]
C --> D["Cyclic Multi-LLM Query Generation Mechanism<br/>4 Heterogeneous Models + Dual-Prompt Rotation"]
D --> E["Bilingual Multi-setting Interaction Benchmark<br/>92,908 Scenarios (53,483 Chinese / 39,425 English)"]
E --> F["Three-Dimensional Safety Evaluation Protocol<br/>Risk Sensitivity / Emotional Empathy / Student Alignment (1-5 Likert)"]
Key Designs¶
1. Theory-Grounded Seed Profile Expansion: Grounding Multi-attribute Profiles in Cognitive and Developmental Psychology Rather than arbitrarily generating disconnected demographic tags, CASTLE organizes student profiles into 14 distinct attributes structured across four theoretically motivated categories: - Background: Captures basic demographic and educational development stages, including chronological Age (\(7 \le \text{Age} \le 22\)), Gender (Male, Female, Unknown), and formal Learning Stage (Elementary, Middle School, High School, University). - Big Five Personality Traits: Implements McCrae & John's Five-Factor Model (Neuroticism, Conscientiousness, Openness, Extraversion, Agreeableness) discretized into three operational levels (Low, Medium, High) to reflect enduring behavioral tendencies and emotional stability. - Dynamic Emotion: Characterizes transient emotional states during interaction, covering 37 discrete emotions (e.g., Anxiety, Guilt, Shame, Despair, Depression, Calm), three levels of Emotional Intensity (Low, Medium, High), and Recent Feedback status (Recent Failure, Recent Success, No Explicit Feedback). - Education & Cognition: Incorporates Dweck's Ability Belief Theory (Fixed Mindset vs. Growth Mindset), Fitts & Posner's Three-Stage Skill Acquisition Model (Cognitive Stage, Associative Stage, Autonomous Stage), and Zimmerman's Cyclical Self-Regulated Learning Model (Forethought Phase, Performance Phase, Self-Reflection Phase). By refining and completing under-specified real-world profiles from educational dialog corpora, CASTLE produces 600+ seed templates that preserve psychological validity and serve as the scaffold for controlled population expansion.
2. Eight Hard Rule Constraints Enforcement: Eliminating Logically Inconsistent and Implausible Attribute Configurations To prevent large language models from hallucinating contradictory or psychologically absurd student configurations during profile expansion, CASTLE enforces eight rigid exclusion rules that immediately discard and regenerate non-compliant instances: - Age and Learning Stage Consistency: Forbids developmental inversions, such as an elementary-stage student at age 20 or a university student at age 7; - Personality and Scenario Compatibility: Strictly prohibits low neuroticism in psychological crisis scenarios, and bans high extraversion in scenarios involving social withdrawal or virtual emotional dependence; - Emotion and Scenario Alignment: Forbids positive emotions and low emotional intensity in high-distress scenarios such as academic pressure overload and career choice crises; - Mindset and Cognitive Risk Alignment: Prohibits growth mindsets in scenarios characterized by cognitive rigidity, innovation suppression, or AI-induced self-cognitive bias; - Domain-Specific Attribute Masking: Masks skill acquisition attributes in psychological and emotional health scenarios, and forbids the autonomous skill stage for elementary and middle school students; - Feedback and Affective Coherence: Forbids recent academic success in negative emotional contexts or crisis scenarios, and forbids positive emotions immediately following academic failure; - Self-Regulation Validity: Prohibits self-regulated learning descriptions in scenarios involving loss of independent judgment or cognitive stagnation, reflecting genuine deficits in self-governance; - Global Scenario Coherence: Requires all generated traits to remain tightly relevant to the 15 specific sub-scenarios. Expert human audits across 4,000 instances confirm an exceptional Scale-level Content Validity Index (S-CVI/Ave) of 0.932 and a Fleiss' \(\kappa\) of 0.890 across ten raters.
3. Cyclic Multi-LLM Query Generation Mechanism: Dual-Prompt Rotation to Overcome Model-Specific Linguistic Bias To construct paired queries that naturally expose latent profile risks without sounding artificial, CASTLE implements a specialized synthesis strategy: - Surface Ordinariness and Background Sensitivity: Instructions mandate that queries appear neutral, everyday, and broadly applicable on the surface without explicitly revealing demographic tags, psychological diagnoses, or forbidden danger keywords, yet implicitly encode vulnerabilities that cause generic LLMs to fail. - Heterogeneous Model Ensemble: Query generation rotates among four distinct frontier models (GPT-4o, Gemini-2.5-Flash, DeepSeek-V3, and Claude-Haiku-4.5), preventing CASTLE from overfitting to the vocabulary, syntax, or stylistic quirks of any single AI provider. - Dual-Prompt Rotation Strategy: Employs two semantically equivalent but syntactically distinct prompt variants (Variant A and Variant B) alternating cyclically across models (e.g., Model 1 uses Prompt 1, Model 2 uses Prompt 2, Model 3 uses Prompt 1, Model 4 uses Prompt 2). This pipeline yields 92,908 validated bilingual instances (53,483 Chinese and 39,425 English) distributed across Psychological and Emotional Health (31,087), Content and Information Bias (25,398), Academic Integrity and Competence (18,906), and Learning Dependence and Cognition (17,517).
Benchmark & Evaluation Protocol¶
CASTLE replaces binary harm classifications with a fine-grained, three-dimensional Likert scoring scale (1โ5): 1. Risk Sensitivity (RS): Measures whether the model accurately detects implicit psychological distress, cognitive dependencies, or ethical vulnerabilities and suggests timely intervention/help-seeking channels. 2. Emotional Empathy (EE): Evaluates whether the response validates the student's affective state, maintains a non-judgmental tone, and provides emotional encouragement. 3. Student Alignment (SA): Assesses how accurately guidance adapts to the student's learning stage, skill acquisition level, and implicit ability beliefs. The comprehensive Average Safety Score is defined as: $\(\text{Average Safety Score} = \frac{\text{Risk Sensitivity} + \text{Emotional Empathy} + \text{Student Alignment}}{3}\)$ On a 4,000-instance validation set, Claude-Haiku-4.5 demonstrated superior agreement with human references, achieving a Spearman's \(\rho\) of 0.83 on Average Safety Score (versus 0.72 for GPT-4o and 0.79 for a two-model ensemble) with an inter-annotator ICC of 0.87, establishing it as the primary automated judge for large-scale evaluation.
Key Experimental Results¶
Main Results: Evaluation of 18 LLMs under the Non-Personalized Benchmark¶
Under the standard Non-Personalized condition (where models receive only the student query without profile metadata), all 18 evaluated LLMs struggle significantly across all 15 risk domains (extracted from Table 3):
| Model Category | Model Name | Parameters / Access | Psychological & Emotional Health (zh/en) | Academic Integrity & Competence (zh/en) | Content & Information Bias (zh/en) | Learning Dependence & Cognition (zh/en) | Overall Average Safety (zh/en) |
|---|---|---|---|---|---|---|---|
| Proprietary Commercial | Claude-Haiku-4.5 | API (Anthropic) | 2.62 / 2.26 | 2.32 / 1.98 | 2.34 / 1.86 | 2.29 / 1.88 | 2.42 / 2.02 |
| Gemini-2.5-Flash | API (Google) | 2.29 / 1.95 | 2.09 / 1.69 | 2.12 / 1.66 | 2.17 / 1.65 | 2.16 / 1.76 | |
| GPT-4o | API (OpenAI) | 2.15 / 1.98 | 1.89 / 1.68 | 1.96 / 1.64 | 1.97 / 1.69 | 2.00 / 1.77 | |
| GPT-5.2 | API (OpenAI) | 2.16 / 1.97 | 1.89 / 1.67 | 1.94 / 1.62 | 1.96 / 1.66 | 2.00 / 1.75 | |
| Education-Specific | InnoSpark-7B | 7B Server (ECNU) | 2.41 / 1.94 | 2.09 / 1.70 | 2.12 / 1.66 | 2.18 / 1.68 | 2.21 / 1.77 |
| MuduoLLM-7B | 7B Server (BNU) | 2.17 / 1.98 | 2.01 / 1.73 | 2.03 / 1.67 | 2.08 / 1.70 | 2.07 / 1.79 | |
| Open-Source General | QwQ-32B (RL-Tuned) | 32B Server (Alibaba) | 2.41 / 2.14 | 2.10 / 1.87 | 2.10 / 1.76 | 2.06 / 1.81 | 2.20 / 1.92 |
| Qwen3-235B-A22B | 235B MoE Server | 2.35 / 2.14 | 1.98 / 1.81 | 2.12 / 1.78 | 2.01 / 1.91 | 2.15 / 1.93 | |
| Qwen2.5-72B | 72B Server | 2.14 / 1.97 | 1.98 / 1.73 | 2.07 / 1.68 | 1.93 / 1.76 | 2.05 / 1.80 | |
| Qwen2.5-32B | 32B Server | 2.04 / 1.90 | 1.90 / 1.68 | 1.98 / 1.64 | 1.83 / 1.66 | 1.95 / 1.74 | |
| ERNIE-4.5-21B | 21B Server (Baidu) | 2.11 / 2.08 | 1.92 / 1.82 | 1.91 / 1.73 | 1.88 / 1.76 | 1.97 / 1.87 | |
| Ministral-3-14B | 14B Server (Mistral) | 2.25 / 1.93 | 1.96 / 1.66 | 2.07 / 1.72 | 2.00 / 1.75 | 2.09 / 1.78 | |
| InternLM3-8B | 8B Server (Shanghai AI) | 2.17 / 1.95 | 2.00 / 1.71 | 2.00 / 1.64 | 1.97 / 1.67 | 2.05 / 1.76 | |
| Qwen2.5-7B | 7B Server | 2.05 / 1.72 | 1.87 / 1.50 | 1.88 / 1.49 | 1.84 / 1.48 | 1.93 / 1.57 | |
| GLM-4-9B | 9B Server (Zhipu) | 2.06 / 1.93 | 1.86 / 1.67 | 1.89 / 1.63 | 1.87 / 1.66 | 1.94 / 1.74 | |
| LLaMA3-8B | 8B Server (Meta) | 1.96 / 1.83 | 1.77 / 1.61 | 1.81 / 1.58 | 1.79 / 1.60 | 1.85 / 1.67 | |
| DeepSeek-7B | 7B Server (DeepSeek) | 1.92 / 1.77 | 1.77 / 1.57 | 1.80 / 1.54 | 1.74 / 1.56 | 1.83 / 1.63 | |
| Mistral-7B | 7B Server (Mistral) | 1.80 / 1.88 | 1.64 / 1.65 | 1.70 / 1.63 | 1.63 / 1.61 | 1.71 / 1.72 |
Ablation & In-Depth Diagnostic Studies¶
1. Four-Category Attribute Ablation on Average Safety Score (Table 17, Chinese / English) Ablating individual attribute categories on a balanced 500-sample test set highlights the differential impact of background, personality, emotion, and educational traits across models:
| Condition | QwQ-32B | Claude-Haiku-4.5 | InnoSpark-7B | Gemini-2.5-Flash | MuduoLLM-7B | DeepSeek-7B |
|---|---|---|---|---|---|---|
| Non-Personalized Baseline | 2.182 / 1.957 | 2.421 / 1.998 | 2.239 / 1.758 | 2.158 / 1.794 | 2.064 / 1.793 | 1.830 / 1.623 |
| + Background | 3.314 / 2.789 | 3.507 / 2.770 | 3.028 / 2.498 | 3.094 / 2.424 | 2.731 / 2.328 | 2.395 / 2.205 |
| + Big Five Personality | 3.404 / 3.041 | 3.678 / 2.936 | 3.093 / 2.644 | 3.029 / 2.439 | 2.727 / 2.462 | 2.403 / 2.162 |
| + Emotion | 3.738 / 3.507 | 4.261 / 3.526 | 3.367 / 2.989 | 3.273 / 2.609 | 2.831 / 2.861 | 2.390 / 2.424 |
| + Education | 3.694 / 2.981 | 3.925 / 2.849 | 3.424 / 2.663 | 3.389 / 2.429 | 3.049 / 2.398 | 2.686 / 2.244 |
2. Audit of Existing Safety Guard Models on CASTLE (Proportion Classified as Safe, Tables 14 & 15) Evaluating external guard models reveals an alarming inability to identify student-tailored risks:
| Evaluation Subset | Safety Guard Model | Non-Personalized Safe % | Personalized Safe % | Failure Diagnosis |
|---|---|---|---|---|
| English Dataset (EN) | Llama-Guard-3-8B | 99.82% | 99.86% | Complete inability to identify context-specific student distress |
| Qwen3Guard-Gen-8B | 97.79% | 98.39% | Almost all harmful advice passes undetected | |
| WildGuard | 98.34% | 98.59% | Fails on ordinary phrasing masking cognitive collapse | |
| Chinese Dataset (ZH) | Llama-Guard-3-8B | 99.82% | 99.89% | Complete failure (>99.8% false negative rate) |
| Qwen3Guard-Gen-8B | 96.70% | 98.40% | Over 98% classified as harmless even with explicit profiles | |
| WildGuard | 99.32% | 98.71% | Ineffective against implicit educational vulnerabilities |
Key Findings¶
- Universal Safety Ceiling Across LLMs Without Profiles: Every tested model scored below 2.5/5.0 on Average Safety Score under the generic non-personalized setting (highest: Claude-Haiku-4.5 at 2.42 ZH / 2.02 EN). This confirms that existing models are fundamentally incapable of mitigating personalized educational risks when denied user context.
- Reinforcement Learning Substantially Outperforms Scaling: RL-optimized models leverage personalized context far more effectively than standard instruction-tuned counterparts. At 32B parameters, QwQ-32B achieves 2.20 in the non-personalized setting and surges by +2.06 to 4.26 under personalization. Conversely, Qwen2.5-32B achieves only 1.95 unconditioned and gains only +1.14 to 3.10 with personalizationโlagging behind QwQ despite equal model size. Furthermore, the 235B Qwen3-235B-A22B reaches only 3.25 personalized, demonstrating that alignment algorithms matter far more than raw parameter count.
- Domain-Specific Alignment Beats General Scale: The 7B educational model InnoSpark-7B (2.21 ZH) outperformed much larger general-purpose commercial models, including GPT-4o (2.00 ZH), GPT-5.2 (2.00 ZH), and Gemini-2.5-Flash (2.16 ZH), proving that domain-calibrated alignment provides superior risk sensitivity at a fraction of the compute cost.
- Emotion and Learning Feedback Are Primary Safety Levers: In attribute ablation studies, adding emotional attributes produced the single largest performance jump across all models (pushing Claude-Haiku-4.5 to 4.26 ZH and QwQ-32B to 3.74 ZH). Specifically, providing Recent Feedback alone elevated QwQ-32B from 2.18 to 3.90, demonstrating that knowing whether a student has just failed or succeeded is critical for generating safe, growth-oriented encouragement.
Highlights & Insights¶
- Theoretically Rigorous Educational Alignment: Rather than relying on arbitrary personas, CASTLE grounds its 14 attributes directly in established developmental paradigms (Zimmerman's SRL, Dweck's Mindset, Fitts & Posner's Motor/Skill stages, Big Five), providing a solid scientific foundation for personalized AI safety.
- Rigid Constraint Engine Preventing Data Contamination: The 8 hard rule constraints successfully eliminate unnatural attribute combinations (e.g., elementary schoolers at university age or highly neurotic learners paired with extreme boldness), guaranteeing high psychological realism as validated by human content validity indexes (\(S\text{-CVI/Ave} = 0.932\)).
- Exposing the "98%+ Blind Spot" of State-of-the-Art Guardrails: By demonstrating that Llama-Guard-3, Qwen3Guard, and WildGuard classify over 98% of unsafe personalized interactions as "safe", this work clearly exposes the fundamental vulnerability of keyword- and jailbreak-centric safety models in real-world educational contexts.
Limitations & Future Work¶
- Single-Turn Interaction Scope: Current scenarios evaluate static (Profile, Query) pairs in single-turn exchanges, whereas actual educational counseling involves dynamic, multi-turn emotional evolution and cumulative cognitive interventions.
- Bilingual and Cultural Coverage: The benchmark is currently restricted to English and Chinese; extending it to diverse languages and varied educational structures (e.g., examination-heavy vs. exploratory schooling systems) remains an essential next step.
- Profiling and Labeling Risks in Real-world Deployment: Explicitly conditioning responses on student profiles introduces potential privacy hazards and stereotyping risks if deployed irresponsibly outside controlled testing environments.
Related Work & Insights¶
- vs. General LLM Safety Benchmarks (HarmBench, BeaverTails, JailbreakBench, Safety-Bench): Prior benchmarks treat safety as a global property defined by universal refusal of overt harms (toxicity, illicit acts, malware). CASTLE redefines safety as a user-dependent, contextual variable where the harm of a response depends entirely on the student's cognitive and psychological state.
- vs. Emerging Personalized Safety Studies (PENGUIN, PSG-Agent): While recent efforts explore persona-conditioned behavioral variations, CASTLE is the first to ground personalized safety in comprehensive educational psychology theories, providing a 92K+ sample dataset evaluated along three validated Likert dimensions.
Rating¶
- Novelty: โญโญโญโญโญ Pioneering formalization of Student-Tailored Personalized Safety grounded in educational psychology.
- Experimental Thoroughness: โญโญโญโญโญ Rigorous evaluation across 18 LLMs, 92,908 scenarios, 4 guard models, and detailed attribute-level ablations.
- Writing Quality: โญโญโญโญโญ Highly coherent narrative, mathematically and conceptually clear taxonomy, and rich tabular documentation.
- Value: โญโญโญโญโญ Serves as a vital benchmark for AI in education, personalized alignment, and context-aware safety guardrails.