Skip to content

Aligning Cultural Behaviors in Arabic Language Models via Preferences

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/mbzuai-nlp/ArabPref
Area: LLM Alignment
Keywords: Cultural Preference Alignment, Arabic Language Models, Pan-Arab Heterogeneity, Direct Preference Optimization, Cultural MCQ Benchmark

TL;DR

To address the lack of country-specific cultural behavioral alignment in Arabic and multilingual LLMs, this paper introduces ARABPREF—the first large-scale pan-Arab preference dataset and native-validated evaluation benchmark covering all 22 Arab League nations across 16 categories and 200 aspects—demonstrating significant gains in cultural appropriateness and robustness via preference optimization.

Background & Motivation

In recent years, Arabic and multilingual large language models have advanced substantially on standard linguistic and factual benchmarks. However, when deployed in culturally sensitive domains involving social norms, religion, and regional etiquette, models frequently exhibit glaring cultural misalignments. Empirical studies have consistently demonstrated that major open-source and proprietary LLMs predominantly embody Western and American-centric value profiles. As a result, even when an LLM's response is factually accurate, it often violates local etiquette, generates offensive or insensitive recommendations, and undermines human-AI trust in non-Western communities.

The core tension lies in the oversimplification of the Arab world as a monolithic cultural entity. Prior cultural alignment efforts, such as CIDAR and PALM, have focused almost exclusively on instruction tuning for factual compliance or broad regional commonsense, without modeling contrastive normative preferences. In reality, the 22 Arab League member states exhibit substantial national heterogeneity across behavioral expectations, ranging from alcohol consumption and dress codes to public displays of affection and honorific greetings. Meanwhile, existing multilingual preference corpora (e.g., CARE, PRISM) either suffer from limited Arabic coverage or capture subjective, individualized preferences rather than culturally grounded societal norms, leaving country-conditioned behavioral preference alignment systematically underexplored.

This work addresses this challenge by framing culture as context-dependent, locale-grounded behavioral expectations. Core idea: construct ARABPREF, the first comprehensive pan-Arab preference learning and evaluation resource covering all 22 Arab League nations across 16 categories and 200 aspects, and leverage native-in-country human verification alongside preference optimization (DPO/KTO) to align models with fine-grained regional behavioral norms.

Method

Overall Architecture

The ARABPREF curation and alignment pipeline is organized around distinct "country–aspect pairs" to capture local behavioral heterogeneity. The overall process comprises multi-source raw guideline compilation, automated paraphrasing and filtering, a rigorous two-stage Human-in-the-Loop (HITL) verification, held-out Multiple-Choice Question (MCQ) benchmark derivation, and post-training alignment via preference optimization.

The framework collects travel advisories, consular guidance, and CultureBank entries across all 22 Arab League member countries. Contrastive preferred and discouraged behavioral candidates are generated via Gemini-2.5-Flash, refined through MinHash near-deduplication and GemmaEmbedding semantic filtering, verified by in-country native linguists, and translated into Modern Standard Arabic (MSA). The resulting corpus powers both preference tuning (31,060 training pairs) and zero-leakage evaluation across 1,984 native-validated MCQs.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multi-Source Cultural Guidelines<br/>Official Advisories + CultureBank"] --> B["Country-Aspect Decoupling & Paraphrasing<br/>Gemini-2.5-Flash + Semantic Filtering"]
    B --> C["Two-Stage HITL Native Validation<br/>Cultural Appropriateness + MSA Verification"]
    C --> D["Held-Out MCQ Benchmark Derivation<br/>Inverse-Polarity + Cross-Country Distractors"]
    D --> E["Preference-Based Behavioral Alignment<br/>SFT / DPO / KTO Optimization & Evaluation"]

Key Designs

1. Country-Aspect Decoupling & Paraphrasing: Structuring Pan-Arab Cultural Diversity

Treating Arabic culture homogeneously obscures critical local distinctions. The authors establish a comprehensive taxonomy spanning 16 primary categories—such as Social Etiquette, Religion & Rituals, Dining & Food, Hospitality & Home Life, and Business & Work—expanded into 200 unique, fine-grained aspects mapped across 22 Arab League nations. For each country–aspect pair, Gemini-2.5-Flash generates contrastive candidate pairs of preferred behaviors (\(y^+\)) and discouraged behaviors (\(y^-\)) conditioned on regional background documents. To eliminate model hallucinations and textual redundancy, an automated multi-stage filter is applied: MinHash removes structural near-duplicates, and GemmaEmbedding cosine similarity pruning (with a threshold of 0.90) removes semantic redundancies, yielding 15,530 high-quality English seed preference pairs.

2. Two-Stage HITL Native Validation: Ensuring Cultural Authenticity and Flawless MSA

Automated generation alone cannot guarantee genuine cultural fidelity. The authors partnered with a professional linguistic service provider, deploying a specialized team of 40 native annotators (with 2–3 experts assigned per country). To ensure cultural competence, a strict "Native-In-Country" policy was enforced: 95% of annotators resided directly in the target country, while the remaining 5% had lived in the region for over 15 years. Validation follows two sequential stages: - Stage 1 (Cultural Preference Quality Check): Native reviewers inspect English \((x, y^+, y^-)\) triples, accepting or rejecting instances based on whether they authentically reflect local societal norms. Notably, annotators rejected 15.25% of candidate triples as inaccurate or neutral, substantially reshaping the dataset. - Stage 2 (MSA Verification): Candidates undergo initial LLM translation into Modern Standard Arabic, followed by expert human post-editing to guarantee grammatical correctness, natural fluency, and semantic alignment with the English source. The process achieved an inter-annotator agreement rate of 90.1%, establishing a rigorous bilingual ground truth.

3. Held-Out MCQ Benchmark Derivation: Leakage-Free Cultural Decision Probing

To evaluate whether models can recognize culturally appropriate behavior among plausible alternatives, the authors construct the ARABPREF MCQ benchmark comprising 1,984 questions across English and MSA. To strictly prevent data leakage, dataset splitting is executed strictly at the country–aspect level: test MCQs and test preferences are derived exclusively from held-out country–aspect pairs that never appear in the training split. Each question features one correct answer and three challenging distractors: - Distractor 1 represents the "inverse polarity behavior" (the discouraged action within the same country); - Distractors 2 and 3 represent valid, preferred behaviors from the same aspect but associated with different Arab countries (e.g., contrasting venue-restricted alcohol in the UAE with prohibition in Saudi Arabia). GemmaEmbedding similarity filtering (threshold 0.80) ensures option distinctiveness, while explicit prompts sanitize all country mentions and subjective adjectives (e.g., "progressive" or "traditional"), forcing the model to rely purely on behavioral appropriateness. The benchmark achieved a Cohen's \(\kappa\) of 0.693.

4. Preference-Based Behavioral Alignment: Post-Training Under Diverse Supervision

To optimize cultural alignment, the authors benchmark three post-training paradigms across multilingual backbones (Llama-3.1-8B-Instruct, Qwen-2.5-7B-Instruct) and an Arabic-centric foundation model (Jais-Adapted-7B-Chat). For pairwise preference data, Direct Preference Optimization (DPO) optimizes the implicit reward margin: $\(\Delta_\theta(x, y^+, y^-) = \beta \log \frac{\pi_\theta(y^+ \mid x)}{\pi_{\text{ref}}(y^+ \mid x)} - \beta \log \frac{\pi_\theta(y^- \mid x)}{\pi_{\text{ref}}(y^- \mid x)}\)$ minimizing the objective: $\(\mathcal{L}_{\text{DPO}}(\theta) = -\mathbb{E}_{(x, y^+, y^-)} \left[ \log \sigma\left( \Delta_\theta(x, y^+, y^-) \right) \right]\)$ with \(\beta = 0.1\). For Kahneman–Tversky Optimization (KTO), instances are decoupled into positive \((x, y^+, 1)\) and negative \((x, y^-, 0)\) examples relative to an empirical reference point \(\hat{z}_0\). All models are fine-tuned using LoRA (\(r=8, \alpha=16\)) on linear modules, investigating the interplay between ARABPREF, multi-cultural preferences (CARE), and standard Arabic instruction tuning (PALM).

Key Experimental Results

Main Results

Model generations were evaluated using GPT-4o as judge across Cultural Appropriateness, General Quality, and Literacy on a 1–5 Likert scale (demonstrating strong human correlation with Spearman \(\rho > 0.60\)). Downstream cultural reasoning was assessed via zero-shot MCQ accuracy (%) on the 1,984 held-out evaluation items.

Model & Method Training Configuration English Cultural (1-5) English Avg. (1-5) Arabic Cultural (1-5) Arabic Avg. (1-5) English MCQ Acc (%) Arabic MCQ Acc (%)
Llama-3.1-8B-Instruct Vanilla 3.23 3.40 3.13 3.15 - -
w/ PALM (SFT) 3.74 3.91 3.36 3.58 - -
w/ CARE (DPO) 3.85 3.92 3.24 3.30 - -
w/ ARABPREF (DPO) 3.78 3.78 3.49 3.59 - -
w/ ARABPREF + CARE (DPO) 3.78 3.91 3.76 3.69 - -
Qwen-2.5-7B-Instruct Vanilla 3.41 3.69 3.89 3.98 63.3 57.6
w/ PALM (SFT) 4.02 4.29 3.36 3.52 63.0 56.9
w/ CARE (DPO) 4.28 4.30 4.21 4.11 67.5 57.6
w/ ARABPREF (DPO) 4.26 4.41 4.01 4.02 67.0 58.0
w/ ARABPREF + CARE (DPO) 4.38 4.18 4.27 4.25 68.6 58.7
w/ ARABPREF + CARE (KTO) 4.19 4.14 4.32 4.47 67.1 58.3
Jais-Adapted-7B-Chat Vanilla 3.41 3.74 3.40 3.54 49.6 39.4
w/ PALM (SFT) 3.18 3.49 3.26 3.47 40.4 37.1
w/ CARE (DPO) 3.99 4.18 3.36 3.71 51.0 45.6
w/ ARABPREF (DPO) 4.33 4.53 4.24 4.31 52.6 40.2
w/ ARABPREF + CARE (DPO) 4.39 4.60 4.28 4.34 53.7 43.1
w/ ARABPREF + CARE (SFT) 4.50 4.62 4.42 4.43 - -
Proprietary Models (Zero-shot) DeepSeek-V4-Flash 4.80 4.71 4.76 4.77 75.3 71.6
Qwen3.6-Plus 4.75 4.76 4.74 4.66 80.5 76.5
Claude-Sonnet-4 4.91 4.92 4.87 4.58 74.8 73.7

Ablation Study: Scale-Controlled Sampled-Data Analysis

To test whether ARABPREF's performance improvements stem from genuine orthogonal preference signals rather than mere data volume expansion when combined with CARE, a controlled experiment was conducted on Qwen-2.5-7B-Instruct using ARABPREF sampled (strictly 30 pairs per country):

Configuration English Cultural English General English Literacy English Avg. Arabic Cultural Arabic General Arabic Literacy Arabic Avg.
Vanilla 3.41 3.58 4.07 3.69 3.89 4.08 3.98 3.98
PALM (SFT) 4.02 4.28 4.57 4.29 3.36 3.53 3.68 3.52
CARE (DPO) 4.28 4.55 4.06 4.30 4.21 4.46 3.66 4.11
ARABPREF sampled (DPO) 4.02 4.26 4.33 4.20 3.87 4.14 4.09 4.03
ARABPREF sampled (KTO) 3.87 4.31 4.65 4.28 3.54 3.98 4.13 3.88
ARABPREF sampled + CARE (DPO) 4.50 4.32 4.37 4.40 4.01 3.87 4.01 3.96
ARABPREF sampled + CARE (KTO) 4.08 4.14 4.26 4.16 3.67 3.96 4.14 3.92
ARABPREF sampled + CARE (SFT) 4.17 4.35 4.23 4.25 3.76 4.23 4.05 4.01

Key Findings

  • Contrastive Preferences Outperform Imitation Tuning: SFT on positive demonstrations frequently degraded MCQ discriminative accuracy (e.g., Jais dropping from 49.6% to 40.4% in English), whereas contrastive objectives like DPO and KTO successfully penalized culturally discouraged behaviors.
  • Orthogonal Cultural Signal: In the scale-controlled ablation, adding just 30 pairs per country of ARABPREF to CARE pushed English Cultural Appropriateness to 4.50 (surpassing CARE-only DPO at 4.28), proving that country-conditioned fine-grained pairs provide qualitative signals beyond mere data scaling.
  • Cross-Lingual Dynamics: Llama and Jais benefited robustly from bilingual mixed training, demonstrating positive cross-lingual transfer, while Qwen displayed higher sensitivity to language distribution shifts.

Highlights & Insights

  • Granular 22-Country Cultural Ontology: Deconstructs generic Arab identity into 16 categories and 200 distinct behavioral aspects, operationalizing cross-border differences into clean, actionable preference pairs.
  • Rigorous Native-in-Country Verification: By retaining only annotators living within the target countries, the pipeline identified and rejected 15.25% of synthetic candidates, effectively insulating the dataset against synthetic model biases.
  • Cross-Border Distractor Formulation: Constructing MCQ distractors from real, acceptable practices of neighboring nations ensures that models cannot rely on generic superficial heuristics, setting a high bar for cultural discernment.

Limitations & Future Work

  • Lack of Dialectal and Sub-National Granularity: The current corpus is primarily in Modern Standard Arabic and English, leaving rich spoken Arabic dialects (Darija, Levantine, Egyptian, Gulf) and intra-national tribal or regional subcultures for future work.
  • Single-Turn Static Formulation: Real-world cultural negotiation evolves dynamically over multi-turn dialogues, influenced by interpersonal relationships, social hierarchies, and conversational history.
  • vs CARE (Guo et al., 2025): CARE provides multi-cultural preferences but treats Arabic as a broad regional group; ARABPREF introduces the first country-stratified preference architecture across all 22 Arab League nations.
  • vs PALM / CIDAR (Alwajih et al., 2025; Alyafeai et al., 2024): Existing Arabic resources focus primarily on instruction-following or factual QA without negative behavioral contrasts; ARABPREF provides the explicit contrastive supervision required for alignment.
  • vs ArabCulture (Sadallah et al., 2025): ArabCulture is restricted to multiple-choice commonsense evaluation; ARABPREF unifies both large-scale training preference pairs (31K) and an independent evaluation benchmark (2K MCQs).

Rating

  • Novelty: ⭐⭐⭐⭐⭐ First large-scale pan-Arab behavioral preference dataset breaking the monolithic view of Arab culture across 22 nations.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Evaluates multiple model architectures, alignment algorithms (SFT/DPO/KTO), scale-controlled ablations, and zero-shot proprietary baselines.
  • Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally structured with clear taxonomy, rigorous HITL guidelines, and transparent data documentation.
  • Value: ⭐⭐⭐⭐⭐ An essential foundation for deploying culturally safe and respectful LLMs across the Arabic-speaking world.