Skip to content

Cultural Evolution Drives the Emergence of Conventions in Neural Emergent Communication

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://gitlab.ai.vub.ac.be/mtoquebiau1/cultural-evo
Area: LLM Reasoning
Keywords: Emergent Communication, Cultural Evolution, Linguistic Conventions, Bi-directional Agents, Decentralized Learning

TL;DR

This paper proposes a decentralized cultural evolution framework for bi-directional neural agents, demonstrating for the first time in an unsupervised and fully distributed multi-agent population that communicative conventions emerge from local in-context reference games, and proving that conventionalized languages substantially lower the learning barrier for cultural transmission.

Background & Motivation

Computational models of language evolution offer crucial empirical and theoretical instruments for exploring how shared symbolic systems emerge in human populations. Over the past decade, deep reinforcement learning has revitalized neural emergent communication by enabling artificial agents to play communicative reference games in increasingly complex perceptual environments. However, the majority of existing neural approaches rely on artificial architectural simplifications that fundamentally preclude the genuine dynamics of cultural evolution: agents are strictly assigned unidirectional roles as either dedicated speakers or dedicated listeners, games are restricted to isolated dyads of two agents, internal hidden representations or gradients are shared across individuals, or updates are computed over large batched interactions. These designs bypass the core mechanism of cultural transmission—namely, the ratchet effect, whereby individuals in a community observe communicative acts, internalize their underlying intent, and actively reuse them across subsequent interactions.

This methodological divergence produces a profound operational tension. Populations with rigid Speaker-Listener (S-L) specialization possess ample neural capacity to achieve near-perfect communication success; however, listeners inevitably become bespoke translators for disparate speaker dialects, yielding fragmented idiosyncratic repertoires rather than a unified population-wide convention. Furthermore, batched gradient updates average out situational communicative gambits, diluting the immediate behavioral adaptation required to select successful communicative variants. Consequently, existing models fail to capture how shared symbolic norms spontaneously coalesce across communities without central coordination.

Addressing this fundamental gap, this paper investigates whether a decentralized community of neural agents can evolve compact, shared linguistic conventions under purely local interaction pressures. The authors eliminate asymmetric functional roles and centralized coordination by modeling populations of symmetric, bi-directional neural agents that alternate between speaker and listener roles in situated reference games, reinforced by an imitation learning objective mirroring human cultural transmission. Core idea: by combining parameter-shared bi-directional encoders, LogSumExp-smoothed in-context discriminative scoring, and an intentional imitation learning loss under single-interaction online updates, decentralized neural populations spontaneously bootstrap compact, highly conventionalized symbolic languages that directly benefit cultural transmission.

Method

Overall Architecture

The framework instantiates cultural evolution in an in-context reference game within a population \(\mathcal{P}\) of \(N\) symmetric, bi-directional neural agents communicating about a universe of visual entities \(\mathcal{W}=\{e_1, \dots, e_K\}\). Each local interaction samples two distinct agents from the population, assigning them the roles of speaker \(S\) and listener \(L\). Both agents observe a shared scene \(e=\{e_1, \dots, e_C\} \subset \mathcal{W}\) containing \(C\) entity vectors, where the last entity is designated as the target topic \(e_t = e_C\). The speaker utters a discrete symbol \(w_S\) from vocabulary \(\mathcal{V}\) intended to distinguish \(e_t\) from all distractors. The listener receives the identical scene in a randomly permuted entity order along with \(w_S\), pointing to an estimated target entity \(\hat{e}\). A successful interaction yields reward \(R=1\), while a failure gives \(R=0\) and discloses the true target \(e_t\) to the listener.

The neural architecture of each agent is deliberately compact and fully shared across communicative modalities: an Entity Encoder \(f_e(e|\theta)\) (a 2-layer MLP with LayerNorm) maps observed entities into an \(E\)-dimensional embedding space, while a Symbol Encoder \(f_w(w|\theta)\) (an embedding lookup followed by a 2-layer MLP) maps discrete symbols into the same metric space. Both speaking (utterance production) and listening (referent comprehension) utilize the exact same parameters \(\theta_i = \{\theta_{i,e}, \theta_{i,w}\}\). Immediately following each pairwise game, both participating agents update their respective parameters independently using single-sample gradients, immediately carrying the updated knowledge forward into subsequent community exchanges.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["In-Context Scene Input<br/>Entities e and target et"] --> B["Bi-directional Parameter-Shared Encoders<br/>Unified entity and symbol representations"]
    B --> C["Continuous In-Context Discrimination Scoring<br/>LogSumExp soft distractor suppression"]
    C --> D["Speaker REINFORCE Exploration<br/>Policy update weighted by game reward R"]
    C --> E["Listener Cultural Imitation Learning<br/>Joint prediction CE + speaking policy alignment"]
    D --> F["Single-Sample Online Policy Evolution<br/>Decentralized ratchet effect drives conventions"]
    E --> F

Key Designs

1. Bi-directional Parameter-Shared Encoders: Bridging Production and Comprehension

Prior emergent communication setups split populations into dedicated speakers and listeners, creating representational divergence where listeners simply overfit each speaker's idiosyncrasies. This paper enforces complete functional interchangeability: every agent \(i\) possesses a unified parameter set \(\theta_i = \{\theta_{i,e}, \theta_{i,w}\}\). For any entity \(e_j \in \mathbb{R}^O\) and symbol \(w_k \in \mathbb{R}^V\), the encoders generate representations \(f_e(e_j|\theta_i) \in \mathbb{R}^E\) and \(f_w(w_k|\theta_i) \in \mathbb{R}^E\). Their semantic affinity is evaluated using a scaled dot product: $$ \text{sim}(e_j, w_k | \theta_i) = \frac{f_e(e_j|\theta_i) \cdot f_w(w_k|\theta_i)^\top}{\sqrt{E}} $$ Because the parameters governing symbol comprehension are identical to those generating utterances, any semantic refinement acquired during listening immediately informs subsequent speaking decisions, and vice versa. This shared latent space provides the requisite cognitive substrate for cultural learning.

2. Continuous In-Context Discrimination Scoring: Differentiable Context-Aware Production

To succeed in a contextual reference game, a symbol must not merely correlate with the target entity; it must actively discriminate the target from co-present distractors. Replacing non-differentiable hard-maximum operations that propagate sparse gradient signals, the speaker computes a smooth discrimination score using a LogSumExp (LSE) approximation: $$ \delta(e_t, w_k | \theta_S) = \text{sim}(e_t, w_k | \theta_S) - \tau_{\text{lse}} \log \sum_{j \neq t} \exp\left(\frac{\text{sim}(e_j, w_k | \theta_S)}{\tau_{\text{lse}}}\right) $$ where \(\tau_{\text{lse}}=1.0\). This formulation integrates information from every distractor simultaneously. The speaker samples its utterance \(w_S\) from a categorical policy distribution \(\pi(w_k | e, \theta_S) = \text{softmax}(\delta(e_t, w | \theta_S)/\tau_s)_k\) with temperature \(\tau_s=0.1\), generating targeted linguistic expressions tailored to the specific context.

3. Listener Cultural Imitation Learning: Social Learning via Intentional Alignment

Task success alone often leads populations into fragmented equilibrium states where different sub-groups maintain divergent vocabularies. Inspired by Tomasello's cultural learning theory, the listener incorporates an explicit social imitation objective alongside target prediction. While the primary goal minimizes cross-entropy prediction loss \(\mathcal{L}_{\text{pred}}(\theta_L) = -\log p_L(e_t | e, w_S, \theta_L)\), the listener simultaneously minimizes an imitation loss that aligns its own speaking policy distribution \(\pi(\cdot | e, \theta_L)\) with the symbol \(w_S\) produced by the speaker: $$ \mathcal{L}_{\text{imit}}(\theta_L) = -\log \pi(w_S | e, e_t, \theta_L) $$ The composite listener loss is \(\mathcal{L}_L(\theta_L) = \beta_p \mathcal{L}_{\text{pred}}(\theta_L) + \beta_i \mathcal{L}_{\text{imit}}(\theta_L)\). By minimizing \(\mathcal{L}_{\text{imit}}\), the listener internalizes the speaker's discriminative communicative strategy, ensuring that when this listener subsequently acts as a speaker, it re-emits the same symbol under similar context, catalyzing rapid population-wide convergence toward shared conventions.

4. Single-Sample Online Policy Evolution: Unbatched Cultural Ratchet Effect

Conventional machine learning accumulates large batches of interactions before performing gradient updates, averaging behavioral variability across disparate partners. In contrast, this approach operates strictly at batch size \(B=1\). Interacting agents update parameters immediately via RMSProp after every single game, with speaker gradients clipped to a maximum norm of 10. This unbatched adaptation allows successful conventions to be iteratively bootstrapped and locked in through a cumulative ratchet effect, preventing multi-partner interference and accelerating community-wide consensus.

Loss & Training

The training scheme is completely decentralized and decoupled between agents: - Speaker Optimization: The speaker optimizes the policy gradient objective \(\mathcal{L}_{\text{speak}}(\theta_S) = - R \log \pi(w_S | e, e_t, \theta_S)\). When an interaction fails (\(R=0\)), no update is performed for the speaker. This deliberate speaker-centric bias reflects natural communication: the speaker's communicative choice is treated as an intentional reference, placing the communicative adaptation burden onto the listener. - Listener Optimization: The listener optimizes \(\mathcal{L}_L(\theta_L) = \beta_p \mathcal{L}_{\text{pred}}(\theta_L) + \beta_i \mathcal{L}_{\text{imit}}(\theta_L)\), with balanced hyperparameters \(\beta_p = 1.0\) and \(\beta_i = 0.2\). In failed interactions, ground-truth disclosure provides strong supervision for representation realignment. - Hyperparameters: Default settings include vocabulary size \(V=100\), hidden size \(H=64\), embedding dimension \(E=64\), learning rate \(\alpha=0.0001\), and temperatures \(\tau_{\text{lse}}=1.0\) and \(\tau_s=0.1\). Notably, no entropy regularization is required.

Key Experimental Results

Main Results

Experiments are conducted across four visual domains: ColorReference (\(C=3, O=3\)), ManyNames (\(C=2, O=512\)), CelebA (\(C=10, O=384\) via DINOv2), and MSCOCO (\(C=10, O=384\) via DINOv2). Baselines comprise No-Imitation (\(\beta_i=0\)), Similarity (selecting symbols via dot product without distractor subtraction), and Speaker-Listener (S-L, fixed functional specialization). Evaluations record Communicative Success (%), Population Coherence / Conventionality (Conv. %, measuring the probability that two randomly selected agents utter the identical symbol for the same scene), and \(V_{\text{used}}^{95\%}\) (number of symbols accounting for 95% of utterances).

Dataset Method Success (%) Conv. (%) \(V_{\text{used}}^{95\%}\)
ColorReference (\(C=3, O=3\)) Ours \(99.13 \pm 0.15\) \(\mathbf{93.18 \pm 0.59}\) \(\mathbf{8.8 \pm 0.42}\)
No-Imitation \(99.54 \pm 0.05\) \(76.46 \pm 1.72\) \(20.5 \pm 1.51\)
Similarity \(85.46 \pm 0.78\) \(52.40 \pm 9.17\) \(21.9 \pm 4.25\)
Speaker-Listener \(97.36 \pm 0.35\) \(5.84 \pm 2.60\) \(30.4 \pm 2.41\)
ManyNames (\(C=2, O=512\)) Ours \(99.61 \pm 0.13\) \(\mathbf{94.11 \pm 0.37}\) \(\mathbf{4.1 \pm 0.32}\)
No-Imitation \(99.11 \pm 0.15\) \(53.81 \pm 4.42\) \(13.6 \pm 1.07\)
Similarity \(82.17 \pm 11.74\) \(15.77 \pm 8.14\) \(19.5 \pm 3.95\)
Speaker-Listener \(97.91 \pm 0.20\) \(1.55 \pm 1.24\) \(19.9 \pm 1.20\)
CelebA (\(C=10, O=384\)) Ours \(97.41 \pm 0.66\) \(\mathbf{82.74 \pm 1.17}\) \(\mathbf{35.9 \pm 2.51}\)
No-Imitation \(98.63 \pm 0.28\) \(68.42 \pm 1.80\) \(76.57 \pm 5.68\)
Similarity \(38.07 \pm 25.09\) \(15.51 \pm 13.06\) \(29.2 \pm 17.61\)
Speaker-Listener \(88.54 \pm 1.46\) \(13.83 \pm 2.70\) \(62.3 \pm 4.57\)
MSCOCO (\(C=10, O=384\)) Ours \(96.53 \pm 0.50\) \(\mathbf{81.91 \pm 1.59}\) \(\mathbf{32.5 \pm 1.35}\)
No-Imitation \(97.91 \pm 0.47\) \(66.51 \pm 1.64\) \(77.6 \pm 1.35\)
Similarity \(9.73 \pm 0.24\) \(1.30 \pm 1.53\) \(9.7 \pm 1.16\)
Speaker-Listener \(82.55 \pm 0.85\) \(10.05 \pm 1.28\) \(57.5 \pm 2.32\)

Ablation Study

1. Impact of Imitation Coefficient \(\beta_i\) and Bi-directionality (CelebA, 2M Interactions)

The comparison below evaluates bi-directional agents against fixed Speaker-Listener populations across varying imitation intensities \(\beta_i \in \{0, 0.2, 1.0\}\), demonstrating that imitation alone cannot rescue conventionality in unidirectional systems.

Architecture Imitation Weight \(\beta_i\) Success (%) Conv. (%) Note
Bi-directional \(0.0\) (No-Imitation) \(98.63 \pm 0.28\) \(68.42 \pm 1.80\) High success but dispersed vocabulary and lower coherence
\(0.2\) (Ours) \(97.41 \pm 0.66\) \(\mathbf{82.74 \pm 1.17}\) Optimal trade-off between task success and convention
\(1.0\) (Strong Imitation) \(89.25 \pm 1.38\) \(87.30 \pm 1.09\) Maximum convention but slight drop in discriminative power
Speaker-Listener \(0.0\) (Standard S-L) \(88.54 \pm 1.46\) \(13.83 \pm 2.70\) Listeners overfit individual speakers; conventions collapse
\(0.2\) \(86.14 \pm 1.01\) \(18.94 \pm 2.25\) Imitation yields negligible alignment gain without bi-directionality
\(1.0\) \(76.62 \pm 2.22\) \(33.83 \pm 5.80\) Severe success penalty to 76.62% with poor convention

2. Population Scaling Properties (ManyNames, 100K Interactions per Agent)

Scaling evaluations from \(N=10\) up to \(N=1000\) agents demonstrate that decentralized cultural evolution reliably achieves high mutual understanding across large populations.

Population Size \(N\) Total Interactions Success (%) Conv. (%) Note
\(N = 10\) \(5 \times 10^5\) \(98.97 \pm 0.19\) \(91.23 \pm 0.50\) Rapid convergence with high coherence
\(N = 100\) \(5 \times 10^6\) \(97.40 \pm 1.11\) \(91.62 \pm 0.80\) Tenfold population increase preserves convention fidelity
\(N = 1000\) \(5 \times 10^7\) \(95.89 \pm 0.16\) \(\mathbf{92.01 \pm 0.28}\) Stable community-wide convention in large decentralized groups

Key Findings

  • Conventions Provide Direct Learnability Advantages: Evaluating new naive listener agents learning pre-converged languages demonstrates that higher conventionality directly accelerates cultural transmission. A new agent requires ~10,000 interactions to achieve 95% proficiency on an unaligned language (\(\beta_i=0\)), but requires fewer than 5,000 interactions on a conventionalized language (\(\beta_i=1.0\)). Compact, consistent vocabularies minimize target ambiguity and enhance sample efficiency.
  • Role Specialization Induces Babel-like Fragmentation: In S-L populations, even when communicating at 95% success, speaker agents develop disjoint symbol preferences (Figure 3 & 8). Listeners compensate through memorizing individual dialects, leading to abysmal conventionality (13.83%–18.94%). In contrast, bi-directional interchangeability forces convergence onto a compact shared code (\(V_{\text{used}}^{95\%} \approx 33\) vs 62–77).
  • Online Single-Sample Updates Outperform Batching: Comparing unbatched interactions (\(B=1\)) against batched baselines (\(B \in \{10, 100, 500\}\)) reveals that increasing batch size substantially degrades both success and conventionality. Even in "paired" baselines where agents interact across multiple scenes before swapping partners, immediate per-game parameter updates consistently yield superior convergence, highlighting the necessity of immediate feedback loops for cultural ratcheting.

Highlights & Insights

  • Cognitive Closure via Bi-directional Symmetry: Enforcing parameter sharing across speaking and listening roles eliminates the perverse incentive for listeners to over-accommodate idiosyncratic speakers. Symmetrical functional pressure compels all participants to negotiate and adhere to a single common symbolic ground.
  • Spontaneous Discovery of Attribute Bundles: Semantic co-occurrence analysis on CelebA reveals that emerging symbols align with coherent attribute bundles (e.g., Symbol 71 capturing "smiling, young, blond hair"; Symbol 93 capturing "male, gray hair, older") rather than isolated atomistic concepts, closely paralleling human lexical categorization.
  • Natural Convergence without Entropy Regularization: Unlike standard reinforcement learning communication architectures that mandate artificial entropy maximization to evade policy collapse, this model converges stably to shared vocabularies through social imitation dynamics and situated discriminative pressures alone.

Limitations & Future Work

  • Single-Symbol Utterance Constraint: Restricting communication to atomic, single-symbol utterances precludes the emergence of compositional syntax and grammatical markers. Extending the agent architecture with autoregressive or recurrent sequence policies represents a critical path to study compositional syntax evolution.
  • Pairwise Interaction Topology: Real-world cultural transmission frequently occurs within group contexts (one-to-many broadcasts). Exploring multicast configurations (\(1\) speaker to \(k\) listeners) could further accelerate convergence dynamics and deepen communal alignment.
  • Fixed Vocabulary Size Hypothesis: Presetting a static vocabulary capacity (\(V=100\)) limits the investigation of open-ended symbolic innovation, where agents actively invent novel tokens in response to communicative deadlocks. Developing dynamically expanding vocabulary mechanisms offers a promising extension.
  • vs S-L Emergent Communication (Chaabouni et al., 2022; Rita et al., 2022a): Prior multi-agent setups enforced static speaker/listener partitions or global representations. This work establishes that functional symmetry and local imitation are essential prerequisites for true linguistic conventions to emerge.
  • vs Iterated Learning Frameworks (Chaabouni et al., 2019; Ren et al., 2020): Traditional neural iterated learning models emphasize vertical transmission across linear chains of single-agent generations. In contrast, this paper formalizes horizontal, decentralized transmission across large, interacting populations of contemporaries.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Pioneering computational simulation of cultural evolution and convention emergence in decentralized, unsupervised neural agent populations.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous validation spanning four visual datasets, population scaling up to 1000 agents, semantic attribute discovery, and learnability transmission curves.
  • Writing Quality: ⭐⭐⭐⭐⭐ Exceptional clarity in theoretical framing, mathematical derivations, and narrative synthesis.
  • Value: ⭐⭐⭐⭐⭐ Provides profound foundational insights and architectural blueprints for evolutionary linguistics, embodied multi-agent coordination, and emergent language alignment.