Code-Switching and Syntax: A Large-Scale Experiment¶

Conference: ACL 2025 (Findings)
arXiv: 2506.01846
Code: 无
Area: 其他
Keywords: 语码转换, 句法分析, 双语处理, 最小对实验, 跨语言泛化

TL;DR¶

Through large-scale, multilingual, and cross-phenomenon experiments, this study systematically validates the linguistic consensus that "syntactic information is sufficient to explain code-switching (CS) patterns" for the first time. Using only syntactic features, the model achieves judgment accuracy comparable to bilingual humans, and the learned syntactic patterns generalize to unseen language pairs.

Background & Motivation¶

Background: Code-switching (CS) is a natural phenomenon where bilinguals switch languages within a sentence. Theoretical linguistics literature contains numerous pointwise investigations on specific language pairs and grammatical phenomena to explain why bilinguals prefer to switch languages at certain positions in a sentence.

Limitations of Prior Work: Although a broad consensus has emerged that CS can be explained by the syntax of the participating languages, this conclusion lacks large-scale, multilingual, and cross-phenomenon experimental validation. Most existing studies focus on a single language pair or syntactic phenomenon, failing to yield comprehensive and reproducible quantitative conclusions.

Key Challenge: Designing such experiments faces a critical challenge: how to ensure that the prediction system only utilizes syntactic information, without relying on lexical, semantic, or other non-syntactic cues. If the system incorporates other information, even if it achieves good results, it cannot prove the proposition that "syntax is sufficient to explain CS."

Goal: To design a strictly controlled large-scale experiment to verify whether syntactic information alone can distinguish natural CS sentences from artificially constructed minimal pairs, and to test whether this capability can generalize across languages.

Key Insight: The authors construct minimal pairs of CS sentences, where each pair is identical in lexical content but differs only in the position of code-switching. This control ensures that the model can only make decisions based on syntactic differences.

Core Idea: To validate the long-standing theoretical hypothesis that "syntax is sufficient to explain CS patterns" on large-scale multilingual data using the minimal pair paradigm combined with syntax-only feature encoding.

Method¶

Overall Architecture¶

The entire experimental pipeline consists of three steps: (1) automatically constructing a large-scale CS minimal pair dataset from multilingual parallel corpora; (2) encoding CS sentences with syntax-only features (dependency trees, POS tags, etc.) and training a classifier to distinguish natural CS from unnatural CS; (3) testing generalization capability on unseen language pairs and comparing performance against human bilingual judgments.

Key Designs¶

Minimal Pair Construction:
- Function: Generates CS sentence pairs with controlled lexical content and differing only in syntactic positions.
- Mechanism: Given a bilingual sentence pair (source + translation + alignment), language switching is performed at different syntactic boundaries to generate a set of minimal pairs. Natural CS positions are derived from switching frequency statistics in actual bilingual corpora, while unnatural positions are low-frequency or unseen switching points. This approach avoids lexical and semantic confounding.
- Design Motivation: By only changing the switching position, it ensures that the classifier's judgments are entirely based on syntactic structural differences.
Syntax-Only Feature Encoding:
- Function: Encodes CS sentences into syntax-only representations, stripping away all lexical and semantic information.
- Mechanism: Uses a dependency parser to parse CS sentences and extracts features such as dependency types, POS tags, and syntactic links at the switching point. All lexical forms are replaced with syntactic role tags, ensuring that the classifier cannot utilize word-form details.
- Design Motivation: To strictly control information sources and prevent the classifier from exploiting non-syntactic cues like n-grams or lexical co-occurrences.
Cross-Lingual Generalization:
- Function: Validates whether the learned syntactic patterns are language-independent.
- Mechanism: Train the model on a subset of language pairs and test it on entirely unseen language pairs to observe the degree of performance degradation. Good generalization indicates that syntactic constraints in CS possess cross-lingual universality.
- Design Motivation: To address debates in theoretical linguistics regarding the universality of CS constraints.

Loss & Training¶

A standard binary cross-entropy loss is employed to classify the minimal pairs as "natural vs. unnatural". Training uses a mixture of multilingual data, and evaluation is conducted on seen and unseen language pairs separately during testing.

Key Experimental Results¶

Main Results¶

Evaluation Setup	Accuracy	Comparison with Humans	Description
Seen Language Pairs	~82%	≈ Human Level	Syntax-only features
Unseen Language Pairs	~78%	Close to Human Level	Cross-lingual generalization
Human Bilinguals	~83%	—	Upper bound
Random Baseline	50%	—	Binary choice on minimal pairs

Ablation Study¶

Feature Configuration	Accuracy	Description
Full Syntactic Features	~82%	Dependency + POS + Context at switching point
Dependency Only	~75%	Excluding POS tag information
POS Tags Only	~70%	Excluding dependency structure
Adding Lexical Features	~85%	Non-pure syntax, as an upper-bound reference

Key Findings¶

Pure syntactic features are sufficient for the classifier to achieve a judgment level comparable to human bilinguals (~82% vs. ~83%), directly supporting the theoretical hypothesis that "syntax is sufficient to explain CS".
Cross-lingual generalization performance drops by only about 4%, demonstrating that syntactic constraints in CS are highly language-independent.
Dependency relation types are the most informative features, indicating that CS patterns are primarily driven by hierarchical relationships in syntactic structures.
On certain language pairs (such as morphologically rich languages), the syntactic model's performance is even more prominent.

Highlights & Insights¶

Rigorous Experimental Design: Syntactic information is perfectly isolated through the minimal pair paradigm. This methodology is highly worth adapting for other empirical linguistics experiments. Traditional studies are often less strict with controlled variables.
Automation of Theoretical Verification: Bridging the gap between theory and computation, this work converts a theoretical linguistics hypothesis into an automated, large-scale experiment.
Cross-Lingual Universality: The capability to generalize to unseen language pairs suggests the existence of a cross-lingual "universal syntactic constraint." This inspires multilingual NLP model design: syntactic representations can serve as an effective, language-independent intermediate representation.

Limitations & Future Work¶

The study only validates that "syntax is sufficient," but does not rule out other factors (such as prosody and pragmatics) that may also play a role — syntax could be a sufficient but not a necessary condition.
Minimal pair construction relies on the quality of automatic alignment and dependency parsing; parsing errors may introduce noise.
The CS data in the experiments stems from written text; spoken CS may have different syntactic constraint patterns.
Intra-word (morphological) CS is not covered, which is very common in certain language pairs.
Future work could extend this method to more language families and conduct fine-grained analyses of syntactic phenomena.

vs. Poplack (1980) and other classic CS theories: Classic theories proposed concrete syntactic constraints (e.g., Equivalence Constraint, Free Morpheme Constraint) but were mostly evaluated on small-scale data. This study provides empirical evidence for the more general proposition that "syntax is sufficient" through large-scale experiments.
vs. LM-based CS prediction methods: Some existing works use pre-trained language models to predict CS positions, but these models conflate lexical and syntactic information. This paper's contribution lies in strictly isolating and stripping away non-syntactic signals.
Inspiration for Multilingual NLP: If CS is primarily syntax-driven, then syntax-aware multilingual models could hold a natural advantage when processing CS texts.

Rating¶

Novelty: ⭐⭐⭐⭐ Systematic verification of classical theoretical hypotheses in a large-scale, cross-lingual manner for the first time, though the methodology is more confirmatory than highly innovative.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Multilingual, cross-phenomenon, with human evaluation, ablation studies, and generalization tests.
Writing Quality: ⭐⭐⭐⭐ Outstanding clarity in logic; the progression from motivation to experiments is natural.
Value: ⭐⭐⭐⭐ Provides crucial computational validation for linguistic theories, offering valuable guidance to the CS-NLP community.