Skip to content

The Multilingual FrameNet Corpus

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/beatrice-f/mFNC
Area: Multilingual / Machine Translation
Keywords: Frame Semantic Parsing, Cross-Lingual Transfer, Semantic Role Labeling, Multilingual Corpus, Berkeley FrameNet

TL;DR

This paper introduces the Multilingual FrameNet Corpus (mFNC), the first large-scale harmonized benchmark covering 10 typologically diverse languages (1.5M tokens, ~70k sentences), demonstrating that training discriminative and generative parsers on multilingual frame data dramatically outperforms English-only baselines in both multilingual and zero-shot cross-lingual settings.

Background & Motivation

Frame Semantic Parsing (FSP), grounded in Fillmore's Frame Semantics, aims to automatically identify semantic frames evoked by target lexical units (LUs) and classify the textual spans that instantiate their frame elements (FEs). By formalizing raw text into explicit predicate-argument structures and semantic roles, FSP provides an essential semantic foundation for downstream information extraction, knowledge graph construction, perspective detection, and narrative analysis. However, despite over two decades of development, the vast majority of state-of-the-art FSP research remains predominantly tied to the English Berkeley FrameNet (BFN) corpus, leaving multilingual and cross-lingual frame parsing substantially underexplored.

The fundamental barrier hindering cross-lingual frame semantic parsing lies in the interplay between deep cross-lingual morphosyntactic variations and distinct cultural conceptualizations of identical real-world situations. For instance, in the Italian expression "piacere" (to like), the entity being liked syntactically acts upon the experiencer, whereas in English "like" the experiencer is framed as the active agent; similarly, the Korean predicate "๊ณ ์ „ํ•˜๋‹ค" casts a struggling entity as acted upon by an external adversary, whereas English frames "struggle" as an active participant encounter. Although independent FrameNet resources have been developed across different languagesโ€”such as German SALSA, French FrameNet, Chinese FrameNet, and Korean FrameNetโ€”they have evolved in isolation across conflicting methodologies (bottom-up corpus induction, top-down translation projection, and hybrid approaches), exhibiting divergent annotation schemas, inconsistent tokenization schemes, topic-specific domain biases, and unaligned private frame extensions that preclude unified training and benchmarking.

Prior efforts have attempted to bridge this gap either by deploying multilingual pre-trained encoders (e.g., XLM-RoBERTa in LOME) solely fine-tuned on English BFN, or through manual cross-linguistic alignment of tiny sample sets that lack sufficient scale to train statistical parsers. Without a consolidated, harmonized training and evaluation corpus, models inevitably fail to capture cross-lingual syntactic-semantic mappings and suffer from severe performance collapse when applied outside English. Core idea: systematically construct and harmonize the Multilingual FrameNet Corpus (mFNC) by collecting ten heterogeneous language-specific FrameNets into a unified BFN 1.7 semantic space with bidirectional tokenization standardization and balanced multi-label stratification, creating a 1.5M-token benchmark that unlocks superior multilingual and cross-lingual frame semantic parsing.

Method

Overall Architecture

The development and evaluation pipeline of mFNC encompasses four sequential stages: cross-lingual semantic ontology alignment, bidirectional document detokenization and tokenization normalization, iterative multi-label stratified dataset splitting, and comprehensive benchmarking across sequence-tagging and generative parsing paradigms. The pipeline ingests 10 independent language resources, strips language-specific private frames to align with BFN 1.7, constructs dual-track plain and tokenized texts via SacreMoses, balances long-tail frame distributions across train/validation/test splits, and evaluates both the multi-stage discriminative parser (LOME) and sequence-to-sequence generative parsers (mT5).

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Raw Multi-Source FrameNet Corpora<br/>(10 Heterogeneous Languages)"] --> B["Cross-Lingual Semantic Ontology and Schema Harmonization"]
    B --> C["Bidirectional Document Reconstruction and Tokenization Normalization"]
    C --> D["Stratified Multi-Label Sample Balancing Strategy"]
    D --> E["Sequence-Tagging and Generative Multilingual Parsing Benchmark"]
    E --> F["Downstream Multilingual and Zero-Shot Cross-Lingual Evaluation"]

Key Designs

1. Cross-Lingual Semantic Ontology and Schema Harmonization: Resource Heterogeneity and Private Frame Conflicts -> Strict Mapping to BFN 1.7 Core Subsets -> Unified 10-Language Semantic Interoperability The original 10 FrameNet resources adhere to divergent lexicographic methodologies: German, French, Italian, and Chinese follow bottom-up corpus annotation preserving native idiomatic structures but introducing idiosyncratic private frames; Korean follows a top-down projection from English BFN translations; and Swedish adopts a hybrid expansion. To prevent private label incompatibility during cross-lingual joint training, the authors enforced a strict ontology alignment protocol: all language-specific private frames were systematically stripped (while being archived in an extended supplementary release for specialized lexical research), and all remaining annotations were rigorously projected onto the canonical Berkeley FrameNet 1.7 frame and frame-element inventory. This yielded an aligned multilingual semantic inventory comprising 1,070 unique frames and 8,954 unique frame elements across 1.5 million tokens.

2. Bidirectional Document Reconstruction and Tokenization Normalization: Inconsistent Text Formats and Missing Raw Documents -> Closed-Loop Language-Specific Detokenization and Tokenization -> Universal Character and Subword Boundary Consistency A major technical impediment in integrating historical language-specific FrameNets is the divergence in textual representations: several source datasets provided only pre-tokenized whitespace-separated tokens, discarding original sentence typography, while others provided only raw continuous text with custom segmentation conventions. To resolve boundary discrepancies across tokenizers, the authors deployed language-specific detokenizers from the SacreMoses library to reconstruct full-text continuous documents from tokenized sources, while conversely applying SacreMoses tokenizers to raw text corpora. This bidirectional pipeline guarantees dual availability of both raw untokenized text and standardized token streams, ensuring seamless alignment for modern subword-based neural architectures.

3. Stratified Multi-Label Sample Balancing Strategy: Steep Zipfian Frame Skew and Uncontrolled Splits -> Iterative Multi-Label Stratification and Exemplar Pruning -> Leakage-Free and Statistically Robust Generalization Frame frequencies exhibit an extreme Zipfian long-tail distribution: ubiquitous frames like Causation and Statement appear thousands of times, while rare frames like Causation_scenario appear only once, and approximately 12% of BFN frames never occur in any corpus. Furthermore, non-English resources lacked canonical train/validation/test partitions. To prevent rare frames from being entirely omitted from test sets or concentrated in training subsets, the authors implemented an iterative multi-label stratification algorithm that optimizes for balanced multi-label frame distributions across splits. For the English portion, the authors reused the full-text split established by Swayamdipta et al. (2017) and discarded non-representative, incompletely annotated exemplar sentences, adhering to empirical findings that exemplar training introduces label noise and impairs generalization.

4. Sequence-Tagging and Generative Multilingual Parsing Benchmark: Lack of Standardized Multilingual Baselines -> Comparative Evaluation of Discriminative CRF vs. Seq2Seq Sentinel Generation -> Establishing State-of-the-Art Cross-Lingual Protocols To establish a rigorous parsing benchmark, two contrasting neural paradigms were adapted: the discriminative sequence-labeling architecture LOME, which fine-tunes a multilingual XLM-RoBERTa encoder driving a Conditional Random Field (CRF) for candidate span detection followed by dual MLPs for frame and FE classification; and the generative text-to-text paradigm using mT5 (Small and Base) fine-tuned with sentinel tokens for end-to-end linearized span and label prediction. By training and evaluating both architectures under English-only (BFN) and multilingual (mFNC) conditions, the benchmark systematically evaluates model resilience against span boundary degradation and error propagation from frame identification to argument classification.

Corpus Construction and Cross-Lingual Validation Protocol

To quantitatively measure topical divergence and annotation bias across the 10 language resources, the authors devised the Frame Frequency-Inverse Corpus Frequency (FFICF) metric, adapting TF-IDF to the semantic frame level. For a language sub-corpus \(l \in L\) within the set of all languages \(L\) and a frame \(f\), typicality is computed as:

\[FFICF(f, l) = FF(f, l) \times \log\left(\frac{|L|}{|\{l' \in L : f \in l'\}|}\right)\]

where \(FF(f, l)\) represents the raw count of frame \(f\) in language \(l\). Computing cosine similarity between language centroid vectors in FFICF space reveals structural clustering: Dutch and French diverge drastically from other corpora due to narrow domain selection (Dutch is restricted to disaster events like wildfires and disease outbreaks, making Catastrophe its top frame; French centers on legal, medical, and parliamentary debates); conversely, Korean and Swedish exhibit high cosine similarity to English BFN due to their translation-projective annotation origins.

For evaluation, traditional exact-match span metrics are notoriously unforgiving of minor token boundary shifts. The protocol adopts the FairEval framework, which provides fine-grained, micro-averaged precision, recall, and F1 scores by assigning partial credit to spans that are correctly identified in extent but misclassified in label, or whose predicted boundaries partially overlap with gold spans. This prevents catastrophic score collapse in languages with differing morphological agglutination and word-boundary complexities.

Key Experimental Results

Main Results

The primary benchmark evaluated LOME and mT5 models trained either on English BFN or on the multilingual mFNC, tested across the combined 10-language mFNC test set using FairEval micro-averaged metrics. The Frame columns measure target LU identification and frame attribution; the FE columns measure end-to-end argument span identification and role classification (which accumulates frame classification errors).

Model Architecture Training Data Frame P Frame R Frame F1 FE P FE R FE F1
LOME BFN (English only) 0.25 ยฑ 0.23 0.60 ยฑ 0.22 0.32 ยฑ 0.22 0.15 ยฑ 0.18 0.33 ยฑ 0.20 0.19 ยฑ 0.18
LOME mFNC (Multilingual) 0.77 ยฑ 0.11 0.65 ยฑ 0.20 0.69 ยฑ 0.15 0.61 ยฑ 0.15 0.54 ยฑ 0.18 0.56 ยฑ 0.15
mT5 BFN (English only) 0.14 ยฑ 0.15 0.43 ยฑ 0.19 0.19 ยฑ 0.15 0.08 ยฑ 0.10 0.19 ยฑ 0.13 0.10 ยฑ 0.10
mT5 mFNC (Multilingual) 0.53 ยฑ 0.12 0.57 ยฑ 0.13 0.55 ยฑ 0.12 0.40 ยฑ 0.13 0.41 ยฑ 0.13 0.40 ยฑ 0.13
mT5-small BFN (English only) 0.11 ยฑ 0.12 0.37 ยฑ 0.17 0.16 ยฑ 0.13 0.06 ยฑ 0.08 0.15 ยฑ 0.10 0.08 ยฑ 0.09
mT5-small mFNC (Multilingual) 0.59 ยฑ 0.14 0.60 ยฑ 0.14 0.59 ยฑ 0.14 0.42 ยฑ 0.13 0.42 ยฑ 0.14 0.42 ยฑ 0.13

On the standard English BFN target classification benchmark, LOME trained on mFNC achieves an F1 of 0.812, surpassing its BFN-trained counterpart (0.800) and previous state-of-the-art systems including Swayamdipta et al. (0.733), Lin et al. (0.763), and Devasier et al. (0.775), confirming that multilingual training introduces positive regularization without degrading English capabilities.

Zero-Shot Cross-Lingual Generalization Ablation

To verify whether multilingual training induces language-agnostic semantic frame abstractions, the authors conducted a leave-one-language-out ablation: Swedish (SW) was completely removed from the mFNC training set, and LOME was evaluated zero-shot on the Swedish test set.

Training Setup (LOME) Frame P Frame R Frame F1 FE P FE R FE F1
BFN (English-only training) 0.07 0.26 0.11 0.04 0.20 0.07
mFNC \ SW (Trained on 9 languages w/o Swedish) 0.30 0.28 0.29 0.20 0.20 0.20
mFNC (Full 10-language training) 0.55 0.42 0.47 0.36 0.33 0.35

Key Findings

  • Massive Performance Leap from Multilingual Data: Moving from English BFN training to mFNC training increases LOME's aggregate Frame F1 from 0.32 to 0.69 (+0.37 absolute) and end-to-end FE F1 from 0.19 to 0.56 (+0.37 absolute). Individual language gains are dramatic: German Frame F1 surges from 0.346 to 0.927, French from 0.100 to 0.880, and Latvian from 0.231 to 0.577.
  • Cross-Lingual Generalization Beyond Seen Languages: In the zero-shot Swedish ablation, training on the other 9 languages (mFNC \ SW) lifts Swedish Frame F1 from 0.11 to 0.29 and FE F1 from 0.07 to 0.20 compared to English-only training, demonstrating that exposure to diverse multilingual syntax facilitates zero-shot semantic transfer to unseen languages.
  • Sequence Tagging Superiors Generative Seq2Seq: Across all metrics and languages, the discriminative sequence-labeling pipeline (LOME) substantially outperforms generative Seq2Seq models (mT5/mT5-small), showing that structured span extraction via CRF layers provides far greater boundary fidelity and resistance to error accumulation than free-form autoregressive token generation.

Highlights & Insights

  • Unified Foundation for Multilingual Frame Semantics: mFNC resolves 20+ years of fragmentation in frame semantic resources, unifying 10 distinct language corpora under BFN 1.7 into a standard benchmark of 1.5M tokens, 114k frames, and 215k frame elements.
  • FFICF Centroid Profiling: Adapting TF-IDF into Frame Frequency-Inverse Corpus Frequency provides an elegant mathematical tool to quantify domain bias and stylistic convergence across multilingual corpora.
  • Zero Penalty for Multilingualism: Adding 9 non-English languages does not trigger negative transfer on English; on the contrary, English target classification F1 improves from 0.800 to 0.812, demonstrating cross-linguistic positive reinforcement in semantic parsing.

Limitations & Future Work

  • Skew Toward High-Resource Languages: The current 10 languages are predominantly European and East Asian; African, Middle Eastern (e.g., Arabic), and South Asian (e.g., Bengali) languages remain unrepresented.
  • Deterministic Single-Frame Assumption: FSP models and benchmarks assume a single ground-truth frame per LU, neglecting legitimate interpretative ambiguity and perspectivism where multiple frames validly co-exist.
  • Potential Cultural Distortion of BFN Projections: Forcing non-English conceptualizations into an English-centric BFN 1.7 frame ontology may flatten unique language-specific semantic distinctions and constructional nuances.
  • vs BFN (Baker et al., 1998): BFN created the foundational English frame semantic taxonomy; mFNC scales this taxonomy globally by standardizing and aligning 9 additional language corpora.
  • vs LOME (Xia et al., 2021): LOME introduced multilingual XLM-RoBERTa backbones for FSP but only trained and evaluated on English BFN; this work deploys LOME across true multilingual and cross-lingual testbeds, setting new SOTA baselines.
  • vs Global FrameNet (Gilardi & Baker, 2018): Global FrameNet focused on lexical alignment and small-scale shared annotation exercises; mFNC delivers a full-scale, ready-to-train machine learning corpus with standardized splits.

Rating

  • Novelty: โญโญโญโญโญ Creates the first large-scale, standardized 10-language benchmark for Frame Semantic Parsing.
  • Experimental Thoroughness: โญโญโญโญโญ Comprehensive FairEval metric reporting across 10 languages, contrasting sequence-tagging vs. Seq2Seq, with leave-one-out cross-lingual ablations.
  • Writing Quality: โญโญโญโญโญ Rigorous exposition detailing linguistic conceptualization differences, corpus provenance, and error propagation mechanics.
  • Value: โญโญโญโญโญ Essential public infrastructure for multilingual semantic parsing, computational linguistics, and cross-lingual information extraction.