Skip to content

Syntactic Complexity Convergence in Dialogue: Analyzing Human-LLM Alignment Dynamics

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/owuQQQ/syntactic_complexity_in_dialogue
Area: LLM Agent
Keywords: Syntactic Complexity Convergence, Dialogue Alignment Dynamics, Human-LLM Interaction, LLM Detection, Linear Centered Kernel Alignment

TL;DR

By quantifying dependency-based syntactic complexity across four structural dimensions, this work reveals a fundamental dichotomy between within-metric convergence in human dyads and cross-metric compensatory alignment in human-LLM dyads (demonstrated via statistically significant negative Linear-CKA slopes), leveraging these structural dynamics to build an LLM-involved dialogue detector that attains 96.0% accuracy without language-model-based likelihood estimation.

Background & Motivation

In human dialogue, mutual understanding is progressively established through coordinated linguistic adaptation that minimizes communicative effort and cognitive load. Grounded in the Interactive Alignment Model, interlocutors synchronize their linguistic representations across multiple processing tiers, ranging from surface-level lexical repetition to deeper syntactic structures. However, prior investigations into syntactic complexity matching have remained confined to small-scale corpora, isolated grammatical units such as Hunt's T-units, or localized topic episodes. Consequently, whether structural complexity convergence constitutes a robust, macroscopic phenomenon spanning entire multi-turn interactions across typologically diverse languages has remained an open empirical question.

The pervasive deployment of Large Language Models (LLMs) into conversational systems and interactive agents introduces a pressing behavioral inquiry: when an interlocutor is replaced by an autoregressive neural agent, do human-LLM interactions exhibit structural complexity matching comparable to human dyads? Emerging evidence has demonstrated that LLM agents can display surface accommodations such as lexical mimicry or utterance length mirroring, but these observations overlook high-level hierarchical syntactic topology—including clausal embedding depth, dependency arc distributions, and constituent density. Crucially, if human-LLM dialogues deviate systematically from human-human communication in their structural adaptation dynamics, could such divergence be harnessed to detect machine-generated conversations without incurring the prohibitive computational footprint of model-based likelihood evaluation?

Addressing the temporal heterogeneity of natural dialogue and the latent discrepancies between biological and synthetic communicative alignment requires transcending univariate utterance statistics. Core idea: integrate tree depth, dependency length, syntactic complexity score, and complex dependency ratio into a multidimensional framework, employ phase-normalized Linear Centered Kernel Alignment (Linear-CKA) to uncover the contrast between human within-metric convergence and LLM cross-metric compensation, and map complexity fluctuations via Discrete Fourier Transform into a model-free dialogue-level LLM detector.

Method

Overall Architecture

The methodology consists of three interconnected modules: multidimensional syntactic complexity quantification, 8-phase longitudinal alignment dynamics modeling, and model-free dialogue classification via spectral feature extraction. First, unstructured multi-turn dialogue utterances are parsed into dependency trees under Universal Dependencies, yielding four complementary structural metrics averaged at the turn level. Second, to overcome interaction length discrepancies, qualified dialogues are segmented into eight uniform chronological phases, tracking initiator-follower convergence through null-hypothesis permutation tests and multivariate Linear-CKA regression. Finally, temporal complexity trajectories are transformed via the Discrete Fourier Transform (DFT) to extract dominant spectral amplitudes, feeding a lightweight classifier trained under quantile-based length-balanced sampling.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Multi-turn Dialogue Input<br/>(Human-Human / Human-LLM Interaction)"] --> B["Four-dimensional Syntactic Complexity Quantification<br/>(Max Tree Depth + ADL + Complexity Score + Complex Ratio)"]
    B --> C["8-Phase Temporal Convergence Dynamics<br/>(Phase-wise Averaging & Turn-Shuffling Null Permutation)"]
    C --> D["Multivariate Cross-Metric Alignment Modeling<br/>(Linear-CKA Matrix Distance & Linear Regression Slopes)"]
    D --> E["Discrete Fourier Spectral Feature Extraction<br/>(Top-k Power Spectra & Model-Free Dialogue Detection)"]
    E --> F["Dialogue Classification Output<br/>(Natural Human-Human vs. LLM-Involved Dyad)"]

Key Designs

1. Four-dimensional Syntactic Complexity Quantification: Capturing structural volume and hierarchical depth Spoken human dialogue frequently exhibits disfluencies, self-repairs, fragmented clauses, and discourse fillers (e.g., "uh", "um"), which destabilize traditional constituency parsers. To ensure parsing robustness across spoken registers and diverse languages, the framework utilizes the Universal Dependencies-compliant Stanza parser to construct directed syntactic trees. Rather than relying on a single structural proxy, the framework integrates four mutually validating measures: - Maximal Tree Depth \(\beta\): Defined as the longest directed path length from the root token to any leaf node, indexing the vertical hierarchical nesting of the utterance; - Average Dependency Length (ADL): Quantified as the mean absolute linear distance between syntactic heads and their dependents across dependency set \(D\), reflecting transient parsing memory burden: $\(\text{ADL} = \frac{1}{|D|} \sum_{(h, d) \in D} |h - d|\)$ - Syntactic Complexity Score (SC): Formulated to blend linear horizontal span and vertical hierarchy by combining token length \(L\), the total count of syntactic heads \(\alpha\) (including the root), and tree depth \(\beta\) under a balancing factor \(\lambda = 0.5\): $\(\text{SC} = \begin{cases} \lambda \cdot \frac{L}{\alpha} + (1 - \lambda) \cdot \beta & \text{if } \alpha > 0 \\ (1 - \lambda) \cdot \beta & \text{otherwise} \end{cases}\)$ - Complex Dependency Structure Ratio (Complexratio): Computed as the proportion of dependency arcs exhibiting a linear distance \(|h - d| > 3\), isolating embedded relative clauses and non-local syntactic dependencies.

2. 8-Phase Temporal Convergence Dynamics: Harmonizing dialogue length variance across interactional trajectories Conversations vary widely in duration (ranging from a handful of turns to hundreds of exchanges), making raw chronological alignment incomparable. Grounded in conversational psycholinguistics, qualified dialogues (requiring at least 8 turns per participant) are partitioned into eight chronologically uniform phases \(P_1, P_2, \dots, P_8\). For each phase, the mean metric value is computed for both the Dialogue Initiator and Follower. A dialogue is identified as convergent if the final inter-speaker gap contracts relative to the opening phase (\(|P_1^{\text{init}} - P_1^{\text{fol}}| > |P_8^{\text{init}} - P_8^{\text{fol}}|\)) and exhibits at least two progressive reductions across intermediate transitions. To rigorously confirm that empirical convergence ratios are not artifacts of statistical noise, a permutation procedure is established: within-dialogue turn positions are randomly shuffled \(B\) times (1,000 to 1,500 iterations) while preserving participant identities and turn lengths, creating empirical null distributions against which two-tailed \(p\)-values and Cohen's \(d\) effect sizes are computed.

3. Multivariate Cross-Metric Alignment Modeling: Identifying high-level compensatory alignment in non-human dyads Recognizing that linguistic alignment across human and synthetic agents may manifest through cross-representational adaptation rather than identical univariate mirroring (e.g., an interlocutor adapting to shorter clause lengths by adjusting dependency distances), the phase-wise interaction is framed as multi-metric state matrices \(X_p \in \mathbb{R}^{T \times 4}\) and \(Y_p \in \mathbb{R}^{T \times 4}\), where \(T\) denotes the turn count in phase \(p\) and 4 indexes the integrated metrics. The framework employs Linear Centered Kernel Alignment (Linear-CKA) to compute normalized structural representation distance: $\(d_{\text{CKA}}(X_p, Y_p) = 1 - \frac{\|X_p^\top Y_p\|_F^2}{\|X_p^\top X_p\|_F \|Y_p^\top Y_p\|_F}\)$ where \(\|\cdot\|_F\) represents the Frobenius norm. Metric features undergo MinMax normalization prior to distance evaluation. The sequence of phase distances \(d_{\text{CKA}}\) across all eight phases is fitted using ordinary least squares linear regression. A statistically significant negative slope (\(p < 0.05\)) serves as the empirical hallmark of cross-metric convergence, capturing systemic adaptation dynamics that are completely invisible to univariate tests.

4. Discrete Fourier Spectral Feature Extraction: Likelihood-free detection of machine-involved interactions Because syntactic complexity fluctuates cyclically across sequential dialogue turns—mirroring dynamic cognitive processing load—the framework applies the Discrete Fourier Transform (DFT) to the discrete complexity time series \(s^{(d, m)}_t\) of dialogue \(d\) for metric \(m\), obtaining the power spectrum \(p^{(d, m)}_k = |\hat{s}^{(d, m)}_k|^2\). Concatenating the Top-30 frequency spectral peaks across all four metrics yields a compact, fixed-dimensional feature vector \(\mathbf{x}(d) = \mathbf{x}^{(d, 1)} \mathbin{\Vert} \mathbf{x}^{(d, 2)} \mathbin{\Vert} \mathbf{x}^{(d, 3)} \mathbin{\Vert} \mathbf{x}^{(d, 4)}\). To eliminate length bias stemming from the fact that human-LLM interactions tend to have substantially fewer turns than human conversations, a quantile binning procedure balances dialogue length distributions across classes prior to training. The resulting classifier operates entirely on syntactic tree properties, obviating the need for autoregressive token likelihood evaluations or GPU-heavy foundation model inference.

Loss & Training

Two classifier architectures were trained and benchmarked: - Logistic Regression: Trained with balanced class weights using standard binary cross-entropy loss, with maximum iterations set to 1,000; - Transformer Encoder: Configured with embedding dimension \(D=128\), 4 attention heads, 2 encoder layers, a feedforward expansion dimension of 256, a classification hidden layer of 64, and dropout of 0.1. Optimized using Adam (\(\text{lr} = 10^{-3}\)) paired with a ReduceLROnPlateau scheduler (factor 0.5, patience 5) over 20 epochs with a batch size of 32.

Key Experimental Results

Main Results

The empirical evaluation was conducted across five human-human dialogue corpora (MUNDEX in German; Hazumi and BTSJ in Japanese; Candor and Map Task in English) and the LMSYS-Chat-1M corpus (German and English human-LLM interactions). Table 1 presents observed convergence ratios alongside null permutation baselines under the 8-phase protocol. Table 2 details the cross-metric Linear-CKA linear regression slopes.

Dataset (\(N\) Dialogues) Dialogue Type / Language Target Metric Observed Ratio Random Mean \(p\)-value Cohen's \(d\) [95% CI]
MUNDEX (50) Task-oriented / German SC Score
Dependency Length
Tree Depth
Complex Dep. Ratio
0.7600***
0.6600**
0.5600***
0.6000*
0.4775
0.4819
0.2657
0.4662
<0.001
0.009
<0.001
0.032
4.150 [0.189, 0.374]
2.538 [0.085, 0.276]
4.962 [0.189, 0.410]
1.899 [0.034, 0.238]
Hazumi (33) Task-oriented / Japanese SC Score
Dependency Length
Tree Depth
Complex Dep. Ratio
0.5152
0.7576**
0.3333
0.6970**
0.4864
0.4937
0.3723
0.4942
0.438
0.002
0.750
0.010
0.347 [-0.095, 0.164]
3.036 [0.156, 0.376]
-0.490 [-0.162, 0.082]
2.376 [0.094, 0.339]
BTSJ (498) Spontaneous / Japanese Complex Dep. Ratio 0.5321* 0.4917 0.036 1.800 [0.003, 0.073]
Candor (1855) Spontaneous / English Complex Dep. Ratio 0.4642† 0.4467 0.066 1.527 [0.000, 0.033]
Map Task (121) Task-oriented / English SC Score 0.4380 0.4875 0.899 -1.138 [-0.114, 0.019]
LLM_German (234) Human-LLM / German SC Score 0.4316 0.4822 0.945 -1.553 [-0.097, -0.003]
LLM_English (7850) Human-LLM / English SC Score 0.4122 0.4726 1.000 -11.013 [-0.068, -0.052]

In the multivariate cross-metric Linear-CKA setting, regression analysis revealed an inverted behavioral dynamic between natural and machine-involved dyads:

Dataset Regression Slope Coefficient of Determination \(R^2\) Standard Error \(p\)-value Empirical Conclusion
MUNDEX -0.0015 0.0142 0.0050 0.7790 Not Significant
BTSJ -0.0046 0.9077 0.0006 0.0003 Significant Cross-Metric Convergence
Hazumi +0.0075 0.1701 0.0067 0.3099 Not Significant
Candor -0.0004 0.0047 0.0026 0.8721 Not Significant
Map Task -0.0011 0.1644 0.0010 0.3189 Not Significant
LLM (DE) -0.0174 0.9099 0.0022 0.0002 Robust Cross-Metric Convergence
LLM (EN) -0.0156 0.9480 0.0015 <0.0001 Robust Cross-Metric Convergence

Ablation Study & Classification Benchmarks

Evaluating on 21,855 English dialogues (Candor vs. LMSYS-Chat-1M; majority-class baseline 91.5%) under length-balanced quantile sampling demonstrated that syntactic spectral features achieve parity with model-based surprisal estimators:

Feature Representation Classifier Model F1 Score (Std) Accuracy (Std) ROC-AUC (Std)
Syntactic Complexity (Syn Com) Spectrum Length (Baseline) 0.745 (0.074) 0.753 (0.063) 0.757 (0.067)
Syntactic Complexity (Syn Com) Logistic Regression 0.962 (0.042) 0.960 (0.047) 1.000 (0.000)
Syntactic Complexity (Syn Com) Transformer Encoder 0.947 (0.053) 0.947 (0.053) 0.993 (0.012)
Surprisal (FourierGPT Baseline) Spectrum Length (Baseline) 0.955 (0.012) 0.956 (0.011) 0.976 (0.016)
Surprisal (FourierGPT Baseline) Logistic Regression 0.972 (0.014) 0.972 (0.014) 0.997 (0.002)
Surprisal (FourierGPT Baseline) Transformer Encoder 0.961 (0.017) 0.960 (0.018) 0.994 (0.004)

In a robustness ablation filtering out short interactions (retaining 7,850 human-LLM dialogues with \(\ge 8\) turns per participant), syntactic complexity classification accuracy rose to 0.978 (0.017) with F1 of 0.979 (0.016), demonstrating that performance is not an artifact of turn count sparsity.

Key Findings

  • Task-oriented human dialogue exhibits pronounced within-metric convergence: In MUNDEX, explainers systematically simplify utterance syntax over time while explainees produce increasingly articulated queries, resulting in strong within-metric convergence (e.g., Tree Depth Cohen's \(d = 4.962, p < 0.001\)).
  • Human-LLM dialogue exhibits orthogonal cross-metric alignment: While LLM-involved conversations fail to demonstrate statistically significant convergence across individual metrics (\(p \ge 0.60\)), Linear-CKA reveals strong, statistically significant downward slopes (\(-0.0174\) in German, \(-0.0156\) in English; both \(R^2 > 0.90\)), confirming that conversational LLMs adapt holistically across structural feature spaces rather than mirroring specific syntactic dimensions.
  • High-precision model-free LLM detection is practical: Syntactic complexity Fourier spectra achieve 96.0% accuracy and 1.000 AUC without requiring token likelihood evaluations from foundation models, providing an efficient defense against synthetic dialogue injection.

Highlights & Insights

  • Integrates full-dialogue structural metrics with Linear-CKA representation geometry, resolving the theoretical debate over whether conversational LLMs adapt to human communicative complexity.
  • Proves that while humans converge within isomorphic grammatical dimensions, human-LLM dyads engage in cross-metric structural compensation, establishing a new paradigm for psycholinguistic alignment studies.
  • Introduces an ultra-lightweight detection approach that processes raw syntactic dependency trees rather than autoregressive forward passes, bypassing the need for white-box model access.

Limitations & Future Work

  • Language representation skews heavily toward English and German in the human-LLM corpus; future work should examine non-Indo-European languages at scale.
  • Reliance on the Stanza dependency parser introduces potential parsing variance across informal spoken transcripts, warranting testing with specialized spoken-language parsers.
  • Uniform 8-phase partitioning is a heuristic simplification; adaptive topic-boundary segmentation represents a promising future enhancement.
  • LLM architectures and parameter scales were pooled in the dataset; whether larger parameter scales or specialized alignment techniques (e.g., RLHF vs. DPO) recover human-like within-metric convergence remains unexplored.
  • vs Xu and Reitter (2016): Prior work investigated syntactic complexity matching primarily on isolated T-units or local topic shifts; this paper scales the analysis to entire conversations across five corpora in three languages, using permutation tests to ensure statistical validity.
  • vs FourierGPT (Xu et al., 2024): FourierGPT relied on language models to compute token-level surprisal time series; this work adapts DFT to parameter-free syntactic complexity sequences, eliminating model dependency while retaining 96.0% classification accuracy.
  • vs Kandra et al. (2025) / Blevins et al. (2026): While prior studies documented surface accommodation (e.g., sentence length) in artificial agents, this work proves via Linear-CKA that human-LLM alignment operates as a multidimensional compensatory mechanism distinct from natural human alignment.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ First macroscopic full-dialogue syntactic convergence study identifying cross-metric alignment dynamics and proposing a likelihood-free detector.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Evaluated across 3 languages, 7 corpora, combining rigorous permutation null testing, CKA regression, and length-balanced ablations.
  • Writing Quality: ⭐⭐⭐⭐⭐ Grounded deeply in psycholinguistic theory, with clear formal derivations, clean experimental reporting, and transparent visualizations.
  • Value: ⭐⭐⭐⭐⭐ Substantial contributions to human-agent cognitive alignment theory and computationally efficient detection of synthetic interactions.