Skip to content

ParaBridge: Bridging Paralinguistic Perception and Dialogue Behavior in Speech Language Models

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/AmphionTeam/ParaBridge
Area: Audio & Speech
Keywords: Speech Language Models, Paralinguistic Perception, Dialogue Behavior Alignment, On-Policy Self-Distillation, Speech Safety and Empathy

TL;DR

ParaBridge introduces an on-policy self-distillation framework that transfers dense, full-vocabulary next-token probability targets from an inference-time paralinguistic prompt scaffold into a scaffold-free student policy, bridging the perception-behavior gap in speech language models without requiring curated dialogues, human labels, or external reward models.

Background & Motivation

Human speech conveys substantially richer communicative intent than text alone: the identical question voiced by a child versus an adult, spoken with tremor and fear versus calm authority, or uttered against a silent backdrop versus audible household chaos demands qualitatively different conversational behaviors from an intelligent spoken-dialogue assistant. Modern end-to-end Speech Language Models (SLMs, such as Qwen3-Omni and MiMo-Audio) demonstrate commendable acoustic perceptual competence on closed-ended benchmarks such as MMSU, successfully discriminating speaker traits, affective states, and background audio events. Nevertheless, when these non-lexical signals are embedded within natural, open-ended spoken interactions, state-of-the-art models systematically fall back to pure textual semantics, ignoring critical acoustic context. For instance, on the child-voice split of VoxSafeBench, when a child inquires how to operate a dangerous kitchen knife, an unprompted Qwen3-Omni-thinking model achieves a dismal 6.1% Safety-Awareness Rate (SAR), offering step-by-step DIY instructions rather than age-appropriate safeguards. This profound disconnect between perceiving acoustic cues and acting upon them defines the paralinguistic perceptionโ€“behavior gap.

An immediate empirical observation is that prepending a simple paralinguistic instruction scaffoldโ€”such as prompting the model to "attend to paralinguistic cues and background sounds and respond appropriately"โ€”substantially elevates safety compliance and empathetic responsiveness at inference time. This confirms that paralinguistic features are already preserved within the model's latent representations. However, relying on inference-time prompting is fundamentally fragile in practical deployment: prompt influence fades rapidly as multi-turn dialogue context lengthens, models must juggle conflicting instructions regarding persona, tool execution, and formatting, and audio-side prompt injection attacks can easily override system prompts. Prior post-training remedies are similarly constrained: supervised fine-tuning (SFT) on curated dialogues requires labor-intensive annotation and risks degrading general reasoning capabilities; rejection sampling fine-tuning (RFT) collapses rich multimodal policies into single deterministic sequences; and reinforcement learning (e.g., GRPO) relies on sparse scalar rewards from external judges, incurring severe sampling overhead and slow convergence.

The crucial insight is that since a prompted model already serves as an effective generator of paralinguistic dialogue behaviors, it can act as a privileged self-teacher directly on the student model's own exploratory rollout trajectories. Core idea: ParaBridge introduces an on-policy self-distillation framework where a single speech language model evaluates its own scaffold-free student rollouts under a privileged scaffolded teacher context, optimizing a token-level symmetric Jensen-Shannon divergence to stably internalize cue-conditioned dialogue behaviors without human curation, offline filtering, or external reward models.

Method

Overall Architecture

The foundational design of ParaBridge rests on "two views of one model." Given an input audio query \(x\), the shared model parameters \(\theta\) are interrogated under two distinct contextual conditions: a scaffold-free student context \(c_\emptyset\) and a privileged scaffolded teacher context \(c_\text{scaff}\) that includes an explicit paralinguistic reminder. During training, the student model samples an on-policy conversational trajectory \(y \sim \pi_\theta(\cdot \mid c_\emptyset)\). Along this exact token sequence, the teacher view computes dense full-vocabulary probability targets with stopped gradients. A per-token symmetric Jensen-Shannon divergence (JSD) loss penalizes discrepancy between the student's distribution and the teacher's target, guiding the student to adapt its output distribution directly conditioned on acoustic cues. At inference time, the scaffold is completely discarded, yielding a robust, scaffold-free dialogue assistant.

The end-to-end ParaBridge training and deployment flow is illustrated below:

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Audio Query & Context Setup<br/>Speech x + Scaffold-free c_โˆ… vs Scaffolded c_scaff"] --> B["On-Policy Student Rollout<br/>Student samples y ~ ฯ€_ฮธ(ยท|c_โˆ…)"]
    B --> C["Privileged Teacher Forwarding<br/>Teacher evaluates y_&lt;t to get q_t with stop-gradient"]
    C --> D["Per-Token Symmetric JSD Alignment<br/>Minimize JSD between student p_t and teacher q_t"]
    D --> E["Scaffold-Free Inference<br/>Deploy student directly without test-time prompts"]

Key Designs

1. Dual-View On-Policy Self-Distillation Mechanism: Eliminating Exposure Bias and Offline Sequence Collapsing

Standard knowledge distillation supervises a student on teacher-generated outputs, introducing severe exposure bias when the student deviates from teacher trajectories at inference. Similarly, rejection sampling fine-tuning (RFT) filters for high-scoring scaffolded responses and treats them as rigid target sequences, collapsing the diverse probability landscape of the model into single modes. ParaBridge circumvents both issues by having the model serve as its own dense teacher on self-generated trajectories. For each training audio input, the student rolls out a complete response under the unprompted context: \(y = (y_1, \dots, y_T) \sim \pi_\theta(\cdot \mid c_\emptyset)\). Both views then evaluate next-token predictions along the identical prefix \(y_{<t}\): $\(p_t = \pi_\theta(\cdot \mid c_\emptyset, y_{<t}), \quad q_t = \text{sg}[\pi_\theta(\cdot \mid c_\text{scaff}, y_{<t})]\)$ where \(\text{sg}[\cdot]\) denotes the stop-gradient operator. Because supervision is evaluated strictly over the student's accessible state distribution, the training signals remain entirely on-policy. This eliminates distribution shift while capturing the full richness of the model's vocabulary distribution without requiring external reward judges.

2. Token-Level Symmetric Jensen-Shannon Divergence: Balancing Mode Covering and Mode Focusing

Transferring scaffold-induced behaviors requires a divergence objective that provides smooth and balanced optimization gradients. Asymmetric forward KL divergence \(\text{KL}(p_t \parallel q_t)\) penalizes student exploration in regions where teacher probabilities are low, while reverse KL \(\text{KL}(q_t \parallel p_t)\) promotes mode-seeking behavior that often leads to output mode collapse. ParaBridge establishes a per-token symmetric Jensen-Shannon divergence (JSD) loss. Defining the mixture distribution \(m_t = \frac{1}{2}(p_t + q_t)\), the token divergence at step \(t\) is: $\(\text{JSD}(p_t \parallel q_t) = \frac{1}{2} \text{KL}(p_t \parallel m_t) + \frac{1}{2} \text{KL}(q_t \parallel m_t)\)$ The overall optimization objective minimizes the expected token-averaged JSD across training queries and sampled trajectories: $\(\mathcal{L}_\text{ParaBridge}(\theta) = \mathbb{E}_{a \sim \mathcal{D}, y \sim \pi_\theta(\cdot \mid c_\emptyset)} \left[ \frac{1}{T} \sum_{t=1}^T \text{JSD}(p_t \parallel q_t) \right]\)$ Full-vocabulary supervision provides dense gradient feedback for every single token. Unlike reinforcement learning formulations (such as GRPO) that compress an entire response into a single scalar reward, JSD informs the model not only of preferred tokens but also of tokens that are linguistically plausible yet contextually inappropriate given the non-lexical audio cues.

3. Acoustic-Driven Read-Out Adaptation: Preserving General Multimodal Capabilities

A major hazard in post-training speech foundation models is catastrophic forgetting of general audio comprehension and text reasoning capabilities. In ParaBridge, LoRA adapters are attached across all linear projections in all 48 MoE layers of the backbone. Layer-wise Centered Kernel Alignment (CKA) between the baseline and ParaBridge reveals that representation representations remain essentially unchanged from L0 through L46 (\(\text{CKA} > 0.998\)). Significant representational drift is isolated strictly to the final two MoE layers immediately preceding the language model head: L47 (\(\text{CKA} = 0.966\)) and L48 (\(\text{CKA} = 0.907\)). Causal activation patching further confirms that patching mid-stack layers results in negligible next-token distribution changes (\(< 10^{-3}\) nats), whereas patching the L47 read-out layer recovers the behavioral shift. Consequently, ParaBridge preserves the pre-trained feature extractor intact and selectively reprograms the read-out mapping from acoustic representations to output vocabulary logits.

Loss & Training

  • Distillation Hyperparameters: The on-policy rollout fraction is set to \(\lambda = 1.0\), mixture weight \(\beta = 0.5\) (yielding standard symmetric JSD), and temperature \(\tau = 1.2\) for both student rollout generation and logit distillation. Auxiliary SFT loss weight is set to 0.
  • LoRA Configuration & Parameter Freezing: Pre-trained audio and vision encoders are frozen. LoRA is applied to all linear projections across the LLM backbone with rank \(r = 64\), scaling factor \(\alpha = 128\), and dropout 0.05.
  • Training Setup & Efficiency: Experiments run on a single node equipped with 8 NVIDIA H20 (96 GB) GPUs: 1 GPU hosts the vLLM rollout server while 7 GPUs run the training loop using DeepSpeed ZeRO-3 and FlashAttention. Optimization reaches peak performance in only 15 epochs across 1,000 audio queries within \(\sim 2.7\) wall-clock hours, delivering a 5.7x training speedup over GRPO (\(\sim 15.3\) hours).

Key Experimental Results

Main Results

Evaluations span six representative benchmarks: VoxSafeBench (evaluating paralinguistic safety awareness across 6 Tier-2 tasks via SAR %), EchoMind (evaluating empathetic dialogue across 4 dimensions via 5-point GPT ratings), MMSU (closed-ended audio understanding across linguistic and paralinguistic splits), MMAU-Pro (comprehensive general audio benchmark), VoiceBench (spoken assistant instruction following), and GPQA (expert-level scientific text reasoning). Comparisons include the scaffold-free Baseline, RFT, GRPO, and ParaBridge.

Benchmark & Dimension Metric Baseline (Scaffold-free) RFT (Scaffold-free) GRPO (Scaffold-free) ParaBridge (Ours, Scaffold-free) Gain over Baseline (ฮ”free) Baseline (Scaffolded)
VoxSafeBench - Child presence (Child_presence) SAR (%) โ†‘ 0.00 50.68 0.00 58.56 +58.56 45.83
VoxSafeBench - Child voice (Child_voice) SAR (%) โ†‘ 6.11 14.90 6.58 18.43 +12.32 17.13
VoxSafeBench - Affective conflict (Emotion) SAR (%) โ†‘ 0.00 11.29 0.00 55.56 +55.56 29.41
VoxSafeBench - Impaired capacity (Impaired_capacity) SAR (%) โ†‘ 7.99 16.20 4.84 14.93 +6.94 13.19
VoxSafeBench - Macro Average (Average) SAR (%) โ†‘ 14.64 31.64 15.67 40.33 +25.69 29.02
EchoMind - Contextual Fit (CCtxFit) 1-5 Rating โ†‘ 4.01 4.34 4.03 4.39 +0.38 4.86
EchoMind - Response Naturalness (CRespNat) 1-5 Rating โ†‘ 3.18 3.83 3.20 3.89 +0.71 4.66
EchoMind - Colloquial Degree (CColloqDeg) 1-5 Rating โ†‘ 3.23 3.84 3.23 3.92 +0.69 4.24
EchoMind - Speech Info Relevance (CSpeechRel) 1-5 Rating โ†‘ 2.64 3.21 2.66 3.46 +0.82 3.48
EchoMind - Macro Average (Average) 1-5 Rating โ†‘ 3.27 3.80 3.28 3.92 +0.65 4.31
MMSU - Perception / Paralinguistic Acc. (%) โ†‘ 53.07 50.15 52.44 53.04 -0.03 โ€“
MMSU - Reasoning / Paralinguistic Acc. (%) โ†‘ 52.84 51.04 52.99 53.84 +1.00 โ€“
MMAU-Pro - Closed-ended Audio Comprehension Acc. (%) โ†‘ 63.18 62.10 63.81 62.96 -0.22 โ€“
VoiceBench - Spoken Assistant Benchmark Avg. (%) โ†‘ 68.98 68.31 69.90 68.63 -0.35 โ€“
GPQA - Text Scientific Reasoning Acc. (%) โ†‘ 71.34 68.45 71.43 71.43 +0.09 โ€“

Ablation Study

Systematic ablations investigate the divergence objective form, teacher input modality, counterfactual false-alarm rates on benign inputs, and multi-turn conversational robustness.

Ablation Category Variant / Configuration VoxSafeBench (SAR % โ†‘) EchoMind (Avg. Rating โ†‘) MMSU (Acc. % โ†‘) False-Alarm Rate (Avg. % โ†“) Turn-3 SAR in Multi-turn (% โ†‘) Core Insight & Findings
(A) Objective & Modality ParaBridge (JSD, Audio Teacher) 40.33 3.92 72.47 โ€“ โ€“ Symmetric JSD over audio features achieves superior overall alignment
Forward KL (\(\text{KL}(p \parallel q)\)) 39.23 3.90 70.53 โ€“ โ€“ Over-penalizing student tail exploration leads to slight degradation
Reverse KL (\(\text{KL}(q \parallel p)\)) 39.55 3.90 70.87 โ€“ โ€“ Mode-seeking helps safety slightly but trails JSD across dimensions
Text Teacher (Transcripts + Metadata) 29.19 3.54 65.09 โ€“ โ€“ Substituting acoustic inputs with text descriptions causes severe drops (-11.14%)
(B) Counterfactual Controls Baseline (Scaffolded) โ€“ โ€“ โ€“ 6.05 โ€“ System prompts introduce notable false-alarm bias on safe speech
RFT (Scaffold-free) โ€“ โ€“ โ€“ 6.42 โ€“ Fitting discrete sequences triggers spurious refusals from lexical keywords
ParaBridge (Scaffold-free) โ€“ โ€“ โ€“ 3.36 โ€“ Lowest false-alarm rate confirms model responds to acoustics, not keyword shortcuts
(C) Multi-Turn Robustness Baseline (Scaffolded) โ€“ โ€“ โ€“ โ€“ 4.09 Prompt attention dilutes across multi-turn dialogue context
RFT (Scaffold-free) โ€“ โ€“ โ€“ โ€“ 4.95 Fragile single-turn policy deteriorates under multi-turn history
ParaBridge (Scaffold-free) โ€“ โ€“ โ€“ โ€“ 7.89 Internalized cue conditioning demonstrates superior resilience over 3 conversation turns

Key Findings

  • Internalization Surpasses Inference-Time Prompting: In a completely scaffold-free setting, ParaBridge achieves 40.33% SAR on VoxSafeBench, substantially outperforming the unprompted Baseline (14.64%) and surpassing even the prompted Baseline (29.02%). This demonstrates that self-distillation successfully internalizes fragile contextual prompting into a permanent conversational capability.
  • Continuous Acoustic Distillation vs. Discrete Lexical Metadata: Ablation panel A demonstrates that replacing the audio self-teacher with a text teacher fed speech transcripts and paralinguistic text descriptions causes SAR to collapse to 29.19%. This proves that bridging the gap requires direct gradient alignment with continuous acoustic representations rather than reliance on high-level textual approximations.
  • Rapid Data Saturation and Data Efficiency: Training scaling curves show that ParaBridge achieves 37.59% SAR with merely 500 audio samples. Scaling up to 1,000 samples yields 40.33%, while doubling again to 2,000 samples offers a marginal increase of only 1.35 points. This early saturation provides strong evidence that the underlying model already encodes sufficient paralinguistic knowledge, requiring minimal on-policy supervision to activate the corresponding response behavior.
  • Out-of-Distribution and Cross-Task Generalization: When trained exclusively on safety queries (child voice and child presence), ParaBridge generalizes robustly to held-out VoxSafeBench axes (+26.15% on Emotion, +8.48% on Symbolic Background) and transfers seamlessly to EchoMind empathetic dialogue (+0.82 on CSpeechRel), while demonstrating reliable adaptation on the MiMo-Audio architecture.

Highlights & Insights

  • On-Policy Self-Teacher Resolves the Reward Density Bottleneck: By leveraging the asymmetric capabilities of the same model under different prompt contexts, ParaBridge provides dense token-level distribution targets along student-generated trajectories. This completely circumvents the need for expensive manual conversation annotation and accelerates optimization by 5.7x compared to RL with scalar judges.
  • Mechanistic Separation of Perception Stack and Read-Out Policy: Rigorous layer-wise CKA and causal activation patching show that representations from L0 to L46 are virtually unaltered (\(\text{CKA} > 0.998\)), with behavioral adaptation localized almost entirely to the final read-out layers (L47-L48). This elegant localization explains why paralinguistic alignment can be achieved without corrupting general auditory or linguistic reasoning.
  • Extensible Privileged Context Self-Distillation Paradigm: The methodology generalizes beyond speech: whenever an advanced behavior can be reliably stimulated via privileged context (e.g., visual bounding-box prompts, thinking chains, or external metadata), it can be effectively internalized into an unprivileged model via on-policy JSD self-distillation.

Limitations & Future Work

  • Dependence on Model-Intrinsic Gap: The method requires the foundation model to exhibit a pronounced scaffoldedโ€“scaffold-free performance gap. For backbones with deficient acoustic perception where even explicit scaffolds fail to elicit appropriate responses, self-distillation cannot manufacture capability from scratch.
  • Scope of Acoustic and Linguistic Axes: Training currently covers child voice and child presence across bilingual English-Chinese settings. Broader paralinguistic phenomenaโ€”such as sarcasm, drunkenness, extreme distress, and diverse regional dialectsโ€”remain to be investigated.
  • Residual False-Alarm Tendency: In challenging acoustic environments, benign adult queries can occasionally be misclassified as child speech, resulting in unintended cautionary remarks or refusals (though false-alarm rates remain lower than all baselines).
  • vs RFT (Rejection Sampling Fine-Tuning): RFT relies on offline supervised fine-tuning over filtered positive rollouts, reducing multi-modal distributions to single deterministic sequences and suffering from exposure bias. ParaBridge conducts on-policy soft distribution matching, ensuring superior multi-turn resilience and generalization.
  • vs GRPO (Group Relative Policy Optimization): GRPO requires an external reward judge to score rollout groups with a sparse scalar reward, leading to slow training convergence and instability. ParaBridge utilizes dense, full-vocabulary token-level supervision from the model itself, converging in 2.7 hours on H20 GPUs without external judges.
  • vs X-OPD (Cross-Modal On-Policy Distillation): X-OPD uses text LLMs as privileged teachers to guide speech models across modalities; ParaBridge is an intra-modal self-distillation framework that directly resolves the paralinguistic perceptionโ€“behavior gap within speech models.

Rating

  • Novelty: โญโญโญโญโญ Formulates the paralinguistic perceptionโ€“behavior gap in speech language models and introduces an elegant, closed-loop on-policy self-distillation framework.
  • Experimental Thoroughness: โญโญโญโญโญ Rigorous validation across six diverse benchmarks, comprehensive ablations, sample-efficiency analysis, and insightful layer-wise CKA/activation patching mechanics.
  • Writing Quality: โญโญโญโญโญ Well-structured narrative, technically rigorous descriptions, and clean diagrams and empirical tables.
  • Value: โญโญโญโญโญ Establishes a highly efficient, practical blueprint for aligning spoken conversational agents without risking general capability degradation.