Skip to content

AP-GRPO: Anchor-Gated Phonetic Alignment with Policy Optimization for Pathological Speech Reconstruction

Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: Pathological Speech Reconstruction, Speech Language Models, Policy Optimization, Anchor-Gated Reward, Soft-DTW

TL;DR

Addressing severe acoustic distortion and linguistic fragmentation in speech affected by neuromotor and neurodegenerative disorders, AP-GRPO introduces an anchor-gated phonetic reinforcement learning framework that anchors reliable words and aligns corrupted inter-anchor spans against the patient's own speech signals via Soft-DTW, slashing word error rates on severe dysarthria and suppressing hallucinated insertions without requiring ground-truth reference transcripts.

Background & Motivation

Neurodegenerative and neuromotor disorders such as amyotrophic lateral sclerosis (ALS), cerebral palsy (CP), Parkinson's disease, and dementia severely compromise speech motor control and articulation, impairing patients' ability to communicate. Unlike conventional Automatic Speech Recognition (ASR) designed for clear or noisy speech, Pathological Speech Reconstruction (PSR) must recover a patient's intended textual message from severely degraded, acoustically distorted, and linguistically fragmented speech recordings. In clinical scenarios with severe impairment, wide acoustic spans are distorted beyond recognition, yet the underlying communicative intent remains partially salvageable from residual acoustic evidence.

Crucially, speech degradation in dysarthria is rarely uniform across an utterance. Even in a heavily distorted recording, specific words or short phrases remain intelligible and survive as reliable "audible anchors", whereas acoustic degradation and reconstruction errors concentrate heavily within the corrupted inter-anchor spans. Contemporary speech language modeling (SLM) paradigms fail to exploit this non-uniform structural reality: standard supervised fine-tuning (SFT) applies next-token prediction uniformly across all positions, frequently hallucinating unconstrained words over long degraded intervals; conversely, standard reinforcement learning with text-based rewards (such as BERTScore in vanilla GRPO) rewards fluent paraphrasing rather than faithful reconstruction, rewarding substitutions like "ache" for "pain" even when the patient unmistakably attempted the latter. Furthermore, collecting extensive paired transcripts from progressive patients is practically intractable.

The pivotal insight of this work is to leverage the patient's own original acoustic signal as a reference-free self-supervision source within an anchor-constrained reinforcement learning framework. By explicitly separating preserved acoustic regions from corrupted regions, optimization can be focused where it is needed most. Core idea: decompose non-uniformly degraded pathological speech into structural boundary anchors and corrupted inter-anchor spans, introducing Anchor-Gated Phonetic Group Relative Policy Optimization (AP-GRPO) to enforce audible anchor retention via a hard-gated log reward and align inter-anchor hypotheses against patient phoneme posteriorgrams via Soft-DTW enriched with pathological duration and confusion priors.

Method

Overall Architecture

The AP-GRPO pipeline operates through offline pre-extraction of audible anchors, followed by policy sampling, monotonic anchor matching, anchor-gated reward computation, pathologically-styled inter-anchor phonetic alignment, and constrained dual policy optimization. During preprocessing, timestamped high-confidence words are extracted from the raw audio using Whisper large-v3 and WhisperX forced alignment. During reinforcement learning, the speech language model policy \(\pi_\theta(\cdot|x)\) generates a group of candidate reconstructions. For each candidate transcript, monotonic matching maps the extracted anchors to the text, demarcating inter-anchor text spans and corresponding audio segments. The model evaluates candidate quality via an anchor coverage gate and an inter-anchor phonetic reward computed by applying Soft-DTW between the candidate's pathologically duration-expanded phoneme path and the confusion-smoothed phoneme posteriorgram (PPG) derived from a frozen wav2vec2 encoder. The policy parameters are updated via group relative advantages balanced by an adaptive dual multiplier.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400, 'subGraphTitleMargin': {'top': 8, 'bottom': 16}}}}%%
flowchart TD
    A["Pathological Speech Audio x"] --> B["Audible Anchor Extraction & Monotonic Matching<br/>WhisperX Forced Alignment Filtering c(m) > ฯ„"]
    A --> C["Frozen wav2vec2 Phoneme CTC Model<br/>Extract Inter-Anchor Phoneme Posteriorgrams (PPG)"]
    B --> D["SLM Policy Generates Candidate Group<br/>{y_1, ..., y_G} ~ ฯ€_ฮธ(ยท|x)"]
    D --> E["Anchor-Gated Coverage Reward<br/>log g_anchor Penalizes Lost Boundary Anchors"]
    D --> F["Pathological-Style Phoneme Path Generation<br/>G2P Conversion + TTDS Dysarthric Duration Expansion"]
    C --> G["Dysarthric Phoneme Confusion Smoothing<br/>Articulatory Group Weight Matrix C(q, k)"]
    F --> H["Soft-DTW Inter-Anchor Phonetic Alignment Reward<br/>Differentiable Dynamic Time Warping Frame-to-Phoneme Cost"]
    G --> H
    E --> I["Adaptive Momentum Dual Policy Update<br/>A_j = Rฬ‚_norm,j + ฮผ_t ฤ_norm,j Advantage Optimization"]
    H --> I
    I --> J["Reconstructed Intended Text y*"]

Key Designs

1. Audible Anchor Extraction and Monotonic Matching: Constructing Discrete Temporal Structural Scaffolding

Pathological speech degradation leaves intact islands of intelligible speech that provide immutable structural anchors across the timeline. To extract these reliable anchors, AP-GRPO employs a two-pass pipeline: Whisper large-v3 generates candidate transcript words along with confidence estimates, retaining words with confidence \(c(m) > \tau\) (calibrated to \(\tau = 0.85\)) and word length \(\ge 3\) characters; WhisperX then performs word-level forced alignment to extract tight start and end timestamps \([s(m), e(m)]\). The final anchor confidence is conservatively assigned as \(c(m) = \min(c_m^{\text{Whisper}}, c_m^{\text{WhisperX}})\). When candidate hypotheses \(y_j\) are sampled from the policy, anchors are aligned via normalized monotonic matching: scanning anchors from left to right, each anchor \(a(m)\) is greedily matched to the earliest compatible, unmatched candidate token span following the previous match position whose normalized string similarity exceeds threshold \(\rho\). This monotonic enforcement guarantees strictly ordered temporal correspondence and partitions each candidate into matched anchors and intermediate corrupted spans \(y_j^{(n)}\) aligned to audio spans \(x^{(n)}\).

2. Anchor-Gated Coverage Reward: Imposing Hard Structural Boundaries Against Hallucination

Without structural constraints, generative language models tend to exploit language priors to output smooth, plausible sentences that entirely discard difficult audible words. To ensure trusted recovery of audible evidence, AP-GRPO introduces an anchor gate based on confidence-weighted coverage:

\[\text{Match}_{\mathcal{A}}(y_j) = \frac{\sum_{m=1}^M c(m) \cdot \mathbf{1}[a(m) \text{ matched in } y_j]}{\sum_{m=1}^M c(m)}\]

A bounded gate score is established as \(g_{\text{anchor}}(y_j, \mathcal{A}) = \epsilon_g + (1 - \epsilon_g) \text{Match}_{\mathcal{A}}(y_j)\) with \(\epsilon_g = 0.05\). In the reinforcement learning reward formulation, this term enters in log-space: \(\log g_{\text{anchor}}(y_j, \mathcal{A})\). The derivative \(\frac{\partial \log g}{\partial g} = \frac{1}{g}\) provides an aggressive, asymptotically steep gradient when anchor coverage is low, heavily penalizing candidates that drop audible words. Furthermore, because the gradient remains strictly positive across all \(g \in [\epsilon_g, 1]\), the anchor gate maintains continuous structural guidance even when coverage constraints are satisfied, preventing the policy from discarding anchors in favor of local phonetic optimization.

3. Dysarthric Confusion Smoothing and TTDS Pathological Duration Expansion: Modeling Clinical Articulatory Priors

Recovering corrupted text between anchors requires handling systematic articulatory deficits characteristic of motor speech disorders, such as blurred voicing contrasts (/t/ vs. /d/), fricative shifts (/s/ toward /sh/), and vowel centralization (/i/ toward /ih/). Applying canonical phoneme probabilities directly severely penalizes patients for clinically consistent productions. AP-GRPO constructs a row-normalized phoneme confusion matrix \(C \in \mathbb{R}^{K \times K}\) based on articulatory phonetic groups (setting diagonal values to \(1.0\), within-group confusable pairs to \(0.4\), and unrelated pairs to \(0.1\)) to smooth the raw PPG \(P^{(n)}(t, k)\) from a frozen wav2vec2 model:

\[\widetilde{P}^{(n)}(t, q) = \sum_{k=1}^K C(q, k) P^{(n)}(t, k)\]

Simultaneously, dysarthric speech exhibits bradykinesia, prolonged phones, and irregular pauses. To align candidate text to this stretched acoustic timeline, the candidate text span \(y_j^{(n)}\) is converted via G2P into phonemes and expanded using predicted phoneme durations from a disease-specific Text-to-Dysarthric-Speech (TTDS) duration model: each phoneme is repeated \(d_{j, \ell}^{(n)}\) times to yield an expanded path \(\tilde{q}_j^{(n)}\). This temporal expansion does not force rigid frame alignment; rather, it places candidate phonemes in the appropriate temporal order and approximate duration frame, enabling dynamic time warping to function within a discriminative regime.

4. Soft-DTW Inter-Anchor Phonetic Alignment: Differentiable Continuous Acoustic Grounding

To evaluate whether candidate text spans are phonetically substantiated by the corresponding audio spans without ground-truth text, a frame-to-phoneme cost matrix is constructed:

\[D_j^{(n)}(t, \ell) = -\log \left(\widetilde{P}^{(n)}(t, \tilde{q}_{j, \ell}^{(n)}) + \epsilon\right)\]

Standard Dynamic Time Warping (DTW) computes a non-differentiable hard minimum over discrete warping paths, which is noisy and unsuited for optimization. AP-GRPO utilizes Soft-DTW with smoothing parameter \(\gamma = 0.1\), which computes a differentiable soft-minimum over all valid monotonic alignment paths. The resulting score is normalized by the combined length of speech frames and expanded phonemes \(T_n + L_j^{(n)}\) to remove length bias:

\[R_{\text{span}, j}^{(n)} = -\frac{\text{SoftDTW}_\gamma(D_j^{(n)})}{T_n + L_j^{(n)}}\]

Averaging over all inter-anchor spans yields the global phonetic reward \(R_{\text{inter}}(y_j, x, \mathcal{A}) = \frac{1}{|I_j|} \sum_{n \in I_j} R_{\text{span}, j}^{(n)}\). Because hallucinated insertions create phoneme paths unsupported by acoustic energy in the PPG, Soft-DTW incurs heavy cumulative costs, penalizing hallucinated content words effectively.

Loss & Training

AP-GRPO formulates pathological speech reconstruction as a constrained policy optimization problem where the policy maximizes the log-space reward subject to an anchor coverage constraint \(\mathbb{E}_{\pi_\theta}[g_{\text{anchor}}] \ge \alpha\) (with \(\alpha = 0.95\)) and a KL divergence constraint \(\mathbb{E}_x[\text{KL}(\pi_\theta \parallel \pi_{\text{ref}})] \le \delta\):

\[\max_\theta \mathbb{E}_{\pi_\theta} \left[\log g_{\text{anchor}}(y, \mathcal{A}) + R_{\text{inter}}(y, x, \mathcal{A})\right], \quad \text{s.t.} \quad \mathbb{E}_{\pi_\theta}[g_{\text{anchor}}] \ge \alpha\]

Because the acoustic phonetic score \(R_j\) and the anchor constraint signal \(g_j\) exhibit different variance profiles, AP-GRPO applies decoupled within-group normalization across \(G = 32\) sampled candidates:

\[A_j = \frac{R_j - \bar{R}}{\hat{\sigma}_R + \epsilon} + \mu_t \frac{g_j - \bar{g}}{\hat{\sigma}_g + \epsilon}\]

The Lagrange multiplier \(\mu_t \ge 0\) is dynamically updated via momentum-accelerated projected subgradient descent:

\[\tilde{v}_{t+1} = \gamma_m \tilde{v}_t + (1 - \gamma_m)(\alpha - \hat{g}_t), \quad \beta_t^{\text{eff}} = \beta_\mu (1 + c [\tilde{v}_{t+1}]_+), \quad \mu_{t+1} = \left[\mu_t + \beta_t^{\text{eff}} (\alpha - \hat{g}_t)\right]_+\]

When constraint violations persist (\(\hat{g}_t < \alpha\)), the accumulated violation \(\tilde{v}_{t+1} > 0\) temporarily boosts the effective step size, increasing \(\mu_t\) to enforce anchor preservation. Once anchor coverage is restored, \(\mu_t\) decays, allowing the phonetic alignment reward to guide inter-anchor text generation. The framework adopts a linear weak-to-strong KL schedule \(\eta_t = \eta_0 + (\eta_T - \eta_0)t/T\) to ensure stable policy convergence within 40 optimization steps.

Key Experimental Results

Main Results

The framework was evaluated across four clinical pathological speech benchmarks covering distinct motor-speech and neurodegenerative conditions: TORGO (ALS, severe dysarthria), UASpeech (Cerebral Palsy, atypical articulation), ADReSSo (Alzheimer's/Dementia, linguistic fragmentation), and SJTU-PD (Parkinson's disease, hypokinetic dysarthria). Experiments were conducted using Qwen2.5-Omni-7B and Audio-Flamingo 3 as speech-language backbones:

Dataset (Condition) Method WER โ†“ CER โ†“ BLEU-4 โ†‘ CONTENT F1 โ†‘
TORGO (ALS) Align-SLM 0.8101 0.7894 0.3133 0.4967
Diff-DSR 1.0191 0.7442 0.1138 0.3075
Colm-DSR 0.9533 0.7698 0.1087 0.3168
Gemini-Flash-3.5 0.4919 0.3022 0.2324 0.4558
Audio-Flamingo 3 + SFT 0.5699 0.5461 0.3577 0.5709
Audio-Flamingo 3 + AP-GRPO 0.3577 0.2345 0.4409 0.6364
Qwen2.5-Omni + SFT 0.7495 0.7566 0.3599 0.5725
Qwen2.5-Omni + AP-GRPO 0.2885 0.1764 0.4573 0.6597
UASpeech (Cerebral Palsy) Align-SLM 0.6299 0.5574 0.3155 0.4231
Audio-Flamingo 3 + SFT 0.3755 0.2960 0.5286 0.6205
Audio-Flamingo 3 + AP-GRPO 0.2862 0.2086 0.6418 0.7254
Qwen2.5-Omni + SFT 0.4511 0.4660 0.6224 0.7214
Qwen2.5-Omni + AP-GRPO 0.3696 0.3462 0.6249 0.7225
ADReSSo (Dementia) Qwen2.5-Omni + SFT 0.3090 0.1350 0.5519 0.8760
Audio-Flamingo 3 + AP-GRPO 0.2075 0.1141 0.6589 0.9083
Qwen2.5-Omni + AP-GRPO 0.2234 0.1199 0.6472 0.8983
SJTU-PD (Parkinson's) Qwen2.5-Omni + SFT 0.1724 0.0799 0.6856 0.9087
Qwen2.5-Omni + AP-GRPO 0.1592 0.0701 0.6992 0.9141

Ablation Study

Ablation experiments conducted on Qwen2.5-Omni on the TORGO dataset delineate the necessity of each reward component and constraint mechanism:

Configuration \(\log g_{\text{anchor}}\) \(R_{\text{inter}}\) Confusion (Cons) Pathological Style (PaS) Multiplier \(\mu\) WER โ†“ CER โ†“ BLEU-4 โ†‘ CONTENT F1 โ†‘
SFT-Only Baseline โ€” โ€” โ€” โ€” โ€” 0.7495 0.7566 0.3599 0.5725
GRPO (BERTScore Reward) ร— ร— โ€” โ€” โ€” 0.7457 0.7540 0.3691 0.5178
Anchor-Gate only โœ“ ร— โ€” โ€” adaptive 0.7466 0.7527 0.3637 0.5742
Phonetic Reward only ร— โœ“ โœ“ โœ“ adaptive 0.6901 0.6425 0.2012 0.2811
Gate + smoothed PPG (no PaS) โœ“ โœ“ โœ“ ร— adaptive 0.7404 0.7494 0.3682 0.5814
Gate + raw PPG + PaS โœ“ โœ“ ร— โœ“ adaptive 0.3725 0.2461 0.4086 0.6235
No constraint (\(\mu = 0\) fixed) โœ“ โœ“ โœ“ โœ“ none 0.4932 0.4435 0.3997 0.5793
Fixed \(\mu = 1.0\) โœ“ โœ“ โœ“ โœ“ fixed 0.4699 0.3077 0.4259 0.5955
Full AP-GRPO (adaptive \(\mu\)) โœ“ โœ“ โœ“ โœ“ adaptive 0.2885 0.1764 0.4573 0.6597

Key Findings

  • Transformation on Severe Dysarthria: On the most challenging TORGO (ALS) benchmark, AP-GRPO slashes WER from ~0.75 down to 0.2885 on Qwen2.5-Omni and from ~0.57 down to 0.3577 on Audio-Flamingo 3. This moves the output across the intelligibility threshold from unintelligible word salad (where over half of the words are incorrect) into partially correct transcripts suitable for downstream clinical error correction.
  • Suppression of Hallucinated Insertions: SFT models frequently hallucinate repetitive or spurious phrases on short inputs (e.g., prompt "bat" \(\to\) "bat bay a at"), producing 77.9 content-word errors per 100 reference words. AP-GRPO drops content edits to 26.1 per 100 words. Because hallucinated phonemes have no supporting acoustic energy in the speech PPG, Soft-DTW heavily penalizes them, acting as an intrinsic deterrent against ungrounded generation.
  • Synergy of Pathological Styling and Confusion Smoothing: Omitting TTDS duration expansion (PaS) degrades WER from 0.2885 to 0.7404 because canonical phoneme durations create a severe temporal mismatch against elongated pathological speech. Omitting confusion smoothing further degrades WER to 0.3725, proving that accounting for clinical phonetic shifts is critical for accurate alignment.
  • Superiority of Adaptive Dual Variable: Fixed multiplier baselines (\(\mu = 1.0\)) achieve a sub-optimal WER of 0.4699. The adaptive momentum-driven dual variable dynamically oscillates to balance anchor preservation and phonetic infilling, adapting to the varying difficulty of each training sample.

Highlights & Insights

  • Exploiting Non-Uniform Pathology: Rather than treating speech degradation as homogeneous noise, AP-GRPO builds upon the clinical observation that pathological recordings contain intact audible anchors interspersed with degraded spans, providing a natural divide-and-conquer strategy for SLMs.
  • Reference-Free Grounding via Soft-DTW: By computing alignment directly against the patient's own phoneme posteriorgrams, AP-GRPO eliminates the reliance on ground-truth text during RL alignment, bypassing the bottleneck of collecting large-scale paired dysarthric data.
  • Dual Multiplier as an Interpretable Clinical Diagnostic: The learned trajectory of the dual variable \(\mu_t\) naturally reflects clinical articulatory severity (Cerebral Palsy \(\mu \approx 1.5 >\) ALS \(\mu \approx 1.0 >\) Parkinson's \(>\) Dementia), providing an unsupervised, interpretable indicator of articulatory impairment.

Limitations & Future Work

  • Transcript-Only Scope: AP-GRPO focuses strictly on faithful textual reconstruction and does not synthesize clean speech audio directly; end-to-end clinical deployment requires pairing it with an expressive, personalized dysarthric TTS system.
  • Dependency on Upstream Anchor Extraction: The model relies on Whisper and WhisperX for initial anchor detection; severe misrecognitions or inaccurate timestamps during preprocessing can bias the anchor gate and mislead phonetic alignment.
  • Ceiling Effects on Mild Impairments: On conditions with relatively preserved articulation such as Parkinson's, where SFT baselines already reach low error rates (WER ~0.17), the margins for improvement are constrained by dataset scale and the acoustic diversity of the patient cohort.
  • vs Align-SLM / SpeechAlign: Existing speech language model alignment methods predominantly adapt text-based preference objectives (e.g., DPO) or high-level semantic feedback, which fail to penalize fluent paraphrases in pathological speech. AP-GRPO enforces fine-grained acoustic grounding at the sub-word phoneme frame level.
  • vs Unit-DSR / CoLM-DSR: Prior dysarthric speech reconstruction systems rely on discrete speech unit normalization or neural codec language models trained on substantial paired corpora. AP-GRPO leverages reinforcement learning to directly elicit faithful reconstruction capabilities from general-purpose speech language models without reference transcripts.

Rating

  • Novelty: โญโญโญโญโญ Formulates pathological speech reconstruction as anchor-gated phonetic alignment under policy optimization, ingeniously integrating Soft-DTW with clinical phonetic priors.
  • Experimental Thoroughness: โญโญโญโญโญ Evaluated across four distinct neurological conditions with exhaustive ablations, error decomposition, and dual variable trajectory analyses.
  • Writing Quality: โญโญโญโญโญ Clear motivation, rigorous theoretical formulation of constrained optimization, and transparent presentation of empirical findings.
  • Value: โญโญโญโญโญ Represents a vital step toward accessible assistive speech communication for patients with severe neurodegenerative disorders.