Unlocking Fine-Grained Translation Quality Estimation in LRMs through Mutually Boosting Implicit and Explicit Reasoning¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/RenfeiDang/RIEQE
Area: Multilingual & Translation
Keywords: translation quality estimation, large reasoning models, implicit reasoning, explicit reasoning, reinforcement learning
TL;DR¶
Addressing the failure of large reasoning models (LRMs) in fine-grained translation quality estimation due to massive search spaces and fluency illusions, this paper proposes RIEQE, a two-stage framework that decomposes quality estimation into subtasks, pre-trains implicit layer-wise representations via NonThinking-SFT, and then enhances explicit token-wise reasoning via Thinking-RLVR, achieving bidirectional mutual reinforcement and new SOTA performance.
Background & Motivation¶
Translation Quality Estimation (QE) aims to assess machine translation quality without ground-truth reference translations. While early efforts primarily targeted coarse sentence-level scalar scores, modern downstream applicationsโsuch as computer-assisted post-editing and mission-critical translation deploymentโurgently demand fine-grained QE, which identifies specific error spans and their severity levels (e.g., MINOR versus MAJOR). Recent Large Reasoning Models (LRMs) built on top of extensive multilingual LLMs demonstrate state-of-the-art performance in sentence-level scoring via explicit chain-of-thought (CoT) reasoning. However, when evaluated on fine-grained token-level or span-level error detection benchmarks (such as the WMT QE Shared Task), LRMs consistently trail behind specialized encoder-based architectures like xCOMET and DCSQE. Empirical analyses reveal that LRMs do not lack cross-lingual competence; rather, they are easily deceived by surface sentence fluency and struggle to faithfully attend to and isolate subtle semantic corruptions across entire sentences.
The core tension stems from the mismatch between the dense discriminative requirements of fine-grained error localization and the vulnerability of long explicit reasoning chains. Direct rollout of long-CoT generation across entire sentences forces the model into an enormous search space, frequently resulting in post-hoc rationalization, hallucinations, and unfaithful reasoning. More critically, applying reinforcement learning with verifiable reward (RLVR) directly from scratch to elicit error localization fails because untrained models struggle to produce valid reasoning trajectories in the first place, leading to immediate policy collapse. Meanwhile, traditional single-turn end-to-end setups compel the model to identify errors across dozens of tokens and assess severity levels simultaneously, overtaxing the hidden layer capacity and incentivizing shortcut reliance rather than genuine discriminative reasoning.
This paper tackles the challenge by reconsidering the synergy between internal layer-wise representations and token-level explicit reasoning. Instead of forcing long reasoning chains from the outset, the framework builds upon the finding that layer-wise implicit computations strongly anchor and guide explicit reasoning trajectories. Core idea: decompose fine-grained translation quality estimation into semantic unit segmentation, error detection, and severity classification, and employ a two-stage training scheme that first strengthens implicit discriminative representations via NonThinking-SFT without reasoning chains, and subsequently bootstraps explicit token-wise reasoning via Thinking-RLVR with verifiable rewards, establishing a bidirectional mutually boosting loop.
Method¶
Overall Architecture¶
RIEQE decomposes the fine-grained translation quality estimation problem into three sequential, modular subtasks to dramatically simplify the per-step reasoning load: Unit Segmentation, Error Detection, and Severity Classification. Model optimization follows a two-stage paradigm: Stage 1 conducts NonThinking-SFT by feeding empty reasoning chains (<think>\n\n</think>) to force the Transformer hidden layers to solve the subtasks purely via internal layer-wise computation, instilling strong implicit discriminative intuition. Stage 2 applies Thinking-RLVR on a modest 1k-sample subset with task-specific verifiable rewards, encouraging the model to spontaneously deploy structured, explicit chain-of-thought reasoning upon its enhanced implicit base.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input: Source Sentence & Target Translation"] --> B["Semantic Unit Segmentation<br/>Partition target into ~5-word coherent units"]
B --> C["Two-Stage Training Synergy<br/>NonThinking-SFT Implicit Compression + Thinking-RLVR Explicit Exploration"]
C --> D["Unit-Level Error Detection<br/>Inspect each unit sequentially to identify error spans"]
D --> E["Severity Classification<br/>Iteratively classify detected spans into MINOR or MAJOR"]
E --> F["Output: Word-level & Span-level Error Boundaries and Severity"]
Key Designs¶
1. Semantic Unit Segmentation: Contextual Dimensionality Reduction to Prevent Fluency Oversights Conventional LLM-based QE methods prompt the model to review an entire translated sentence in a single forward pass, where global structural fluency frequently masks local mistranslations and terminology slips. To overcome this limitation, RIEQE introduces an initial Semantic Unit Segmentation subtask using a pre-trained model to divide the target sentence into ordered, semantically coherent segments averaging roughly 5 words each. In training, instances where an error span inadvertently crosses a segmentation boundary are merged according to ground-truth spans to maintain supervision purity, while during testing the model evaluates units sequentially from left to right. This decomposes an unconstrained whole-sentence search problem into localized, short-span comparisons, effectively preventing the attention mechanism from diluting subtle semantic misalignments across long contexts.
2. Two-Stage Training Synergy: Bypassing Distillation Noise and Mitigating Cold-Start Collapse
Standard reasoning post-training relies heavily on supervised fine-tuning over synthetic long-CoT traces (Thinking-SFT) followed by reinforcement learning. However, for dense discriminative tasks like QE, synthetic reasoning chains produced by teacher models inevitably contain flawed deductions, unfaithful explanations, and hallucinated justifications that compromise student convergence. RIEQE addresses this by coupling NonThinking-SFT with Thinking-RLVR. In the NonThinking-SFT phase, the model is trained with empty reasoning tokens to predict target answers directly, compelling all intermediate Transformer layers to form linearly separable representations of error tokens, as evidenced by marked increases in layer-wise ROC-AUC and normalized Average Precision (nAP). Building upon this reliable implicit foundation, the subsequent Thinking-RLVR stage prompts explicit reasoning by prefilling <think>\nOkay,, allowing the model to explore robust reasoning paths without suffering from early cold-start degeneration. Furthermore, RLVR feedback actively refines deep hidden-layer separability across all layers.
3. Fine-Grained Verifiable Rewards: Direct Policy Optimization Without Reward Model Vulnerabilities
During the Thinking-RLVR stage, RIEQE completely eliminates subjective LLM-as-a-judge scorers by introducing rigorous, task-specific verifiable rewards for both error detection and severity assignment. For Subtask 2 (Error Detection), the reward is defined as the character-level Matthews Correlation Coefficient (MCC) over the target sentence:
$$
\text{Reward}{\text{ErrorDetection}} = \text{MCC}
$$
For error-free sentences, the reward is binary (1 for predicting no errors, 0 otherwise). For Subtask 3 (Severity Classification), the reward measures the exact match ratio of predicted severity categories against ground-truth error spans:
$$
\text{Reward}}{\text{SeverityClassification}} = \frac{E
$$
A punitive reward of -1 is enforced if the format violates JSON requirements, lacks a closing }}}{E_{\text{total}}</think> tag, or generates less than 1000 characters of reasoning. Crucially, to counteract severe label imbalance where error-free tokens dominate, error-free instances are subsampled to 30% in RLVR, effectively precluding the model from collapsing into an overly conservative strategy of predicting zero errors to maximize rewards.
Loss & Training¶
- NonThinking-SFT: Built on Qwen3-4B-Thinking-2507 using LoRA (rank=64) in LlamaFactory; learning rate 1e-4 with cosine scheduling; batch size 64 for 2 epochs. Input prompt specifies
<think>\n\n</think>, forcing direct structured JSON prediction without explicit thoughts. Error-free samples are downsampled to a 1:1 ratio. - Thinking-RLVR: Conducted with GRPO within the verl framework; rollout group size 8, global batch size 128, learning rate 2e-6, max response length 3000 tokens for up to 6 epochs. Prompt is suffixed with
<think>\nOkay,to enforce explicit chain expansion. At inference, RIEQE supports both fast zero-thought implicit forward passes (RIEQE-NonThinking) and 5-pass majority voting explicit decoding (RIEQE).
Key Experimental Results¶
Main Results¶
Evaluated on high-resource WMT 2023 span-level benchmarks (zh-en, en-de), WMT 2022 word-level (en-ru), and low-resource WMT 2023 word-level (en-mr), RIEQE establishes new state-of-the-art results, decisively outperforming specialized encoder architectures (DCSQE, CometKiwi-23, xCOMET-XXL) and proprietary reasoning models (GPT-5.5):
| Language Pair | Model / System | Word MCC | Word F1 | Span F1 | Span Precision | Span Recall |
|---|---|---|---|---|---|---|
| zh-en | GPT-5.5 | 0.2520 | 0.2643 | 0.2062 | 0.1208 | 0.7050 |
| zh-en | CometKiwi-23 | 0.2690 | - | 0.2720 | - | - |
| zh-en | xComet-XXL | 0.2291 | 0.2793 | 0.2196 | 0.1427 | 0.4759 |
| zh-en | DCSQE (Prev. SOTA) | 0.2812 | 0.2861 | 0.2771 | 0.2219 | 0.3688 |
| zh-en | Qwen3-4B-Thinking (Backbone) | 0.2195 | 0.2756 | 0.2240 | 0.1551 | 0.4031 |
| zh-en | Qwen3-4B-Thinking (Subtask) | 0.2343 | 0.2910 | 0.2384 | 0.1746 | 0.3755 |
| zh-en | RIEQE-NonThinking | 0.2903 | 0.3417 | 0.2934 | 0.2419 | 0.3727 |
| zh-en | RIEQE (Full) | 0.3073 | 0.3520 | 0.2917 | 0.2407 | 0.3700 |
| en-de | GPT-5.5 | 0.2507 | 0.2833 | 0.2282 | 0.1425 | 0.5732 |
| en-de | CometKiwi-23 | 0.2150 | - | 0.2350 | - | - |
| en-de | xComet-XXL | 0.2653 | 0.3183 | 0.2423 | 0.1705 | 0.4181 |
| en-de | DCSQE (Prev. SOTA) | 0.2711 | 0.3061 | 0.2589 | 0.2120 | 0.3326 |
| en-de | Qwen3-4B-Thinking (Backbone) | 0.1581 | 0.2173 | 0.1873 | 0.1655 | 0.2157 |
| en-de | Qwen3-4B-Thinking (Subtask) | 0.1733 | 0.2226 | 0.1970 | 0.1905 | 0.2040 |
| en-de | RIEQE-NonThinking | 0.2682 | 0.3107 | 0.2500 | 0.2568 | 0.2435 |
| en-de | RIEQE (Full) | 0.2858 | 0.3263 | 0.2643 | 0.2742 | 0.2550 |
| en-ru | DCSQE | 0.3510 | 0.3733 | - | 0.2984 | 0.4985 |
| en-ru | RIEQE (Full) | 0.3583 | 0.4092 | - | 0.3325 | 0.5318 |
| en-mr | DCSQE | 0.2806 | 0.3423 | - | 0.2517 | 0.5347 |
| en-mr | RIEQE-NonThinking | 0.2868 | 0.3361 | - | 0.4126 | 0.2836 |
| en-mr | RIEQE (Full) | 0.2847 | 0.3331 | - | 0.4223 | 0.2750 |
Ablation Study¶
Ablations systematically assess task decomposition against whole-task learning (Table 3) and compare NonThinking-SFT against conventional Thinking-SFT (Table 4) on WMT 2023 en-de:
| Configuration / Training Setting | Word MCC | Word F1 | Note |
|---|---|---|---|
| WholeTask (A: equal epochs) | 0.2101 | 0.2623 | End-to-end full sentence without decomposition (same epochs as SubTask) |
| WholeTask (B: equal samples) | 0.2320 | 0.2869 | End-to-end full sentence without decomposition (same total training instances) |
| SubTask (Ours: Decomposed) | 0.2674 | 0.3160 | Three sequential subtasks under NonThinking-SFT alone |
| Backbone (Qwen3-4B-Thinking) | 0.1733 | 0.2226 | Raw untrained base model |
| + Direct RLVR (1k samples) | 0.1882 | 0.1890 | RLVR without SFT; policy suffers from cold-start failure; F1 collapses |
| NonThinking-all (10k SFT) | 0.1953 | 0.2316 | SFT without CoT using full answers |
| NonThinking-all + RLVR (1k) | 0.2293 | 0.2755 | Our two-stage recipe; achieves highest score after RLVR |
| Thinking-all (10k SFT) | 0.1918 | 0.2439 | SFT with synthetic CoT from Qwen3-235B (closest overlap sample) |
| Thinking-all + RLVR (1k) | 0.2147 | 0.2382 | Synthetic CoT introduces distillation noise, yielding lower gains post-RLVR |
| Thinking-correct + RLVR (1k) | 0.2085 | 0.2375 | SFT retaining only strictly correct CoT samples; smaller data volume degrades RL |
Key Findings¶
- Bidirectional Co-evolution of Dual Reasoning Modes: Probing experiments using layer-wise MLP classifiers reveal that NonThinking-SFT drastically elevates the linear separability (ROC-AUC and nAP) of error-related representations in hidden states during input processing. Thinking-RLVR further amplifies this internal separability across all layers, demonstrating that explicit reasoning feedback enriches latent implicit representations.
- Superior Precision Suppresses False Positives: In contrast to proprietary LLMs like GPT-5.5 that display extreme over-flagging (Recall 0.7050 vs. Precision 0.1208 on zh-en), RIEQE maintains high precision (0.2407 to 0.4223), surpassing human annotator agreement benchmarks and providing practical, trustworthy error detections.
- Unlocking Dormant Low-Resource Knowledge: On the low-resource en-mr benchmark, the raw backbone achieves an abysmal MCC of 0.0915, but jumps to 0.2868 under RIEQE-NonThinking, outperforming DCSQE (0.2806) trained on hundreds of thousands of parallel sentences.
Highlights & Insights¶
- The NonThinking SFT Advantage over Distilled CoT: Contrary to the common belief that reasoning models must always be fine-tuned on elaborate CoT data, this work demonstrates that for dense discriminative tasks, distilled CoT trajectories inject pervasive reasoning noise. Training without reasoning chains acts as a powerful compressive regularization, forcing the network to internalize robust layer-wise discriminative representations.
- Dual Inference Modes for Computationally Efficient Deployment: RIEQE-NonThinking delivers competitive SOTA accuracy without generating a single thinking token, slashing per-sample latency to 0.22s (compared to 0.77s for explicit reasoning). This makes it an ideal candidate as a high-throughput, low-latency reward model for MT reinforcement learning.
Limitations & Future Work¶
- Hyperparameter and Data Composition Sensitivity: The study does not systematically explore the optimal data ratios across diverse language pairs, segmentation window lengths, or RLVR exploration parameters, leaving potential headroom unexploited.
- Occasional Cross-Unit Error Boundary Splits: While empirical evaluation indicates that only ~3% of error spans cross segmentation boundaries (mostly due to excessively long ground-truth annotations), pathological splits could theoretically diminish contextual clues in complex morphological constructions.
- Scope Limited to Translation Quality Estimation: The framework has been validated strictly on WMT QE tasks; evaluating its generalizability to broader dense verification domains, such as factual consistency verification and automated vulnerability auditing, remains an open avenue.
Related Work & Insights¶
- vs DCSQE / xCOMET: While encoder-based models leverage 500k-scale synthetic data for token-level regression, they face capacity bottlenecks in capturing semantic ambiguity and transferring to low-resource languages. RIEQE activates the broader latent capacity of a 4B LRM with modest data budgets.
- vs TASER / GEMBA-MQM: Previous LLM-based QE methods rely on sentence-level scores or direct zero-shot prompting of commercial APIs, suffering heavily from surface fluency biases. RIEQE offers the first systematic training framework to solve the internal representation deficits of LRMs in fine-grained discriminative tasks.
Rating¶
- Novelty: โญโญโญโญโญ Innovatively introduces the mutual boosting of implicit compression and explicit RLVR, challenging the default presumption of long-CoT distillation.
- Experimental Thoroughness: โญโญโญโญโญ Comprehensive evaluation covering 4 language pairs, span and word levels, layer-wise probes, consistency audits, and latency profiling.
- Writing Quality: โญโญโญโญโญ Clear exposition, tightly connected logical arguments, and insightful diagnostic experiments.
- Value: โญโญโญโญโญ Provides a foundational training methodology for empowering LRMs in complex, low-tolerance discriminative evaluation tasks.