Rethinking Stepwise Model Routing: A Cost-Efficient Table Reasoning Perspective¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: Model Compression
Keywords: table reasoning, stepwise model routing, uncertainty estimation, failure risk mapping, inference efficiency
TL;DR¶
EcoTab presents a table-aware stepwise model routing framework that decouples table structure tokens from natural language text tokens via a word-level Trie, independently maps their uncertainties to next-step failure risks using offline counterfactual suffix replacement, and fuses the risks via Noisy-OR logic to guide small/large model selection without generating extra verification tokens.
Background & Motivation¶
Large Reasoning Models (LRMs) such as DeepSeek-R1 and the OpenAI o-series have demonstrated remarkable performance on complex table reasoning tasks spanning data analytics, fact verification, and scientific reporting. However, their test-time compute scaling relies on generating lengthy chain-of-thought traces, which incurs prohibitive computational overhead and memory footprints. In latency-sensitive or compute-constrained deployment scenarios, running monolithic multi-billion-parameter LRMs across every intermediate reasoning step is computationally unsustainable. To mitigate this efficiency bottleneck, stepwise model routing (e.g., SpecCoT, SpecReason, GlimpRouter, STEER) dynamically decomposes reasoning into distinct steps, delegating routine deduction steps to smaller reasoning models (SRMs) and reserving expensive LRMs only for intricate transitions. On free-form text reasoning tasks such as MATH500 and AIME24, these existing routers reliably achieve near-LRM accuracy while expending only about 60% of full LRM FLOPs.
Nevertheless, existing stepwise routing techniques experience a pronounced efficiency collapse when transferred to structured tabular reasoning. Empirical benchmarks on WikiTQ and TableBench demonstrate that existing routers require nearly 80% of total LRM FLOPs to approach comparable accuracy levels. An extensive manual error analysis across 1,000 failure cases reveals that 82.7% of routing blunders stem from "table-specific steps"—specifically subtable retrieval and numerical aggregation across rows and columns—where existing routers systematically misroute critical steps to the underpowered SRM. Investigating the root cause reveals that tabular reasoning steps comprise two heterogeneous token categories: table tokens grounded in structured schema elements and cell values, and text tokens conveying natural language deductions. While free-text steps exhibit a pronounced average entropy gap of 0.14 between correct and incorrect predictions, this diagnostic gap contracts to a meager 0.06 in table-specific steps. Crucially, failure cases are heavily distributed across asymmetric regimes where either table-token uncertainty or text-token uncertainty spikes in isolation. Conflating both types into a single step-level scalar washes out local vulnerabilities and misleads model selection.
This paper tackles the challenge by moving beyond monolithic step representations and explicitly decoupling the uncertainty dynamics of tabular grounding and linguistic reasoning. Core idea: construct an offline, model-free table prefix tree to split tokens in each step into table and text subsets, fit dual offline sigmoid mappings via counterfactual suffix replacement to calibrate raw uncertainties into next-step failure probabilities, and combine them through Noisy-OR probabilistic fusion to achieve highly accurate, zero-draft-overhead stepwise routing.
Method¶
Overall Architecture¶
The EcoTab framework operates via a four-stage inference pipeline: First, prior to inference, it constructs a lightweight, word-level Table Trie from the input table schema and cell entries to partition any generated step into table tokens and text tokens using longest-prefix matching. Second, during multi-step reasoning, it computes Shannon entropy over the model vocabulary for each token in the current step and separately calculates the mean entropy for the table-token and text-token sets. Third, it passes these two distinct uncertainty scalars through pre-calibrated offline sigmoid mappings—fitted using counterfactual suffix replacement on held-out trajectories—to obtain well-calibrated next-step SRM failure probabilities. Finally, it integrates both failure probabilities via Noisy-OR logic into a unified routing score to trigger either SRM continuation or LRM escalation for generating the subsequent step.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Input Table & Natural Language Query"] --> B["Construct Word-Level Table Trie<br/>Normalize & insert headers and cell values"]
B --> C["Dual-Domain Token Partitioning<br/>Longest-prefix match separating table & text tokens"]
C --> D["Step-Level Dual Uncertainty Estimation<br/>Calculate mean Shannon entropy for each token subset"]
D --> E["Calibrate Offline Failure-Risk Mappings<br/>Sigmoid mapping fitted via counterfactual suffix replacement"]
E --> F["Noisy-OR Probabilistic Risk Fusion<br/>Synthesize unified failure risk & evaluate threshold"]
F -->|Risk > Threshold| G["High Failure Risk: Delegate Next Step to LRM"]
F -->|Risk <= Threshold| H["Low Failure Risk: Continue Next Step with SRM"]
Key Designs¶
1. Construct Word-Level Table Trie and Dual-Domain Token Partitioning: Eliminating the overhead of neural parsing while precisely separating tabular entities from surrounding text
Conventional approaches to token classification either deploy auxiliary neural token classifiers or resort to heuristic regex rules, introducing non-trivial latency or failing on multi-word cell phrases. EcoTab constructs an efficient, model-free word-level Table Trie directly from input table \(T\). Prior to insertion, all column headers and cell values undergo lightweight canonical normalization, including lowercasing, collapsing consecutive whitespace, and standardizing numerals and punctuation marks. Multi-token entries (e.g., "new york" or "gross domestic product") are inserted as contiguous word paths. During generation, each reasoning step \(s_i = (t_{i,1}, t_{i,2}, \dots, t_{i,k_i})\) is normalized under identical rules and scanned from left to right using longest-prefix matching over the Trie. Matched character spans are mapped back to token indices to construct a boolean mask \(m^{(i)} \in \{0, 1\}^{k_i}\). This cleanly partitions \(s_i\) into a table-token set \(V_{\text{tab}}^{(i)} = \{t_{i,j} \mid m_j^{(i)} = 1\}\) and a text-token set \(V_{\text{text}}^{(i)} = \{t_{i,j} \mid m_j^{(i)} = 0\}\) on the CPU with sub-millisecond execution and zero model invocation.
2. Step-Level Dual Uncertainty Estimation: Overcoming the dilution of critical grounding errors caused by whole-step entropy averaging
In multi-step tabular reasoning, a model may express high linguistic confidence in transitional discourse while harboring severe ambiguity regarding which numerical cell to extract. EcoTab computes token-level Shannon entropy \(c_{i,j}\) across vocabulary \(\mathcal{V}\) for each token position \((i, j)\):
$\(c_{i,j} = -\sum_{v \in \mathcal{V}} p_{i,j}(v) \log p_{i,j}(v)\)$
where \(p_{i,j}(v)\) represents the predictive probability of token \(v\). Instead of collapsing all tokens into an undifferentiated average, EcoTab computes two separate step-level uncertainty metrics:
$\(\Phi_{\text{tab}}^{(i)} = \frac{1}{|V_{\text{tab}}^{(i)}|} \sum_{t_{i,j} \in V_{\text{tab}}^{(i)}} c_{i,j}, \quad \Phi_{\text{text}}^{(i)} = \frac{1}{|V_{\text{text}}^{(i)}|} \sum_{t_{i,j} \in V_{\text{text}}^{(i)}} c_{i,j}\)$
If a step contains no table tokens, \(\Phi_{\text{tab}}^{(i)}\) defaults to 0. This dual formulation guarantees that spikes in cell selection ambiguity remain prominent and will not be masked by lengthy, predictable natural language phrasing.
3. Calibrate Offline Failure-Risk Mappings: Harmonizing disparate uncertainty distributions into standardized next-step failure probabilities
Because table tokens and text tokens operate under distinct linguistic priors and vocabulary distributions, raw entropy values across the two sets exhibit divergent numerical scales that cannot be combined directly via naive linear weighting. EcoTab resolves this by mapping each raw uncertainty \(\Phi_*^{(i)}\) (\(* \in \{\text{tab}, \text{text}\}\)) into a calibrated failure probability \(d_*^{(i)} \approx \Pr(\text{SRM fails on } s_{i+1} \mid \Phi_*^{(i)})\). To acquire clean step-level supervision without human annotation, EcoTab implements Counterfactual Suffix Replacement on held-out validation trajectories where the LRM succeeded but the SRM failed. For each trajectory of length \(T\), suffixes of length \(m \in \{1, \dots, \min(T-1, 8)\}\) are replaced by SRM completions for \(k=5\) stochastic trials. The smallest suffix length \(m^\star\) yielding an outcome flip rate of at least 80% marks the critical routing boundary \(b = T - m^\star\). Step \(s_b\) is labeled positive (demanding LRM escalation), prior steps \(s_1, \dots, s_{b-1}\) are labeled negative, and trailing steps are discarded. EcoTab then fits two independent, monotonic sigmoid mapping functions via binary cross-entropy:
$\(d_*^{(i)} = \sigma(a_* \Phi_*^{(i)} + b_*)\)$
where parameters \(a_*\) and \(b_*\) are learned entirely offline, introducing zero online parameter tuning.
4. Noisy-OR Probabilistic Risk Fusion and Threshold Decision: Fusing dual failure probabilities under causal fault tolerance without draft verification tokens
Given calibrated failure probabilities \(d_{\text{tab}}^{(i)}\) and \(d_{\text{text}}^{(i)}\), EcoTab models next-step failure under the principled causal assumption that an error in either tabular grounding or logical deduction independently suffices to compromise the entire reasoning trajectory. Hence, it synthesizes the final routing score via Noisy-OR gating:
$\(d_{\text{final}}^{(i)} = 1 - (1 - d_{\text{tab}}^{(i)})(1 - d_{\text{text}}^{(i)})\)$
If \(d_{\text{final}}^{(i)} > \tau\), the framework routes the generation of step \(s_{i+1}\) to LRM \(\theta_M\); otherwise, it continues with SRM \(\theta_m\). For the unconditioned initial step \(s_1\), the SRM generates a preliminary draft; if its evaluated \(d_{\text{final}}^{(1)} > \tau\), \(s_1\) is regenerated once by the LRM. All subsequent steps proceed strictly autoregressively, completely bypassing the need for speculative draft tokens, lookahead verifications, or rollback mechanisms.
Key Experimental Results¶
Main Results¶
The experiments benchmark EcoTab against state-of-the-art stepwise routing baselines on five complex tabular datasets: WikiTQ, TabFact, TableBench, HiTab, and FinQA. The evaluations feature both homogeneous collaboration (Qwen3-1.7B as SRM and Qwen3-14B as LRM) and heterogeneous cross-family collaboration (Qwen3-1.7B as SRM and DeepSeek-R1-Distill-Qwen-14B as LRM). Metrics include accuracy at 60% LRM FLOPs budget (Acc), FLOPs required to achieve 98% of LRM-only accuracy (FLOPs, lower is better), and Accuracy-per-FLOPs (A/F).
| Model Setup / Routing Method | Extra Tokens | WikiTQ (Acc/FLOPs) | TabFact (Acc/FLOPs) | TableBench (Acc/FLOPs) | HiTab (Acc/FLOPs) | FinQA (Acc/FLOPs) | Avg Acc ↑ | Avg FLOPs ↓ | A/F ↑ |
|---|---|---|---|---|---|---|---|---|---|
| Qwen3-14B (Homogeneous) | |||||||||
| SRM-Only (1.7B) | – | 67.34 / 3.84 | 82.42 / 2.51 | 42.55 / 6.12 | 59.34 / 4.82 | 47.52 / 7.38 | 59.83 | 4.93 | 15.20 |
| LRM-Only (14B) | – | 84.12 / 21.6 | 90.47 / 14.6 | 54.85 / 42.4 | 80.17 / 20.5 | 67.74 / 48.0 | 75.47 | 29.4 | 3.34 |
| Random Routing | ✗ | 75.82 / 18.5 | 86.76 / 10.6 | 46.12 / 36.7 | 73.43 / 17.4 | 58.78 / 40.4 | 68.18 | 24.7 | 3.84 |
| RSD | ✓ | 79.73 / 16.2 | 88.24 / 9.32 | 49.21 / 30.4 | 75.88 / 15.2 | 63.87 / 36.4 | 71.39 | 21.5 | 4.55 |
| SpecCoT | ✓ | 78.50 / 15.8 | 87.98 / 8.98 | 49.87 / 29.7 | 76.04 / 15.0 | 64.01 / 36.0 | 71.28 | 21.1 | 4.66 |
| SpecReason | ✓ | 78.76 / 15.7 | 87.74 / 9.21 | 50.02 / 29.9 | 76.41 / 15.0 | 64.21 / 36.0 | 71.43 | 21.1 | 4.62 |
| STEER | ✗ | 79.46 / 15.4 | 88.44 / 8.43 | 50.33 / 29.4 | 77.04 / 14.9 | 63.73 / 35.4 | 71.80 | 20.7 | 4.87 |
| GlimpRouter | ✓ | 79.04 / 16.4 | 88.85 / 8.56 | 50.52 / 30.1 | 76.22 / 15.4 | 63.98 / 36.1 | 71.72 | 21.3 | 4.72 |
| EcoTab (Ours) | ✗ | 80.83 / 14.0 | 89.14 / 8.42 | 51.77 / 28.1 | 78.32 / 13.9 | 65.48 / 34.3 | 73.11 | 19.7 | 5.15 |
| DeepSeek-R1 (Heterogeneous) | |||||||||
| LRM-Only (R1-14B) | – | 81.71 / 20.8 | 89.21 / 15.3 | 53.65 / 40.5 | 70.08 / 22.9 | 69.40 / 30.9 | 72.81 | 26.0 | 3.28 |
| STEER | ✗ | 78.32 / 14.4 | 87.64 / 8.83 | 49.22 / 29.1 | 67.01 / 15.3 | 66.15 / 21.3 | 69.67 | 17.8 | 4.91 |
| GlimpRouter | ✓ | 78.04 / 14.8 | 87.81 / 8.78 | 49.54 / 28.1 | 66.71 / 14.8 | 65.81 / 22.8 | 69.58 | 17.8 | 4.89 |
| EcoTab (Ours) | ✗ | 79.56 / 13.1 | 88.04 / 8.81 | 50.41 / 27.4 | 67.97 / 13.9 | 67.63 / 21.4 | 70.72 | 16.9 | 5.19 |
Ablation Study¶
The component ablation on TableBench evaluates the impact of removing token-level decoupling, retaining only a single token modality, and replacing the Noisy-OR formulation with linear weighting.
| Ablation Variant | Qwen3-Instruct Acc ↑ (Δ) | Qwen3-Instruct FLOPs ↓ (Δ) | DeepSeek-R1 Acc ↑ (Δ) | DeepSeek-R1 FLOPs ↓ (Δ) |
|---|---|---|---|---|
| Full EcoTab | 51.8 (–) | 28.1 (–) | 50.4 (–) | 27.4 (–) |
| w/ Average Token (No Token Splitting) | 49.9 (-1.9) | 30.3 (+2.2) | 48.9 (-1.5) | 30.0 (+2.6) |
| w/ Only Table Token | 50.5 (-1.3) | 29.4 (+1.3) | 49.6 (-0.8) | 28.3 (+0.9) |
| w/ Only Text Token | 50.2 (-1.6) | 29.7 (+1.6) | 49.4 (-1.0) | 28.6 (+1.2) |
| w/ Linear Weighting (Replacing Noisy-OR) | 50.0 (-1.8) | 29.8 (+1.7) | 49.1 (-1.3) | 29.0 (+1.6) |
Key Findings¶
- Token Decoupling is Crucial: Collapsing table and text tokens into an undifferentiated average (
w/ average token) causes the sharpest performance deterioration across both LRM settings (-1.9% and -1.5% accuracy, accompanied by +2.2 and +2.6 FLOPs increases), empirically demonstrating that unified step-level entropy washes out localized uncertainty signals. - Both Token Types Provide Complementary Value: Isolating either table tokens or text tokens alone leads to consistent performance degradation of 0.8% to 1.6%, proving that neither schema retrieval precision nor deductive linguistic coherence is individually sufficient for robust table routing.
- Noisy-OR Outperforms Linear Combination: Replacing probabilistic Noisy-OR with linear weighting lowers accuracy by 1.8% and increases FLOPs by 1.7, confirming that table reasoning failures follow an independent disjunctive hazard model rather than an additive balance.
- Robust Domain Transferability and Minimal Latency: The risk mapping fitted on WikiTQ transfers seamlessly to TableBench in an out-of-domain (OOD) evaluation, attaining 0.54 Acc@60%FLOPs and 29.1 FLOPs@98%Acc, maintaining a strong lead over STEER (0.50 / 29.4). Furthermore, EcoTab requires only 48.6 seconds of total routing overhead across TableBench—matching STEER (52.1s) and dramatically undercutting methods that generate extra verification tokens like SpecCoT and RSD.
Highlights & Insights¶
- Model-Free Trie for Zero-Overhead Token Disentanglement: By leveraging a normalized word-level Trie on the input table, EcoTab achieves instant, deterministic extraction of table entities within generated sequences without invoking secondary language models or attention introspection.
- Counterfactual Boundary Discovery for Unsupervised Step Labels: The suffix replacement protocol elegantly automates step-level supervision discovery by identifying the exact phase-shift boundary where SRM continuation causes irreversible trajectory divergence, avoiding noisy trajectory-level labels.
- Extreme-Value Probabilistic Modeling for Structured Reasoning: Framing dual failure risks under Noisy-OR provides a mathematically grounded, non-linear gating mechanism that triggers LRM escalation whenever either semantic retrieval or numerical execution exhibits elevated uncertainty.
Limitations & Future Work¶
- Lack of Adaptive Trajectory Length Control: EcoTab determines which model generates the upcoming step but does not regulate termination length; in degenerate scenarios, the SRM may continue generating repetitive, low-utility deductive loops despite low failure risk.
- Sensitivity to Extreme Numerical Reformatting: While the Table Trie handles whitespace, lowercasing, and standard punctuation, it relies on exact string overlap and may miss fuzzy numerical aliases, scientific notation, or unit-converted abbreviations (e.g., "$1.5M" vs. "1,500,000").
- Promising Future Directions: Future extensions could incorporate table-aware early stopping heuristics that terminate generation once sufficient cell constraints are fulfilled, alongside soft neural-symbolic embedding matchers for enhanced entity linkage robustness.
Related Work & Insights¶
- vs. GlimpRouter (Zeng et al., 2026): While GlimpRouter glances at the next draft token and assesses overall step entropy, it treats tabular steps as homogeneous text and requires draft token generation; EcoTab eliminates draft generation overhead and isolates structured table tokens, significantly reducing misrouting on table-specific steps.
- vs. STEER (Lee et al., 2026): STEER relies on monolithic confidence scores that fail to distinguish schema grounding from textual narration; EcoTab decouples the two modalities and recalibrates them through counterfactual risk mappings, achieving superior Pareto efficiency on tabular benchmarks.
- vs. RSD (Liao et al., 2025) / SpecReason (Pan et al., 2025): RSD and SpecReason rely on multi-path speculation or speculative draft verifications, incurring heavy extra token overhead; EcoTab operates strictly on already-generated prefixes, delivering state-of-the-art accuracy at substantially lower real-world routing latency.
Rating¶
- Novelty: ⭐⭐⭐⭐☆ Pioneers the empirical and theoretical study of token-level uncertainty heterogeneity in tabular stepwise routing, introducing an elegant Trie-based decoupling and Noisy-OR fusion mechanism.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluation spanning five mainstream tabular benchmarks, homogeneous and heterogeneous LRM pairings, component ablations, OOD transfer tests, and detailed routing latency profiling.
- Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally clear narrative progression connecting initial empirical anomalies to rigorous root-cause dissections and targeted architectural solutions.
- Value: ⭐⭐⭐⭐⭐ Offers a highly practical, drop-in, zero-speculative-token routing paradigm to accelerate large reasoning models on structured enterprise data.