Skip to content

TabSieve: Explicit In-Table Evidence Selection for Tabular Prediction

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/Kazeya27/TabSieve
Area: LLM Reasoning
Keywords: Tabular Prediction, In-Context Learning, Explicit Evidence Selection, GRPO, Task-Advantage Balancing

TL;DR

Addressing the lack of explicit evidence utilization and high vulnerability to noisy context in tabular in-context learning, TabSieve establishes a select-then-predict framework supported by the TabSieve-SFT-40K trajectory dataset and TAB-GRPO with dynamic task-advantage balancing, delivering significant performance gains across 75 classification and 52 regression tables.

Background & Motivation

Tabular data analysis and predictive modeling form the empirical backbone of critical decision-making systems across finance, healthcare, climate modeling, and energy management. Classical tabular predictors—ranging from gradient-boosted decision trees (e.g., XGBoost, CatBoost) to deep tabular architectures (e.g., FT-Transformer, TabPFN)—primarily operate under an isolated instance-wise inference paradigm. Consequently, they struggle to generalize across diverse tabular domains without per-table fine-tuning and cannot leverage the natural language semantics embedded in table column headers and metadata. With the advent of Large Language Models (LLMs), serializing tabular structures into natural language prompt sequences has emerged as an attractive frontier, enabling cross-table zero-shot and few-shot in-context learning (ICL) without parameter modifications.

However, naively prepending few-shot demonstration rows into prompts yields fragile and inconsistent gains. Attention-level diagnostic probes expose two fundamental failure modes in standard LLM tabular reasoning: first, unspecialized models fail to concentrate attention on genuinely relevant reference rows, dispersing attention weights across irrelevant table regions or dwelling on uninformative masked query targets; second, real-world few-shot demonstrations inevitably contain noisy or weakly relevant instances. When prompted without explicit guidance, LLMs can be actively misled by these noisy demonstration rows, performing significantly worse than under zero-shot settings due to negative context bias.

Resolving this tension requires transforming row selection from an implicit, passive byproduct of generation into an explicit, constrained intermediate reasoning decision that dynamically screens evidence prior to target derivation. The core idea is to establish a select-then-predict reasoning architecture where the model explicitly declares informative in-table evidence rows via an auditable selection trace before computing the final target, optimized through high-quality trajectory synthesis and multi-reward reinforcement learning (TAB-GRPO) equipped with dynamic task-advantage balancing.

Method

Overall Architecture

TabSieve employs a two-stage training paradigm: cold-start supervised trajectory learning on TabSieve-SFT-40K, followed by reinforcement learning using Group Relative Policy Optimization with Task-Advantage Balancing (TAB-GRPO). Given an input table prompt containing dataset metadata (table description, column descriptions, and valid candidate target ranges), a set of candidate demonstration rows, and a query row containing a [MISSING] target cell, TabSieve executes explicit multi-step reasoning.

The inference pipeline proceeds in three sequential phases: schema and dependency reasoning, where the model analyzes column semantics and hypothesizes underlying feature-target dynamics; explicit evidence selection, where the model isolates the most salient reference rows within <select> row_indices </select> tags; and noise-filtered quantitative deduction, where the model grounds its calculations directly on the selected instances and outputs the final prediction within \boxed{}.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Table Metadata + Candidate Rows + Query Row"] --> B["Mutual Information Filtering & Embedding Retrieval<br/>Construct high-SNR context pool and pseudo-evidence"]
    B --> C["Teacher-Driven Trajectory Synthesis & Strict Filtering<br/>Assemble TabSieve-SFT-40K supervised corpus"]
    C --> D["Decoupled Multi-Reward Design & Group Advantage Whitening<br/>Format rf + Evidence re + Correctness rc"]
    D --> E["Dynamic Task-Advantage Balancing Mechanism<br/>EMA-tracked rescaling of early-stage regression gradients"]
    E --> F["TAB-GRPO Policy Optimization & Explicit Inference<br/>Select salient evidence rows, then predict target value"]

Key Designs

1. Mutual Information Filtering & Embedding Retrieval: High-SNR Context Construction

To eliminate extraneous metadata clutter and spurious correlations that distract few-shot inference, preprocessing first purges high-cardinality identifier columns (e.g., user IDs, email addresses). The algorithm then calculates the mutual information \(I(X_j; Y)\) between each remaining candidate feature \(X_j\) and target \(Y\), ranking features in descending order and selecting the minimal subset whose cumulative mutual information reaches a 90% threshold: $\(m^\star = \min \left\{ m : \sum_{t=1}^m s_{(t)} \ge \rho \sum_{t=1}^{d'} s_{(t)} \right\}, \quad \rho=0.9\)$ Next, the target column is removed, and row embeddings from Qwen3-Embedding-8B are computed to retrieve the top-32 cosine nearest neighbors of the query row as a candidate evidence pool. In-context demonstration sets of size \(K \in \{0, 4, 8, 16, 32\}\) are then drawn by mixing retrieved neighbors with background table rows while strictly constraining the evidence ratio to \([0, 0.5]\), constructing an authentic testbed of semi-noisy few-shot context.

2. Teacher-Driven Trajectory Synthesis & Strict Filtering: Building TabSieve-SFT-40K

To supervise both the explicit evidence selection stage and the subsequent reasoning trace, a high-capacity teacher (Qwen3-Next-80B-Thinking) synthesizes step-by-step trajectories across 331 authentic tables. In the selection phase, the teacher identifies salient rows and rewrites the trace to eliminate any statements revealing prior knowledge of the target evidence budget. In the prediction phase, the teacher attempts up to five reasoning runs: classification trajectories must match ground-truth labels exactly, while regression tasks require an initial MAPE \(< 25\%\) followed by iterative chain-of-thought refinement until exact numerical agreement is achieved. A jury model (Qwen3-235B-A22B) subsequently filters out traces exhibiting hallucinated logic, abnormal length, or target leakage, yielding 41,083 high-fidelity cold-start trajectories.

3. Decoupled Multi-Reward Design & Group Advantage Whitening: Joint Evidence and Target Optimization

Directly optimizing end-to-end task loss fails to guarantee that the model uses valid demonstration rows. TAB-GRPO establishes a three-part reward system for each candidate rollout \(o_i\): - Format reward \(r_f \in \{0, 1\}\): Verifies strict syntactic compliance with both <select> bounding tags and \boxed{\cdot} answer wrapping; - Evidence reward \(r_e\): Computes the set-level F1 score between predicted evidence indices \(\hat{E}\) and pseudo-gold nearest neighbors \(E^\star\); - Correctness reward \(r_c\): Evaluates prediction accuracy via exact match \(1[\hat{y} = y]\) for classification, or exponential error shaping \(r_c = \exp(-\gamma \cdot \text{MAPE}(\hat{y}, y))\) with \(\gamma=1.0\) for regression.

For each group of \(G\) sampled rollouts per prompt, component rewards are individually whitened within the group to produce relative advantages \(A_i^{(f)}\), \(A_i^{(e)}\), and \(A_i^{(c)}\), providing balanced gradients that concurrently incentivize evidence grounding and final predictive accuracy.

4. Dynamic Task-Advantage Balancing Mechanism: Preventing Early-Stage Regression Dominance

In joint multi-task tabular training, empirical analysis reveals that early in optimization, regression instances produce significantly larger correctness advantage magnitudes \(|A^{(c)}|\) than classification tasks. This disparity skews the overall policy gradient toward regression-specific local optima, degrading classification generalization. TAB-GRPO remedies this optimization imbalance by maintaining an exponential moving average (EMA, decay \(\beta=0.99\)) of task-wise advantage scales: $\(m_t^\tau = \beta m_{t-1}^\tau + (1 - \beta) \, \mathbb{E}_{\tau} \left[ |A^{(c)}| \right]\)$ A bounded dynamic scaling factor is then applied exclusively to regression advantages: $\(\eta(\tau) = \begin{cases} 1, & \tau = \text{cls} \\ \text{clip}\left(\frac{m_t^{\text{cls}}}{m_t^{\text{reg}}}, \eta_{\min}, \eta_{\max}\right), & \tau = \text{reg} \end{cases}\)$ With clipping boundaries set to \([\eta_{\min}, \eta_{\max}] = [0.8, 1.0]\), the modified correctness advantage \(\tilde{A}_i^{(c)} = \eta(\tau) A_i^{(c)}\) enters the aggregated objective \(A_i = \lambda_f A_i^{(f)} + \lambda_e A_i^{(e)} + \lambda_c \tilde{A}_i^{(c)}\), effectively mitigating cross-task gradient conflicts during joint reinforcement learning.

A Worked Example

Consider a regression task predicting brain weight from anatomical measurements (Figure 8): - Input Context: Query row presents a male (Gender=1), young adult (Age=1), with HeadSize=3993 and WeightBrain=[MISSING]. The candidate context contains multiple instances, including Row 0 (Male, Age 1, HeadSize 4512 \(\to\) 1530g), Row 1 (Female, Age 1, HeadSize 3680 \(\to\) 1321g), and Row 2 (Male, Age 2, HeadSize 3415 \(\to\) 1310g); - Schema Reasoning: The model observes that brain weight exhibits strong positive correlation with head size, while gender introduces a categorical baseline offset; - Evidence Selection: Rather than fitting all available rows, TabSieve isolates the closest structural matches, outputting <select> 0, 1 </select>. Row 0 matches both gender and age, while Row 1 provides an adjacent head-size boundary; - Deductive Prediction: Using Row 0 as the primary proportionality anchor (\(1530 / 4512 \approx 0.34 \implies 3993 \times 0.34 \approx 1357\)) and deriving a fine-grained slope adjustment of \(\approx 41\text{g}\) from the gender comparison between Row 0 and Row 1, TabSieve predicts 1361g, closely matching ground truth (1360g). In contrast, Qwen3-8B performs global linear regression across all context rows, yielding an inaccurate estimate of 1398g due to noise interference.

Loss & Training

The underlying backbone is Qwen3-8B. The SFT stage fine-tunes on TabSieve-SFT-40K for 4.0 epochs using LLaMA-Factory with learning rate \(1 \times 10^{-4}\), cosine scheduling, 0.1 warmup ratio, and batch size 16 across 8 \(\times\) H200 GPUs (14.11 hours). The RL stage trains with TAB-GRPO on the VeRL framework for 2.0 epochs with rollout group size \(G=8\), temperature 1.0, learning rate \(1 \times 10^{-6}\), KL coefficient 0.001, and clip ratios \(\epsilon_{\text{low}}=0.20, \epsilon_{\text{high}}=0.28\) (25.08 hours). Reward combination weights are configured as \(\lambda_f=0.1, \lambda_e=0.2, \lambda_c=0.7\).

Key Experimental Results

Main Results

TabSieve is evaluated on a held-out benchmark compiled from GTL and TP-BERTa, comprising 75 classification and 52 regression tables. Evaluation spans zero-shot and few-shot budgets (\(K \in \{0, 4, 8, 16, 32\}\)) evaluated over 5,000 queries per shot budget across three random seeds. Metrics are Accuracy (%) for classification and NMAE (normalized mean absolute error, clipped at 1.0) for regression.

Model Category Model Classification 0-shot (Acc ↑) Classification 32-shot (Acc ↑) Regression 0-shot (NMAE ↓) Regression 32-shot (NMAE ↓) Mean Few-Shot Rank Rank\(_i\) (Acc / NMAE)
Traditional Tabular XGBoost — 62.98 — 0.160 10.63 / 8.23
CatBoost — 72.64 — 0.159 8.95 / 7.56
TabPFN — 73.67 — 0.144 8.09 / 6.70
General LLMs Llama-3.1-8B 38.32 47.50 0.567 0.319 13.51 / 14.07
Qwen3-8B (Base) 48.18 65.67 0.325 0.222 8.56 / 9.10
Qwen3-32B 51.18 70.67 0.317 0.182 6.17 / 8.33
Qwen2.5-72B-Instruct 54.07 70.87 0.297 0.178 6.12 / 7.15
DeepSeek-R1-70B 51.04 69.62 0.270 0.152 6.76 / 6.40
Tabular LLMs Tabula-8B 39.92 42.70 0.648 0.445 14.45 / 15.63
GTL-13B 48.62 67.15 0.304 0.203 7.92 / 8.89
P2T (Qwen3-8B) 48.72 66.34 0.315 0.215 8.36 / 9.25
Ours TabSieve (Ours) 54.63 74.33 0.266 0.139 5.46 / 5.95
Gain over Qwen3-8B \(\Delta\) over base +6.45 +8.65 -0.059 -0.082 -3.10 / -3.15

Ablation Study

Ablation analysis across all 127 benchmark datasets averaged over zero-shot and few-shot configurations:

Stage / Component Configuration Mean Classification Accuracy (Acc ↑) Mean Regression Error (NMAE ↓) Key Takeaway
Qwen3-8B (Base) 59.31 0.258 Raw foundation model without structured tabular reasoning
+ SFT (TabSieve-SFT-40K) 62.63 0.231 Supervised trajectories provide strong initialization
+ Vanilla GRPO 63.75 0.209 Standard RL suffers from early-stage regression domination
w/o Evidence Reward (\(r_e\)) 64.85 0.193 Dropping evidence reward degrades selection precision
+ TAB-GRPO (Full Model) 67.01 0.185 Task-advantage balancing yields optimal joint performance
+ SFT (w/o <select>) 61.26 0.240 SFT trained without explicit intermediate row selection
+ TAB-GRPO (w/o <select>) 64.26 0.218 RL policy trained without explicit intermediate row selection
TAB-GRPO w/o EMA Smoothing (\(\beta=0\)) 65.12 0.207 Local step fluctuations destabilize advantage tracking
TAB-GRPO w/o \(\eta_{\min}\) Lower Bound 66.04 0.202 Over-dampened regression updates hurt convergence

Key Findings

  • Crucial Role of Explicit Selection: Removing the explicit <select> step causes a 2.75% decline in classification accuracy and increases regression NMAE by 0.033. Furthermore, when rows selected by TabSieve are experimentally stripped from the prompt, Qwen3-8B accuracy drops by 26.5% (59.31% \(\to\) 43.57%) and NMAE degrades by 30.2% (0.258 \(\to\) 0.336), proving that TabSieve pinpoints the true informational core of the context.
  • Robustness Under Increasing Context Budgets: While baseline LLMs plateau or degrade when scaled to 32 shots due to context distraction, TabSieve widens its lead: classification gain over Qwen3-8B increases from +7.00% at 4 shots to +8.65% at 32 shots, while regression NMAE reduction deepens from -0.076 to -0.082.
  • Superior Macro-F1 on Imbalanced Classes: Under Macro-F1 evaluation across all classification tables, TabSieve achieves 60.31% at 32 shots, outperforming 70B-parameter reasoning models (DeepSeek-R1-70B at 59.93%) and dramatically surpassing Qwen3-8B (50.92%), proving that evidence selection prevents majority-class collapse in low-resource regimes.

Highlights & Insights

  • Attention Steering via Syntactic Scaffolding: Injecting an explicit <select> block in the reasoning trace alters Transformer internal self-attention distributions, transferring attention mass from noisy background tokens and masked targets onto evidential cells.
  • Cost-Effective Semantic Nearest Neighbors: Validating against leave-one-out influence functions confirms that cosine embedding retrieval captures over 80% of top-influence rows, providing high-quality pseudo-ground-truth evidence without combinatorial inference costs.
  • Seamless Zero-Shot to Few-Shot Transition: TabSieve excels across both paradigms: dynamically selecting and interpolating relevant reference rows when demonstrations are available, while gracefully falling back to world knowledge and scientific formulas under zero-shot conditions.

Limitations & Future Work

  • Text-Only Modality Boundary: Current formulations process strictly textual and numerical tables, leaving multimodal tables containing images, charts, and unstructured documents unexplored.
  • Backbone Scale Limitations: Experiments are conducted primarily on 8B parameter backbones; scaling the select-then-predict paradigm to 32B+ model architectures remains an open direction.
  • Deterministic Categorical Output: TabSieve outputs discrete predicted class tokens rather than full posterior probability distributions, limiting its direct integration into risk-sensitive pipelines requiring calibrated confidence estimates.
  • vs Classical Tree and Deep Tabular Models (Chen & Guestrin, XGBoost; Gorishniy et al., FT-Transformer; Hollmann et al., TabPFN): Traditional models require dataset-specific fitting and cannot understand column semantics; TabSieve achieves robust zero-shot and few-shot cross-table generalization.
  • vs Tabular Foundation LLMs (Wen et al., GTL; Gardner et al., Tabula-8B; Cai et al., TabR1): Existing tabular LLMs rely on passive concatenation and implicit reasoning, remaining vulnerable to noisy demonstrations; TabSieve explicitly isolates informative evidence rows, making context usage auditable and robust.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Formulates explicit evidence selection as an intermediate reasoning step and introduces task-advantage balancing for mixed tabular RL.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous evaluation across 127 datasets, extensive shot budgets, attention heatmaps, and ablation studies.
  • Writing Quality: ⭐⭐⭐⭐⭐ Highly coherent presentation with clear mathematical definitions and thorough empirical documentation.
  • Value: ⭐⭐⭐⭐⭐ Delivers a practical and auditable blueprint for applying reasoning LLMs to structured data and tabular analytics.