StrucTab: A Structured Optimization Framework for Table Parsing¶

Conference: ECCV 2026
Paper: ECCV Paper
Code: https://github.com/VirtualLUOUCAS/StrucTab
Area: Multimodal VLM
Keywords: Table parsing, structured reasoning, curriculum learning, decomposed rewards, destylization

TL;DR¶

StrucTab trains a 1B table parser to count rows and columns, identify merged cells, and then generate HTML, using Uni-TabRL to target the first structural error and normalize content-expression styles, improving TableVerse-5K TEDS from 84.59 for its SFT model to 87.81.

Background & Motivation¶

Converting a table image into HTML requires both text recognition and recovery of row, column, and spanning-cell relationships. Traditional pipelines separate structure recognition from optical character recognition (OCR), allowing upstream mistakes to propagate. End-to-end vision-language models (VLMs) remove the component interfaces but often supervise only the final HTML. The model must consequently infer two-dimensional topology from a linear output sequence, and an early mistake in rowspan or colspan can misalign subsequent cells.

Reinforcement learning (RL) does not automatically resolve this problem. A VLM judging whether two rendered tables match may overlook small structural differences, while direct HTML comparison with TEDS may penalize visually equivalent formulas written differently. Existing document benchmarks also contain relatively few tables and provide limited coverage of photographs, handwriting, and complex merged regions. The paper therefore revisits intermediate supervision, rewards, and evaluation rather than simply increasing model size.

The authors first conduct training-free experiments supplying row-column and merged-cell hints, finding that explicit structural information helps, and then turn those hints into prediction targets. Core Idea: teach two-dimensional structure through supervised intermediate skills, make the model use those skills in a fixed order, and optimize parsing with rewards that respond to early structural deviations while reducing sensitivity to equivalent content expressions.

Method¶

Overall Architecture¶

The input is a table image, and the output is a structured response containing row-column counts, merged-cell analysis, and final HTML. Built on HunyuanOCR, the model undergoes independent-subtask pretraining, sequential-reasoning pretraining, supervised fine-tuning (SFT), and Uni-TabRL. At inference time, a single model generates the response; an external cascade of parsing models is not required.

There are four key designs: structural subtask decomposition, sequential curriculum training, first-error structural reward, and anchor-guided destylization. The first pair establishes structural skills, while the second pair improves structural and content feedback during RL. Validity checking and GRPO use these signals for policy updates.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Input["Training images and HTML labels"] --> Decompose["Structural subtask<br/>decomposition"]
    Decompose --> Curriculum["Sequential curriculum<br/>training"]
    Curriculum --> Rollout["SFT model samples responses"]
    Rollout --> Probe["First-error<br/>structural reward"]
    Rollout --> Anchor["Anchor-guided<br/>destylization"]
    Reference["Reference HTML"] --> Probe
    Reference --> Anchor
    Probe --> Update["Validity checking and GRPO updates"]
    Anchor --> Update
    Update --> Output["Trained model<br/>Counts → merged cells → HTML"]

Rendering, the OCR anchor, and reference HTML belong to training-time reward computation, not mandatory deployment-time post-processing. TableVerse-5K is a separate evaluation contribution rather than a step in the inference chain.

Key Designs¶

1. Structural subtask decomposition: explicitly supervise geometry hidden inside HTML

Row-column counting supplies a global constraint: the model first reports the table dimensions instead of immediately emitting markup. Merged-cell analysis supplies local constraints: the rowspan and colspan attributes in annotated HTML define a merged region represented as <merge>(r1,c1),(r2,c2)</merge>, a rectangle between its starting and ending grid positions. Multiple regions follow a fixed row-wise order. These targets are derived directly from existing HTML annotations, without manually labeling intermediate reasoning traces.

The two targets are complementary. Correct dimensions do not imply that a spanning header is recovered, and plausible merged regions do not guarantee the correct global grid. Separate supervision establishes global dimensions and local dependencies before HTML expresses text and topology. Compared with training on final answers alone, the important change is that intermediate structural decisions receive explicit targets.

2. Sequential curriculum training: learn judgments independently, then condition generation on them

The first stage constructs separate question-answer tasks for row-column counting, merged-cell analysis, and HTML parsing. Only the second stage combines them into one response, ordered as counts, merged regions, and final HTML. This is neither an additional planner invocation nor unrestricted long-form chain-of-thought. A fixed format places verifiable structural conclusions into the context of subsequent autoregressive generation.

The curriculum addresses two distinct issues: independent training avoids imposing every output requirement from the beginning, and sequential integration prevents auxiliary skills from remaining disconnected from final parsing. Table 3 compares independent multitask training, direct sequential training, and the complete curriculum, testing both structural supervision and the progression from independent to integrated tasks. Intermediate outputs remain predictions rather than ground truth, so incorrect counts can still affect later output at test time. The process is not a symbolic constraint solver with a correctness guarantee.

3. First-error structural reward: prioritize a longer correct structural prefix

Uni-TabRL first checks whether the response contains a complete, valid HTML table and correct row-column and merged-cell outputs. Failure of any condition gives a validity reward of 0; otherwise it is 1. The structural reward then removes cell text, retains tags and span attributes, and linearizes predicted and reference structures into cell sequences. Comparison starts at the beginning and stops accumulating credit at the first mismatch.

Using shorthand introduced in this note, let \(k\) be the number of consecutive correct cells before the first error and \(N\) the reference cell-sequence length. The reward described in the text and Figure 5a is:

\[ R_{\mathrm{structure}}=\frac{k}{N}. \]

For a reference sequence containing 6 cells, if the first 3 match and the fourth has an incorrect span attribute, the reward is \(3/6\). Later matches do not extend this correct prefix. This is not an immediate reward assigned separately to each token: the error location determines a response-level score used by GRPO. Compared with an aggregate tree-similarity score, it directly encourages delaying the earliest structural deviation, targeting the downstream propagation of autoregressive errors.

4. Anchor-guided destylization: standardize expressions before comparing content

The same formula may use LaTeX or Unicode symbols, causing direct string comparison to lower TEDS despite visual equivalence. The authors render both predicted and reference HTML into standardized images, then re-parse both with the same frozen OCR model. Common fonts, layout, and parser make the resulting expressions more consistent. TEDS between these destylized representations becomes the content reward.

Unlike a VLM judge answering whether images match, the frozen OCR model serves as an expression-style anchor and the final comparison remains rule-based. It also does not simply run OCR on two original table photographs: the standardized images are newly rendered from the two HTML representations. The paper motivates this as more robust content supervision, but standard TEDS still includes structure and OCR can introduce errors. It should not be interpreted as a guarantee of purely semantic or perfectly style-invariant scoring.

A Worked Example¶

Consider an illustrative table with 3 rows and 3 columns, whose first-row header spans columns 1 and 2. This is a teaching example, not an additional experiment reported in the paper.

The model first outputs 3 rows and 3 columns, then <merge>(1,1),(1,2)</merge>, and finally HTML with colspan="2" in the corresponding first-row cell and recognized text in the remaining cells. Counting establishes the global grid, while the merged region warns the decoder not to split the header into two ordinary cells.

During RL, valid HTML with incorrect counts still receives a validity reward of 0. An early span mistake limits the first-error structural reward to the preceding correct cells. For equivalent formula expressions, anchor-guided destylization attempts to reduce unnecessary content penalties through common rendering and re-parsing. Deployment uses only the model's sequential response, not these reference-dependent reward computations.

Loss & Training¶

All stages use full-parameter training, with experiments conducted on 16 NVIDIA H20 GPUs. Pretraining mixes 6M synthetic tables with public datasets. Independent-task training lasts 1 epoch at a learning rate of \(4\times10^{-5}\), followed by another 1 epoch of sequential integration. The 6M figure refers to synthetic samples, not the total mixed dataset size.

The downstream high-quality dataset contains 130K samples: 50K from public benchmarks and 80K manually annotated tables. SFT uses 100K samples for 2 epochs at \(2\times10^{-5}\), and the remaining 30K are used for RL. GRPO samples multiple responses for the same image, forms advantages from group-normalized rewards, and applies clipped policy-ratio updates without training a separate critic.

Equation (4) is corrupted in the available text extraction, while the prose mentions three weights and a hard validity gate. This note does not reconstruct the exact additive, multiplicative, or gated reward composition, nor invent weights, group size, or RL learning rate. The cache does not include the appendix referenced by the paper; these reproduction details require the original equation and appendix.

Key Experimental Results¶

Main Results¶

TEDS measures both HTML tree structure and cell content; TEDS-S ignores text and measures structure only. Scores below use the paper's 0–100 scale, with higher values better. TableVerse-5K contains 5,000 Chinese and English tables covering scans, photographed printed tables, and photographed handwritten tables, with an average of 5.10 merged cells per table (Table 1).

The following subset comes from Table 2. The two StrucTab rows are interleaved in the text extraction; their TableVerse-5K values are independently confirmed by the separate rows in Table 4.

Model	TableVerse-5K TEDS	TableVerse-5K TEDS-S	Table 2 Average TEDS
GPT-5	67.04	78.96	69.65
Gemini 2.5 Pro	79.46	87.13	81.66
TRivia-3B	78.15	85.41	80.87
HunyuanOCR	77.55	85.22	79.67
StrucTab-SFT	84.59	90.02	85.87
StrucTab	87.81	92.74	88.79

The full model improves TableVerse-5K TEDS by 3.22 points over StrucTab-SFT and 10.26 points over HunyuanOCR. The SFT-to-RL gain in Table 2 Average TEDS is instead 2.92 points; these are different statistics. Table 2 Average is not the simple arithmetic mean of the four benchmark scores and should not be relabeled a four-dataset macro-average. Opensource Avg. in Tables 3 and 4 covers only the three public benchmarks.

Ablation Study¶

Table 3 isolates structural supervision and sequential curriculum training. All configurations below precede RL.

Structural training configuration	Public-benchmark average TEDS	TableVerse-5K TEDS	TableVerse-5K TEDS-S
(a) Final-HTML-only baseline	86.26	81.12	85.51
(b) Add row-column counting	87.43	82.09	86.79
(c) Add merged-cell analysis	87.39	82.39	87.01
(d) Both independent auxiliary tasks	88.93	83.24	88.61
(e) Direct sequential training, no independent-task curriculum	89.54	84.12	89.40
(f) Full StrucTab-SFT	90.11	84.59	90.02

Table 4 compares RL rewards starting from the same SFT model. Every RL configuration includes the validity reward.

Reward configuration	Public-benchmark average TEDS	TableVerse-5K TEDS	TableVerse-5K TEDS-S
StrucTab-SFT, no RL	90.11	84.59	90.02
(a) TEDS-S + TEDS	91.29	86.45	91.67
(b) TEDS-S + VLM judge	90.51	85.89	91.57
(c) TEDS-S + anchor-guided destylization	91.76	87.10	92.16
(d) First-error structural reward + TEDS	91.80	87.07	92.06
(e) First-error structural reward + anchor-guided destylization	92.05	87.81	92.74

Key Findings¶

Combining the two auxiliary targets outperforms either alone. On TableVerse-5K, the full curriculum further improves TEDS by 0.47 points over direct sequential training, supporting learning independent skills before integration without attributing all gains to the curriculum.
Relative to ordinary RL, adding anchor-guided destylization or the first-error structural reward alone improves TEDS by 0.65 and 0.62 points, respectively; combining them improves it by 1.36 points. The VLM-judge variant instead loses 0.56 points, showing that a more complex judge need not provide more reliable feedback.
Table 5 describes repetitive generation as failing to emit a closing </table> and entering a repetition loop. TEDS + TEDS-S produces 60/5,000 cases, or 1.20%; TEDS + first-error structural reward reduces this to 51/5,000, or 1.02%, without eliminating the problem. This is the tested reward variant's repetition rate, not a reported rate for the complete model.

Highlights & Insights¶

Intermediate structural labels can be derived from existing HTML rather than manually authored reasoning traces. The transferable idea is extracting verifiable prerequisite states from output annotations, not merely requesting longer explanations.
The first-error reward introduces autoregressive order into structural scoring. It gives up credit for local recovery after a mistake in exchange for direct pressure on the correct prefix, suggesting applications to other structured outputs with strict sequential dependencies.
Destylization separates content consistency from string consistency. A shared canonicalization process is better suited to formula-rich tables than assuming edit distance captures every form of semantic equivalence.

Limitations & Future Work¶

The main paper does not provide a dedicated systematic limitations section. The following observations concern methodological and experimental boundaries, rather than conclusions already established by the authors.
Correct-prefix scoring gives no additional structural credit beyond the earliest mistake, and validity is binary. Whether this produces excessively sparse feedback on particularly difficult long tables requires difficulty-stratified experiments.
Rendering and frozen OCR add reward-computation costs and depend on anchor accuracy. The main text does not report reward throughput, total training time, or end-to-end inference latency; a 1B parameter count alone does not establish a cheap overall training pipeline.
TableVerse-5K uses three-model pre-annotation, human selection and correction, and a second annotator's verification. Its Chinese and English coverage does not represent all writing systems. Separate results for handwriting, formulas, and very long tables would clarify where the gains concentrate.
Training includes public-benchmark-source data, but the available main text does not establish every training/evaluation split boundary. Further disclosure is needed; this is not evidence sufficient to assert data leakage. The missing appendix also limits reproduction of reward hyperparameters and exact computation details.

vs HunyuanOCR: StrucTab reuses its 1B backbone and primarily changes task organization and post-training objectives. The results therefore concern specialized structural supervision and optimization rather than a new network architecture.
vs FD-RL: Direct TEDS rewards can penalize equivalent expressions. StrucTab adds common rendering and OCR canonicalization before reward comparison, introducing dependence on frozen-OCR accuracy and computation in return.
vs MonkeyOCR-v1.5 / TRivia: The former uses a VLM visual-consistency judge, while the latter uses auxiliary question-answer feedback. StrucTab emphasizes whole-table structural prefixes and computable comparisons of normalized content, operating at a different supervision granularity.
For structured generation: Global dimensions, local relationships, and the final sequence can form separately checkable layers, with rewards distinguishing syntactic usability, structural error location, and content equivalence. Other tasks require their own definitions of these layers rather than direct reuse of table-specific rules.

Rating¶

Novelty: 4/5. Structural subtasks are not entirely new, but their combination with correct-prefix scoring and OCR-anchor canonicalization is well targeted.
Experimental Thoroughness: 4/5. Multiple benchmarks, two distinct ablation groups, and repetition analysis are provided, while cost, stratified results, and unavailable reproduction details remain gaps.
Writing Quality: 4/5. Motivation and validation are clear, but response-level first-error scoring must not be confused with per-token rewards, and different Average statistics require care.
Value: 4/5. The work provides useful guidance for supervision design and RL reward engineering in lightweight table parsing.