Advancing WordArt-Oriented Scene Text Recognition: Datasets and Methods¶

Conference: ECCV 2026
arXiv: 2606.24484
Code: https://github.com/YesianRohn/WATER
Area: Scene Text Recognition / Image Generation
Keywords: WordArt recognition, scene text recognition, synthetic data, arbitrary-shape encoding, autoregressive decoding

TL;DR¶

This paper systematically advances WordArt-oriented scene text recognition (WATER) from both data and model perspectives. On the data side, a 2M synthetic WordArt dataset named WATER-S is constructed, consisting of a rendering-based subset WATER-T and a generative-model-based subset WATER-Z. On the model side, WATERec is proposed, supporting arbitrary-shape input and autoregressive decoding. It achieves over 90% accuracy (90.40%) on WordArt-Bench for the first time, significantly outperforming general VLMs and OCR-specific VLMs.

Background & Motivation¶

WordArt is widely used in design scenarios such as posters, signboards, and magazines. Its font shapes, textures, and layouts are highly customized, with complex graphics and semantic elements often embedded within the character outlines. Consequently, the text serves both as a language medium and a visual symbol. This high level of stylization introduces substantial visual interference unrelated to the underlying text, making WordArt-oriented scene text recognition (WATER) far more challenging than general scene text recognition (STR).

Existing STR datasets and methods are primarily built around regular scene text and fixed template inputs, facing two core bottlenecks in WordArt scenarios. First, real WordArt data is extremely scarce: the largest existing WordArt dataset, WordArt, contains only 4,805 training images. The high annotation cost and poor consistency are insufficient to support the optimization of modern models, failing to cover long-tail styles in real designs. Second, mainstream STR models generally resize inputs to fixed dimensions (e.g., 32x128). However, the aspect ratios of WordArt are extremely volatile (ranging from ultra-wide banners to nearly square vertical text). Forcing these into fixed shapes causes severe geometric distortion, destroying character structures. Additionally, the reading order of WordArt often violates standard left-to-right rules, demanding stronger contextual reasoning capabilities.

The core idea of this work is to break the data bottleneck using two complementary synthetic data pipelines (precise rendering-based control and diverse generative-model-based style generation). Concurrently, it breaks the architectural bottleneck of fixed-template STR using an arbitrary-shape encoder that preserves original aspect ratios mated with an autoregressive decoder. This synergy elevates the WATER task to practical levels of accuracy for the first time.

Method¶

Overall Architecture¶

The methodological framework consists of two collaborative pillars: the construction of the synthetic dataset WATER-S on the data side, and the recognition architecture WATERec on the model side. The data side features two parallel synthetic pipelines: rendering-engine-based WATER-T and generative-model-based WATER-Z, which complement each other in font control, layout diversity, and visual realism. On the model side, WATERec adopts a NaViT-style visual encoder (supporting arbitrary-aspect-ratio inputs along with RoPE position encoding) followed by an autoregressive Transformer decoder. This fundamentally avoids distortions caused by fixed-template resizing and enables modeling of complex WordArt reading orders. During training, WATERec is first trained on the real dataset WATER-R (3.2M samples, strictly de-duplicated via hashing) and then further boosted by incorporating WATER-S.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Artistic Font Library<br/>11K Fonts + Backgrounds"] --> B["SynthWordArt Rendering<br/>Generate WATER-T (1M)"]
    C["Real WordArt Images<br/>WordArt + WAS-R"] --> D["Qwen3-VL Description Mining<br/>273K Prompt Templates"]
    D --> E["Z-Image-Turbo Generation<br/>Generate WATER-Z (1M)"]
    B --> F["WATER-S Synthetic Dataset (2M)"]
    E --> F
    F --> G["Arbitrary-Shape Visual Encoder<br/>NaViT + RoPE"]
    G --> H["AR Transformer Decoder<br/>Char-by-Char Prediction"]
    H --> I["Output Text Sequence"]

Key Designs¶

1. SynthWordArt Rendering Engine: An Instrumental Synthetic Pipeline Custom-Built for WordArt

Traditional STR synthesis tools (e.g., SynthText, SynthTIGER) are designed for regular text, employing general fonts (like Google Fonts) and standard horizontal layouts. The generated images exhibit a massive gap in style and layout compared to real WordArt. SynthWordArt introduces two key upgrades. First, the general font library is replaced with 11,250 collected open-source artistic fonts, covering style labels such as display, handwriting, cartoon, and playful, while ensuring license compliance (OFL, Apache, etc.) and complete character encodings. Second, beyond standard horizontal layouts, it introduces curved arrangements, vertical text, multi-directional layouts, and geometric transformations such as perspective and stretching. Layout types are randomly sampled with a ratio of approximately \(0.2\) (curved) : \(0.2\) (vertical) : \(0.3\) (multi-directional) : \(0.3\) (near-horizontal). Text labels are directly sampled from the real text corpus of existing large-scale real datasets (598,615 unique entries, length 1–25) to narrow the textual distribution gap between synthetic and real data. Generative synthesis using this engine results in 1M images forming WATER-T, whose core advantage lies in precise control over text content and character shapes, with every character rendering strictly aligning with its ground-truth label.

2. VLM-Driven Prompt Mining and Generation Pipeline: Filling the Realism Gap with Generative Models

Although procedural rendering provides precise control, it still lags behind real-world designs in global style consistency, semantic texture blending, and overall visual naturalness. Consequently, this paper explores generating WordArt data using generative models, where the critical factor is prompt quality. An automated few-shot prompt mining pipeline is designed: first, Qwen3-VL-8B is utilized to generate detailed descriptions for each of the 31,335 real WordArt images in the WordArt and WAS-R training sets (using <Text> placeholders in descriptions to replace specific text content), yielding 31,335 image-description pairs. Next, 3 examples are randomly sampled as few-shot exemplars, prompting Qwen3-VL to mimic and generate new prompts. Finally, a strict de-duplication and filtering process yields 273,488 high-quality prompt templates. This prompt library is independent of any specific generative model and can be repurposed for more advanced generators. Z-Image-Turbo is chosen as the backbone generator with an output resolution of 256x256, and after randomly replacing text placeholders in the prompts, 1M images are synthesized to form WATER-Z. Compared to WATER-T, WATER-Z displays much higher diversity and realism in background textures, composition, and global visual styles, although character readability and structural accuracy are harder to guarantee—making the two datasets highly complementary.

3. Arbitrary-Shape Input and RoPE Position Encoding: Breaking the Architectural Bottlenecks of Fixed Templates

WordArt aspect ratios are extremely volatile. Resizing all inputs to a fixed dimension (e.g., 32x128) in traditional STR models severely stretches or squashes irregular text, while multi-scale template schemes like SVTRv2 still require manual pre-definition and lack flexibility. Drawing inspiration from modern VLMs, WATERec adopts a NaViT-style visual encoding strategy: given an input image \(I \in \mathbb{R}^{H \times W \times 3}\) of arbitrary resolution, it is scaled to \(\hat{I}\) while preserving its original aspect ratio, partitioned into patches of size \(p=4\), and linearly projected into \(d=384\)-dimensional visual tokens. The number of tokens \(N = (\hat{H}/p) \times (\hat{W}/p)\) is constrained within the range \([64, 256]\) (the lower bound prevents extremely small images from having too few tokens, while the upper bound ensures fair comparison with existing methods). The patch tokens are flattened into a 1D sequence in raw-major order and fed into a 6-layer Transformer encoder.

The key challenge lies in how the encoder perceives the spatial layout of arbitrary-shape inputs. Absolute Position Encoding (APE) and Sinusoidal Position Encoding (SPE) are highly sensitive to image size and shape, making it impossible for training data to cover all resolutions and layouts. WATERec employs RoPE (Rotary Position Embedding) to apply position-dependent rotations to query and key vectors, allowing dot-product attention to implicitly encode relative positional relationships. Specifically, the \(d\)-dimensional Q/K vectors are paired across adjacent channels and converted into complex forms \(\bar{q}_n, \bar{k}_n \in \mathbb{C}^{d/2}\). Half of the complex channels are allocated to the horizontal axis and the other half to the vertical axis, with the frequency coefficient \(\omega_t = \theta^{-t/d}\) (\(\theta=100\)). A rotary matrix \(R_n\) is constructed for position \(p_n\) and applied via element-wise complex multiplication \(\bar{q}'_n = \bar{q}_n \circ R_n\). This design enables the model to naturally handle variable-length sequences, ensuring strong generalization across different scales and resolutions. Ablation studies demonstrate that under arbitrary-shape inputs, RoPE significantly outperforms APE and SPE, while completely removing positional encoding (NoPE) causes performance to collapse (dropping WordArt from 88.55% to 49.57%).

4. Autoregressive Decoder: Adapting to Complex WordArt Reading Orders

WordArt often violates the standard left-to-right reading order (curved, vertical, multi-directional). Under such scenarios, Connectionist Temporal Classification (CTC) or parallel decoding (PD) approaches are prone to misalignment or omissions. WATERec employs an autoregressive (AR) Transformer decoder that leverages 2 cross-attention layers to attend to the sequence of visual features output by the encoder. Using <B> as the start token and <E> as the end token, it predicts the text sequence char-by-char (maximum length of 25, with a character set of 94 including numbers, letters, and common symbols). Through char-by-char contextual field iteration refinement, AR decoding handles highly distorted text and non-conventional reading orders much more effectively. The model is trained using a standard cross-entropy loss, operating at an inference speed of 361.66 FPS (V100) with 26.22M parameters, achieving an elegant balance between accuracy and efficiency.

Loss & Training¶

WATERec is trained end-to-end using standard cross-entropy loss. The optimizer is AdamW with a weight decay of 0.05, a base learning rate of \(6.5 \times 10^{-4}\), and a total batch size of 2048. A one-cycle learning rate scheduler is utilized with a linear warmup for the first 1.5 epochs, training for 20 epochs in total. Data augmentation follows the PARSeq recipe: random rotations, perspective distortions, motion blur, and Gaussian noise. The maximum text length is set to 25. All models are trained on 8 NVIDIA V100 GPUs. In terms of training data, the base model is first trained on WATER-R (3.2M real samples, formed by merging Union14M-L, WordArt-Train, and WAS-R after strict cryptographic hash de-duplication), and then further boosted by superimposing WATER-S. The optimal ratio is 2M WATER-S + 3.2M WATER-R (synthetic-to-real ratio of approximately 2:3). Continuing to scale up the synthetic data proportion yields diminishing marginal returns or minor fluctuations in certain subsets.

Key Experimental Results¶

Main Results¶

Comparison of mainstream STR methods trained on WATER-R (excluding synthetic data). WATERec achieves the best performance on both the WordArt benchmark A-Bench (WordArt test set, 1,511 images) and the highly challenging U-Bench (Union14M-Benchmark, 7 subsets), while ranking slightly below the fixed-resolution MAERec on the regular-text-dominant C-Bench (6 general benchmarks).

Method	Decoding Type	A-Bench (WordArt)	C-Bench AVG	U-Bench AVG
CRNN	CTC	62.54	86.07	41.63
SVTR	CTC	76.96	93.16	70.40
SVTRv2	CTC	86.56	96.79	86.14
ABINet	PD	84.64	96.13	80.58
PARSeq	AR	84.51	96.11	82.26
MAERec	AR	86.23	96.96	86.52
SVTRv2-AR	AR	87.36	96.48	87.63
WATERec	AR	88.55	96.69	88.14
WATERec + WATER-S (2M)	AR	90.40	97.01	89.38

After superimposing WATER-S, WATERec reaches 90.40% on A-Bench, breaching the 90% barrier for the first time. In comparison, the strongest general VLM (Qwen3-VL-8B) yields only 72.01%, and the strongest OCR-specific VLM (HunyuanOCR) scores 81.54%. Even when Qwen3-VL-8B is fine-tuned via SFT (LoRA fine-tuning for 20k steps) with WATER-R + WATER-S, its performance only reaches 84.78%, remaining far below the 26M-parameter WATERec. This suggests that dedicated specialist models remain irreplaceable for this task.

Ablation Study¶

Ablation studies of WATERec's arbitrary-shape modeling, positional encoding schemes, and token ranges (all trained on WATER-R).

Arbitrary Shape	Positional Encoding	Token Range	A-Bench	C-Bench Regular	C-Bench Irregular	U-Bench AVG
No	NoPE	[256, 256]	86.83	98.25	95.15	86.69
Yes	NoPE	[64, 256]	49.57	89.14	86.07	56.90
Yes	APE	[64, 256]	87.69	98.29	94.41	87.10
Yes	SPE	[64, 256]	87.29	98.36	94.37	86.87
No	RoPE	[256, 256]	87.88	98.71	94.99	86.99
Yes	RoPE	[1, 256]	88.29	98.47	94.80	87.38
Yes	RoPE	[64, 256]	88.55	98.52	94.86	88.14
Yes	RoPE	[64, 512]	88.82	98.61	95.75	89.06

Key findings: The combination of arbitrary shape and NoPE leads to a catastrophic performance collapse (A-Bench 49.57%), indicating that without spatial layout information, the model is completely incapable of interpreting arbitrary-shape inputs. Under arbitrary-shape inputs, RoPE significantly outperforms both APE and SPE. Expanding the token upper limit to 512 further improves accuracy, but the FPS drops from 361.66 to 191.46; thus, [64, 256] represents the optimal trade-off between accuracy and efficiency.

Key Findings¶

Arbitrary-shape modeling is the primary contributor: Removing arbitrary-shape support (degrading to a fixed [256, 256] input + NoPE) drops A-Bench by 1.72 percentage points and U-Bench by 1.45 percentage points. Furthermore, arbitrary shapes require proper positional encoding to unleash their potential—making RoPE a key enabler.
Strong complementarity between WATER-T and WATER-Z: The 1M WATER-S dataset, consisting of 0.5M WATER-T and 0.5M WATER-Z, outperforms the individual 1M versions on A-Bench (89.94% vs. 89.81%/89.41%). This validates the mutual complementarity of the two pipelines in visual styles and scene diversity.
Synthetic data gains follow a "rise-then-plateau" trend: Increasing WATER-S from 1M to 2M yields consistent and stable gains (+0.46% on A-Bench), but scaling to 3M leads to vanishing marginal returns and minor fluctuations in some subsets. The optimal synthetic-to-real ratio is approximately 2:3.
Cross-architectural utility of synthetic data: Superimposing WATER-S on four different models—SVTRv2 (CTC), ABINet (PD), SVTRv2-AR (AR), and WATERec—consistently brings a +1.85% to +2.78% improvement on A-Bench. This demonstrates that high-quality WordArt synthetic data serves as a universal resource rather than a model-specific trick.
A minor regression on C-Bench is an acceptable trade-off: WATERec scores slightly lower than the fixed-resolution MAERec on the regular-text-heavy C-Bench (less than 0.3% gap), but dominates on WordArt and other challenging subsets, representing an inevitable cost for preserving raw aspect ratios.

Highlights & Insights¶

The combination of arbitrary-shape and RoPE is simple yet highly efficient: Without adding extra parameters or modifying the core Transformer backbone, simply preserving the aspect ratio and substituting fixed positional encodings with RoPE brings qualitative improvements in WordArt recognition. This paradigm is highly transferable to any computer vision task dealing with extreme aspect ratio variations (e.g., document understanding, chart parsing, remote sensing).
The prompt library serves as a generator-agnostic asset: The 273K high-quality prompt templates are decoupled from specific generative models, allowing them to be systematically recycled as generative models evolve. This "decoupled description and generation" approach shows strong foresight.
Insights into the "moderate scale" of synthetic data: The experiments clearly illustrate that larger synthetic data volumes are not always better. At 3M, noise accumulation and distribution mismatches begin to interfere; 2M (a synthetic-to-real ratio of ~2:3) remains the optimal configuration. This finding carries direct practical guidelines for researchers building synthetic datasets.
The gap between specialist models and VLMs emphasizes the ongoing importance of task-specificity: Even when fine-tuning the 8B Qwen3-VL via SFT using identical WATER-S data, its WordArt recognition accuracy (84.78%) lags significantly behind a 26M specialist model (90.40%). This reminds researchers not to over-rely on the general capabilities of VLMs—on highly specialized perceptual tasks, tailored architectures combined with task-specific data remain irreplaceable.

Limitations & Future Work¶

Current versions of WATER-S and WATERec are predominantly tailored for English WordArt. Although the authors present a small-scale Chinese verification (achieving 92.08% accuracy on 101 Chinese WordArt samples of BCTR-Test using WATERec + Chinese WATER-S), systematic evaluation across multi-language scenarios is still missing.
The text rendering accuracy of the generative subset WATER-Z is somewhat unreliable (with an error rate of ~12.56%). Although the authors show that OCR filtering provides no noticeable benefits (the model learns robustly despite moderate noise), whether this noise remains "beneficial" under more advanced generators requires closer inspection.
WATERec experiences a minor regression on the regular-text Benchmarks (under 0.3% difference vs. the best fixed-resolution methods). If downstream applications demand simultaneous handling of both regular text and WordArt, multi-branching or dynamic routing strategies may be warranted.
The inference speed of autoregressive decoding (361 FPS) lags behind CTC-based approaches (such as SVTRv2 at 608 FPS), potentially posing throughput bottlenecks in real-time settings. Devising a hybrid scheme featuring parallel decoding and arbitrary-shape encoding presents a promising direction.
The authors mention plans to adopt VLMs for WordArt recognition (e.g., via chain-of-thought reasoning) to approximate specialist performance, yet they lack in-depth analysis regarding why VLMs perform poorly on WordArt (is it due to visual encoder resolution limits, or the lack of character-level perception in decoders?).

vs CornerTransformer: The first baseline tailored for WordArt recognition, employing an AR paradigm. This work systematically upgrades it across both the data dimension (WATER-S vs. a meager 4,805 WordArt images) and the architecture dimension (arbitrary shapes vs. fixed templates), successfully pushing A-Bench scores from around 80% to 90%+.
vs SVTRv2: Also targets irregular text, but SVTRv2’s multi-scale template scheme requires manual pre-definition and applies fixed transformation sets to all samples. In contrast, WATERec's arbitrary-shape strategy is more flexible and bypasses manual setup.
vs TextSSR / SceneVTG: Both works generate STR data using diffusion models, but they target general scene text rather than WordArt. This paper’s VLM prompt mining pipeline explicitly focuses on WordArt style descriptions, making the generation quality in WordArt domains significantly superior to TextSSR.
vs NaViT / Modern VLMs: The arbitrary-shape encoding concept in this study is directly inspired by NaViT, adapting it from unified vision-language domains to the vertical domain of STR and verifying its efficacy. This illustrates that VLM architectural innovations can effectively benefit classic computer vision tasks.

Rating¶

Novelty: ⭐⭐⭐⭐ While individual components (e.g., RoPE, AR decoding, rendering pipelines, diffusion-based generation) are not entirely new, systematically grouping them to solve the under-explored and difficult sub-problem of WATER, and introducing arbitrary-shape inputs to STR for the first time, provides substantial novelty.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ The experiment design is exceptionally solid, wrapping cross-validations of 3 STR paradigms (CTC/PD/AR) across 4 model architectures, marginal return analyses on synthetic data scaling, ablations over positional encodings and token ranges, comparisons with general and OCR-specific VLMs, cross-lingual validations, and inference efficiency analyses.
Writing Quality: ⭐⭐⭐⭐ The text is structurally clear and well-motivated (tackling both data and architectural bottlenecks in parallel) with detailed appendices (including prompt templates, Chinese validations, and efficiency trade-offs). However, certain experimental evaluations could be further deepened (e.g., quantitative profiling of model robustness under varying noise levels).
Value: ⭐⭐⭐⭐⭐ Pushes WordArt-Bench scores past 90%+ for the first time while open-sourcing both data and code, establishing a strong baseline for the WATER subfield. The insights on the "moderate scale" of synthetic data and the comparative analysis of VLMs vs. specialists provide direct references for the research community.