A Scalable Vector Graphics Latent Space¶

Conference: ECCV 2026
Paper: Official paper page · Paper PDF
Project: SVG Latent Space
Area: Image Generation
Keywords: Scalable Vector Graphics, Path-Level Representations, Transformer Autoencoder, BPE Tokenization, Latent Space

TL;DR¶

SLS compresses each SVG path, including its geometry and style, into a decodable dense vector, preserving strong reconstruction quality while reducing the average captioning input from 2432.02 XML tokens to 15.56 path embeddings; its main contribution is a vector-graphics representation interface rather than a new text-to-SVG generator.

Background & Motivation¶

SVG preserves editable curves, coordinates, and appearance attributes rather than merely storing a smaller image. A graphic can be resized or have individual contours and fills edited without abandoning its symbolic structure. Rasterizing it for CLIP or DINOv2 provides useful visual semantics but no direct route from those features back to SVG code. Feeding the XML to a language model retains the code, yet even a modest graphic can consume a long context. Dedicated latent representations such as DeepSVG address part of this problem, but fixed structural assumptions, control-point restrictions, and limited style coverage constrain their use as general interfaces.

The important choice is therefore the unit of compression. A single vector for a whole image may obscure complex structure, whereas fixed token chunks can separate geometry from the appearance information belonging to the same element. A complete path offers an intermediate unit that can be reconstructed and retrieved independently while remaining composable with other paths. The aim is to make downstream computation depend more on the number of paths than on the length of their XML descriptions, without losing the ability to return to editable vector content.

SLS uses text-based BPE rather than a hand-designed command serialization and learns geometry and appearance through the same input vocabulary. It learns the representation through reconstruction before reusing it for retrieval and captioning, rather than adding a retrieval-specific encoder. Core idea: compress each complete SVG path into a reconstructible dense vector that jointly represents commands, coordinates, and style, then replace a long sequence of SVG tokens with one path embedding in downstream models.

Method¶

Overall Architecture¶

An input SVG is preprocessed into independent paths, each serialized with its style attributes. Each path passes through Unified Path Tokenization and the EOS Reconstruction Bottleneck to produce a 1024-dimensional vector; the Hyperspherical Interface normalizes it for the Downstream Path Interface. To recover SVG content, a normalized vector is rescaled to the decoder's expected magnitude and autoregressively decoded into path tokens, after which the reconstructed paths can be composed into the graphic.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["SVG split into paths"] --> B["Unified Path Tokenization"]
    B --> C["EOS Reconstruction Bottleneck<br/>Encoding and autoregressive decoding"]
    C --> D["Hyperspherical Interface"]
    D -->|"Downstream use"| E["Downstream Path Interface"]
    D -->|"Rescale and decode"| F["Reconstructed SVG paths"]
    E --> G["Path retrieval or image caption"]

Key Designs¶

1. Unified Path Tokenization: represent style and geometry through one input channel

Drawable SVG primitives can be expressed as paths, avoiding separate learned representations for circles, rectangles, and other basic shapes. The paper follows vHector preprocessing, decomposes compound objects into paths, and trains BPE on the resulting corpus. BPE learns recurring textual units instead of assigning every command a predetermined set of coordinate slots. Commands, numerical parameters, fill colors, stroke widths, and opacity therefore enter the model through one vocabulary. Unlike geometry-only encodings, this input representation lets the reconstruction objective constrain appearance as well as shape without assigning color to a separate system.

Path boundaries and BPE boundaries serve different purposes: the former determine what is compressed into one vector, while the latter determine its discrete serialization. Fixed-length chunking is not equivalent to path modeling because a chunk can end inside an unfinished graphical element. The paper explicitly evaluates fixed blocks, providing evidence for complete paths as compression units rather than merely showing that shorter inputs are beneficial. This remains a bounded representation system: paths exceeding 1024 tokens are filtered during training, and the supported preprocessing scope limits practical coverage.

2. EOS Reconstruction Bottleneck: test whether one vector can recover the path

Each sequence receives BOS and EOS markers, learned token embeddings, and sinusoidal positional encodings before entering a Transformer encoder. The final-layer hidden state at EOS summarizes the path; no additional learnable CLS token is prepended. The bottleneck contains one vector, not the complete matrix of token states. Long paths still require token processing during encoding, so the main compression benefit appears when downstream models subsequently reuse the representation.

The decoder injects this vector once as a prefix to a causal Transformer and predicts the original path token by token. Training adds Gaussian noise with standard deviation 1.0 to the latent vector, requiring nearby perturbed representations to retain enough information for reconstruction. Inference uses greedy decoding because the task is to recover a particular path rather than diversify generated drawings. Here, “invertible” should be understood as empirically high-fidelity decodability, not a mathematical bijection or guaranteed character-perfect recovery; greedy neural decoding is not itself a formal syntax validator.

3. Hyperspherical Interface: compare directions downstream and restore scale for decoding

The trained embeddings empirically have nearly constant norms; the model is not trained with an explicit constraint that places every representation on the unit sphere. The authors exploit this behavior by normalizing vectors for downstream operations and restoring the empirical mean norm before reconstruction. The relevant transformations in Section 3.2 are:

\[ z_{\mathrm{norm}}=\frac{z}{\lVert z\rVert_2},\qquad z_{\mathrm{decoder}}=\mu_{\lVert z\rVert}\,z_{\mathrm{norm}}. \]

Here, \(\mu_{\lVert z\rVert}\) is the model's empirical mean embedding norm, not a separately retained original norm for each input. This distinction matters because directly decoding a unit vector would change the magnitude distribution expected by the decoder. Experiments with norm regularization and a learnable scale do not outperform the final configuration, suggesting that explicitly encoding magnitude is not the source of its advantage. The claim that normalization loses no useful information nevertheless depends on norm stability; it is an empirical observation, not a guarantee for arbitrary out-of-domain SVGs.

4. Downstream Path Interface: reuse one representation for retrieval and language conditioning

Retrieval encodes and normalizes the query and database paths, then selects nearest neighbors by cosine similarity. For unit vectors, cosine similarity reduces to a dot product, making the representation suitable for repeated queries over a pre-encoded library. The evaluation does not require every baseline to decode its features: to compare fairly with CLIP and DINOv2, it retrieves the original database paths by their selected indices and measures visual and syntactic similarity. Retrieval quality and the ability to reconstruct SVG from a latent vector are consequently distinct pieces of evidence.

For image captioning, the system retains a sequence of path embeddings rather than averaging the whole image into one vector. A linear projection maps them into a language model's token-embedding dimension, and the projected vectors become visual prefix tokens conditioning caption generation. The projection and language model are trained jointly; the main text does not explicitly establish whether the SLS encoder is updated in this stage, so a definite freezing recipe should not be inferred. This interface removes the language model's need to process verbose XML, not the cost of SLS pretraining, initial path encoding, or autoregressive reconstruction when SVG output is needed.

Loss & Training¶

The objective combines token-level cross-entropy reconstruction with a KL term between target and predicted token distributions. This is not the VAE regularizer that matches a latent posterior to a Gaussian prior. Section 3.2 first associates the target with the ground-truth token and later describes a sharpened, self-generated target, without clearly specifying its construction or the numerical KL weight in the main text. The role of this term is therefore reported without inventing a target distribution, temperature, or coefficient.

Training combines StarVector, HeisenVec, ColorSVG, and SVGX-Core. After filtering, the mixture contains approximately 1.5 million training images, 31,000 validation images, and 16,000 test images, corresponding to 25 million, 500,000, and 250,000 paths. The autoencoder trains from scratch for 480,000 steps with batch size 192, learning rate \(10^{-4}\), 6000 warmup steps, and four NVIDIA A100 GPUs. Its hidden dimension is 1024, with eight attention heads and an MLP ratio of two; encoder and decoder together have 135 million parameters. Captioning uses Qwen3 0.6B, Llama 3.2 1B, and Gemma 2 2B, with learning rate \(10^{-4}\) and batch size 384. The local source contains the complete 18-page main paper and references, but not the supplementary training material mentioned in the text.

Key Experimental Results¶

Main Results¶

The selected results below come from Table 1 and concern reconstruction on the mixed SVG data, read in the context of the held-out splits in Section 4.1. DeepSVG embeds whole images, so the direct baseline comparison is image-level and respects its maximum-command constraint per path. Values retain the paper's displayed scale: the column labeled MSE is used as MSE-similarity, with higher values better, not conventional raw mean squared error.

Method	Image MSE-Sim ↑	Image SSIM ↑	Image LPIPS ↓	Image DINO similarity ↑
DeepSVG, zero-shot	76.03	67.85	53.99	49.86
DeepSVG, retrained on the mixture	76.05	67.85	53.31	57.95
SLS, BPE	90.83	87.26	18.79	80.85

Against retrained DeepSVG, SLS improves SSIM by 19.41 displayed-score points and lowers LPIPS by 34.52. This supports better visual reconstruction, but the comparison does not establish equal compute budgets or equal style-representation capacity. Table 1 also reports path mIoU of 78.92 for SLS; this measures geometric overlap without accounting for stylistic differences.

For captioning with Llama 3.2 1B, Table 3 reports BLEU5 of 46.89 for SLS, 12.31 for XML, 18.77 for DeepSVG, and 54.67 for the CLIP reference. CLIP provides strong captioning features but cannot directly decode them back to SVG, making it a non-decodable reference rather than an alternative with identical capabilities. Table 4 reports average context length falling from 2432.02 to 15.56, a factor of 156.29; average training computation falls from 18.06 to 0.108 TFLOPs, with a reported ratio of 167.22. These are captioning context and computation statistics, not measured end-to-end wall-clock speedups or a total-cost comparison including autoencoder pretraining.

Ablation Study¶

The following selection from Table 2 uses reconstruction SSIM, LPIPS, and path mIoU to compare appearance and geometry. Token-overlap metrics are omitted here because their interpretation is less straightforward across different tokenizers.

Configuration	Image SSIM ↑	Image LPIPS ↓	Path mIoU ↑
SLS, full model	87.26	18.79	78.92
Projection dimension 512	71.44	51.47	26.16
CLS pooling	86.26	20.02	78.87
DeepSVG-like tokenizer	67.08	47.73	40.58
IconShop-like tokenizer	67.61	46.01	21.86
Fixed 512-token blocks	83.73	23.99	64.31
Fixed 1024-token blocks	84.13	24.06	61.33

Key Findings¶

Tokenization and capacity matter more than pooling location: CLS reaches SSIM 86.26, close to the full model, whereas DeepSVG-like tokenization reaches 67.08. The main gain should not be attributed solely to EOS.
Complete paths outperform mechanical chunking: path mIoU is 64.31 for 512-token blocks versus 78.92 for SLS, supporting the preservation of coherent graphical elements.
In Table 5 retrieval, SLS obtains MSE-Sim 89.95 and BLEU5 20.91, versus 81.45 and 12.89 for DeepSVG. DINOv2 reaches higher MSE-Sim, 98.86, but BLEU5 10.77, illustrating different preferences for visual similarity and code structure.
The retrieval split is inconsistently named: Table 5 says test set, while Section 4.4 describes 2500 queries and 266,000 candidates sampled from validation data. The concrete setup is used here rather than claiming an additional independent test set.

Highlights & Insights¶

The path bottleneck turns geometry and style into one reusable object, not merely a summary of XML. Reconstruction, retrieval, and captioning test complementary properties of that object.
Unit-norm downstream operations and original-scale decoding have separate interfaces. This is a useful engineering pattern, provided embedding-norm stability is checked first.
The ablations suggest that choosing the right compression unit and vocabulary matters more than swapping a pooling token. Representation boundaries are an important modeling decision for structured graphical documents.

Limitations & Future Work¶

Explicit scope restrictions include length filtering, command limits in DeepSVG comparisons, and automatically generated caption references. Results should not be directly extrapolated to unrestricted complex SVGs or human semantic judgments.
Table 6 reports degradation on out-of-domain UniSVG: SSIM drops from 87.26 in-domain to 78.44, while LPIPS rises from 18.79 to 30.18. Transfer is demonstrated, but distribution shift remains relevant.
Reader assessment: single-path reconstruction does not guarantee explicit modeling of occlusion, layout, or editing dependencies across paths. Image-level applications support composability without proving global structural reliability for arbitrary scenes.
Reader assessment: strict invertibility, universal syntax correctness, and full-pipeline acceleration are stronger claims than the available evidence. Exact recovery rates, invalid-SVG rates, and timing that includes encoding would clarify these boundaries.
Reproduction details remain incomplete: the KL target and coefficient are insufficiently specified, and supplementary material is unavailable locally. No missing implementation choices are treated as verified facts here.

Versus DeepSVG: its hierarchical geometry representation emphasizes structured parameters; SLS uses path-text BPE to include style and reduce fixed-parameterization restrictions. The difference concerns representation assumptions, not simply a stronger encoder.
Versus vHector and IconShop-style token modeling: these approaches model SVG sequences directly, whereas SLS inserts a decodable path bottleneck before downstream processing. Generating SVG and learning a reusable latent interface are related but distinct tasks.
Versus CLIP and DINOv2: raster features remain competitive for semantics and appearance-based retrieval. SLS distinguishes itself through a route back to vector content, not by replacing them on every visual metric.

Rating¶

Novelty: 4/5 — The path-level, style-aware, decodable combination is useful, although BPE and the underlying autoencoder are established components.
Experimental Thoroughness: 4/5 — Reconstruction, captioning, retrieval, and informative ablations are covered; strict invertibility and full-pipeline cost remain less established.
Writing Quality: 3/5 — The main argument is clear, but the KL definition and retrieval-split terminology need clarification.
Value: 4/5 — A promising interface for vector-graphics modeling, with complex SVG coverage and practical editing still requiring further evidence.