Text-Conditioned Background Generation for Editable Multi-Layer Documents¶

Conference: ECCV 2026
arXiv: 2512.17151
Code: Unreleased
Area: Diffusion Models / Image Generation
Keywords: Document Background Generation, Latent-Space Masking, Readability Optimization, Multi-Page Consistency, Layered Editing

TL;DR¶

By treating a multi-page document as a separable "text / image / background" three-layer structure, this work proposes a training-free framework to regenerate only the background layer. By applying soft attenuation to the foreground regions in the diffusion latent space, the text is protected. A contrast-driven algorithm automatically adds semi-transparent backing cards to the text to guarantee readability, while a recursive summarization-instruction memory maintains thematic style across the entire document. Ultimately, this approach significantly outperforms BAGEL and GPT-5 in readability (WCAG 99.75% / OCR 0.97) and user preference (83.57%).

Background & Motivation¶

Diffusion models have matured in text-to-image generation and style transfer, but they are almost entirely optimized for "generating an isolated image." Once applied to documents where text and images are interwoven, such as academic reports, lecture handouts, or presentation slides, they suffer from poor adaptation. Document editing truly cares about three things: attractive backgrounds, readable text, and consistent style across multiple pages—precisely the three aspects that standard diffusion pipelines struggle with. In existing systems, background generation is often treated trivially, resulting in textures obscuring the main text, inconsistent styles across pages, and even altered resolutions and layouts. The comparison diagram at the beginning of the paper directly demonstrates this failure: general-purpose diffusion models erase titles and illustrations, tamper with semantic content, change resolution, and "redesign" a readable document into an unreadable image.

Behind this lies a fundamental misalignment of design goals. While poster generation works (e.g., POSTA, CreatiPoster) can produce beautiful multi-layer editable layouts, their input consists of user-provided text and assets, allowing free restructuring. This is essentially "assembling a poster from scratch." Conversely, general-purpose multimodal models like GPT-4o / GPT-5, when used to change document backgrounds, rewrite the text content itself, directly violating the "do not alter the foreground" editing constraint. The true scenario for document background editing is: a pre-formatted multi-page document is provided, where text, illustrations, and layout are all immutable hard constraints, and the user only wishes to change the background. There are almost no dedicated academic solutions for editing under such constraints; the vast majority of diffusion research competes to generate "sharper, finer, and more content," whereas document editing demands the exact opposite: to maintain text fidelity, the model sometimes needs to do less, not more.

The key insight of this paper follows this philosophy of "doing less": instead of redrawing the document as a flat image, it is treated as a multi-layer structure where text, images, and backgrounds can be independently retained or regenerated. Armed with this, the system only operates on the background layer and actively "attenuates" updates in foreground regions during the generation process to protect the content. Core Idea: Document background generation is modeled as a smooth attenuation field in the diffusion latent space—gently suppressing diffusion updates in text regions similar to a physical soft potential barrier. Combined with a WCAG contrast-driven automatic backing card algorithm and recursive memory for multi-page instructions, a completely training-free framework is achieved to implement readable, consistent, and interactively editable document background generation.

Method¶

Overall Architecture¶

The system takes a structured multi-page document (PDF or slides) as input and outputs a finished document where only the background layer is replaced, leaving the foreground text and illustrations untouched. The entire pipeline is a serial multi-stage process: first, foreground region extraction is performed to parse text lines, paragraph boxes, and illustration areas from each page, yielding the full-page text \(T_i\), the set of precise text bounding boxes \(\mathcal{L}_i\), and representative bounding boxes \(\mathcal{B}_i\) used for masking. Next, the summarization model compresses the verbose page text into a semantic label \(s_i\) of no more than five words. The instruction model then converts the label (optionally overlaid with a user prompt) into a background design instruction \(u_i\), where style consistency across pages is maintained by the Recursive Narrative Bank (RNB). Then, the text-to-image diffusion model synthesizes the background accordingly, constrained by two modules: Latent-Space Masking (LM) uses \(\mathcal{B}_i\) to suppress updates in the foreground regions, and Automatic Readability Optimization (ARO) places semi-transparent rounded backing cards beneath all text boxes \(\mathcal{L}_i\) to boost contrast to WCAG standards. Finally, the background is composited with the original foreground to produce a consistent and readable multi-page document. The entire system requires no training; LM is directly embedded in the BAGEL base, and the LLM for summarization/instruction acts as a replaceable component.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Structured Document<br/>PDF / Slides"] --> B["Foreground Region Extraction<br/>Text Boxes + Representative Boxes + Illustration Areas"]
    B --> C["Summarization Model<br/>Page Text Compressed to ≤5-Word Label"]
    C --> D["Instruction Model + RNB<br/>Label → Background Instruction, Cross-Page Memory"]
    D --> E["Diffusion Background Generation"]
    B -->|Representative Boxes| E
    E --> F["Latent-Space Masking<br/>Soft Attenuation of Foreground Updates"]
    F --> G["Automatic Readability Optimization<br/>Semi-transparent Backing Cards Up to WCAG"]
    B -->|Precise Text Boxes| G
    G --> H["Composition with Foreground<br/>Consistent & Readable Multi-page Document"]

Key Designs¶

1. Foreground Region Extraction: Drawing Precise Boundaries for "Immutable Areas"

Any text-protecting operation must first locate where the text is, so the first step decomposes each page into usable geometric information. The system detects each line of text and its bounding box, then merges adjacent lines into paragraphs according to "consistent left margin + bounded vertical gap". Paragraph boxes are the union of all their line boxes. To prevent merging across large images, detected illustration areas vertically cut the page into three groups: top, side, and bottom. Paragraphs within a group are merged into column-like regions only when their horizontal overlap ratio \(\mathrm{overlap}_x\) exceeds a threshold and the vertical gap is within tolerance. Finally, an NMS-like suppression eliminates redundancy: if a candidate region \(p\) is substantially enclosed by a larger region \(q\) (containment ratio or IoU exceeds a threshold), it is discarded. This step simultaneously outputs two sets of boxes: precise text boxes \(\mathcal{L}_i\) (passed to ARO, requiring pixel-level accuracy to guarantee contrast) and representative boxes \(\mathcal{B}_i\) (passed to Latent-Space Masking, which only needs a rough boundary around text areas to suppress updates rather than pixel-by-pixel classification). The differing precision requirements of these two sets of boxes stem from the distinct needs of the two downstream modules—a highly practical distinction.

2. Foreground-Aware Latent-Space Masking: Softly Suppressing Text Regions with a Potential Barrier rather than Hard Masking

This is the core of the paper. If prominent background objects are allowed to generate freely, they can easily bleed into the text, rendering the document unreadable. The most naive approach is to use a binary mask to completely wipe out the foreground regions, but hard removal creates sharp boundaries and visually jarring disjointedness. The approach in this paper starts from the idea of "deliberately weakening background generation in foreground regions," applying a smooth attenuation field in the latent space—analogous to a diffusion barrier in physics or a weighting function in numerical optimization—allowing the background to evolve naturally around protected areas without leaving artifacts at the boundaries. Specifically, a mask is defined on a 2D latent grid: within a window covering a fraction \(\rho\) of the center, the attenuation factor is set to \(\lambda\in(0,1)\), and outside the window, it is 1. Crucially, the mask is only activated during the later time steps of the diffusion schedule (implemented starting from 0.29 with an intensity of 0.2). Applying the mask \(\mathbf{m}\) to the model's predicted velocity yields the effective velocity:

\[v_t' = \mathbf{m}\odot v_t^{\mathrm{raw}} + (1-\mathbf{m})\odot \mathrm{stopgrad}(v_t^{\mathrm{raw}})\]

which is then updated as \(x_{t-\Delta t}=x_t - v_t'\,\Delta t\). Intuitively, the velocity is attenuated (multiplied by \(\lambda\)) in text regions, and a gradient cut is applied to the suppressed parts, making diffusion updates weaker and more stable there while the background regions evolve richly as usual. Compared to common diffusion inpainting, this work uses time-gated + attenuation-based suppression, deliberately "suppressing foreground updates" rather than "amplifying" them, thus preserving text while allowing the background to grow naturally around it. The authors also point out that this is a soft suppression rather than a strict ban, so residual artifacts may still appear along text boundaries under dense, irregular layouts—representing a trade-off of "graceful degradation" rather than "foreground destruction."

3. Automatic Readability Optimization ARO: Computing the Perfect Minimum Opacity to Make Backing Cards Readable but Unobtrusive

Latent-space masking alone cannot guarantee that every piece of text is sufficiently clear, especially when a bright background yields insufficient contrast. Prior systems solved this by uniformly placing opaque color blocks behind text—simple but aesthetically displeasing. ARO's approach is to place a semi-transparent rounded backing card behind each text box and automatically solve for the "minimum opacity that barely satisfies the WCAG 2.2 contrast standard," ensuring readability while allowing the background to show through as much as possible to preserve beauty. First, sRGB is converted to linear RGB to compute relative luminance \(L\), and the WCAG contrast formula is used: \(CR(L_1,L_2)=\frac{\max+0.05}{\min+0.05}\). Given the backing color luminance \(L_o\) and background pixel luminance \(L_{\mathrm{bg}}\), the blended luminance is \(L_{\mathrm{blend}}(\alpha)=\alpha L_o+(1-\alpha)L_{\mathrm{bg}}\). Then, the minimum \(\alpha\) is searched under a coverage fraction constraint:

\[\alpha^{*}=\min\Big\{\alpha \;\Big|\; \tfrac{1}{N}\sum_{i=1}^{N}\mathbf{1}\big[CR(L_{\mathrm{blend}}^{(i)}(\alpha),L_t)\ge\tau\big]\ge\rho\Big\}\]

This requires at least a fraction \(\rho\) of pixels in the text region to meet the target contrast \(\tau\) (experimentally \(\tau\)=7.0, coverage 0.98), capped by a lower bound to yield the final \(\alpha\). Thus, the opacity of each backing card is calculated adaptively based on the local background luminance beneath it, rather than being a one-size-fits-all fixed value. This allows it to conform to the background, satisfying accessibility requirements without appearing jarring. Since it is a pure image-space post-processing step, it adds almost no GPU memory and takes only about 1.26 seconds.

4. LLM-Based Multi-Page Consistency: Using a Recursive Narrative Bank to "Remember" Style and Prevent Page-to-Page Drift

Unlike videos or dialogues, document backgrounds must be visually consistent across pages while fitting the local content of each individual page; if instructions are generated independently and statelessly for each page, spatial styles will inevitably diverge. This paper solves this in two steps: first, the summarization model \(f_{\text{sum}}\) compresses the verbose, noisy text of each page into a semantic label \(s_i\) under five words, capturing the page's main visual theme (e.g., "fairness in AI ethics," "molecular structures in life sciences"). This reduces noise and drives automatic generation "by content" without requiring users to write instructions page-by-page. Second, the instruction model \(f_{\text{inst}}\) generates a design instruction based on \(s_i\) (optionally combined with user prompt \(p\)) and history. The critical consistency is achieved by porting the "Recursive Narrative Bank (RNB)" to the document domain—maintaining a memory window of size \(N\), \(H_{i}=\{u_i^{(1)},\dots,u_i^{(N)}\}\), storing previous page instructions. The next instruction \(u_i=f_{\text{inst}}(s_i,p,H_{i-1})\) is conditioned on this memory. Consequently, stylistic cues like tones and textures accumulate and pass along the document, achieving global style reuse while allowing local variations guided by \(s_i\). The LLM in this step is a replaceable component; removing it (w/o MPC) only impacts consistency (CLIP MP drops by 0.06, LLM Voting drops by 0.07), while readability metrics remain unchanged, demonstrating that multi-page consistency and readability are decoupled lines.

An Example: Changing the Background of a Three-Page AI Ethics Lecture Handout¶

Take a three-page AI ethics document as an example. First, for the dense text on page one, foreground extraction detects all text lines, merges them into paragraph boxes, splits them into column regions around illustrations, and removes redundancy via NMS to obtain precise text boxes \(\mathcal{L}_1\) and representative boxes \(\mathcal{B}_1\). The summarization model compresses the page into a sub-five-word label \(s_1\) like "AI Fairness and Bias," and the instruction model (with an empty memory for this first round) generates a background design instruction such as "soft abstract geometric background." During diffusion generation, Latent-Space Masking uses \(\mathcal{B}_1\) to attenuate velocity in text regions during later time steps, allowing background textures to unfold only around the text. Once generated, ARO calculates the minimum \(\alpha\) satisfying the contrast requirement for each text box and places a semi-transparent rounded backing card. For the second page, the instruction model is conditioned on memory \(H_1\) containing the first page's instruction, so the second page inherits the same color palette and texture style, adjusting only for the local theme; the third page follows the same logic. Across the three pages, the style remains consistent, the text remains perfectly legible, and the illustrations remain untouched. If the user later requests a "slightly lighter background," the system only needs to recompute \(u_i\) and regenerate the background, leaving the text and illustrations completely untouched. Only when the text content itself changes does the system recalculate layout boxes and rerun the entire pipeline, including masking and ARO.

Key Experimental Results¶

Main Results¶

On a self-built academic document + slide benchmark (each document is exactly three pages, containing long text/bullet points/at least one illustration), the proposed method was compared against BAGEL and GPT-5 across nine metrics covering design quality, readability, and multi-page consistency.

Method	Layout	WCAG (%)	OCR	CLIP Consistency	LLM Voting
BAGEL	4.10	66.98	0.55	0.557	4.23
GPT-5	3.88	55.02	0.52	0.687	4.02
Ours (Prompt+Text)	4.20	99.75	0.97	0.696	4.33
Ours (Prompt only)	4.36	99.38	0.96	0.626	4.51

The most striking result is readability: Ours boosts WCAG contrast coverage to a near-perfect 99.75% and OCR character accuracy to 0.97, whereas the two baselines hover around 0.5x / 5x-6x%—not a marginal gain but an order-of-magnitude leap. In the user study, 30 participants awarded Ours the highest scores across four design dimensions (4.669–4.80 vs. baselines' 1.17–1.65). In overall preference voting, 83.57% chose Ours, while GPT-5 secured 15.24% and BAGEL received only 1.43%.

Ablation Study¶

Under the default Prompt+Text mode, three modules (LM, ARO, MPC) were iteratively removed:

Configuration	WCAG (%)	OCR	CLIP Consistency	LLM Voting	Description
Full model	99.75	0.97	0.696	4.33	Complete model
w/o LM	99.67	0.91	0.691	4.49	Without Latent-Space Masking; OCR drops by 0.06 as background encroaches on text
w/o ARO	97.35	0.90	0.689	4.32	Without Readability Optimization; WCAG drops to 97.35%
w/o MPC	99.69	0.96	0.642	4.26	Without multi-page consistency; CLIP Consistency drops by 0.06

Key Findings¶

LM and ARO form the dual pillars of readability: removing LM allows background textures to encroach on text regions, causing OCR to fall from 0.97 to 0.91; removing ARO causes WCAG to drop more drastically from 99.75% to 97.35%—the two are indispensable, with one preventing encroachment and the other ensuring contrast.
A fascinating anomaly: removing LM slightly increases the CLIP Prompt Score (0.25 vs 0.24), implying that permitting unrestricted background updates can sometimes enhance text-to-image alignment, albeit at the cost of text readability—directly validating the core premise that "document editing requires doing less rather than more."
MPC solely manages consistency and does not touch legibility: removing it leaves WCAG/OCR virtually unchanged, but the cross-page CLIP Consistency drops from 0.70 to 0.64, proving that multi-page coherence and text fidelity are decoupled.
Exceptionally low overhead: ARO offsets generation time by only ~1.26 seconds (0.5%), MPC adds 7.30 seconds (~3%), and GPU memory remains constant throughout (31.35GB). This efficiency arises because the three modules either operate directly in the latent space or function as pure post-processing, avoiding any additional model forward passes.

Highlights & Insights¶

Modeling "text protection" as a soft attenuation field in the latent space instead of binary masking is immensely elegant: drawing on the intuition of a physical soft potential barrier, using stop-gradient + time-gating makes the diffusion updates in text regions weaker and stabler without leaving hard aesthetic boundary defects. This trick can be migrated to any diffusion editing task where "local preservation is desired without the disjointed feel of hard masking."
ARO directly integrates the WCAG accessibility standard into the generation pipeline. By "searching for the minimum opacity," it transforms readability from an "afterthought patch" to an "exact formula calculation," allowing text backing cards to adapt to local background brightness. This paradigm of "mathematically formalizing external compliance standards into optimization objectives" is highly exemplary.
The dual-precision design—delivering precise boxes to ARO and representative boxes to LM—is extremely pragmatic. Recognizing that the two downstream modules have different precision requirements bypasses the need for costly pixel-level detection for masking, saving computation while ensuring robustness.
Being entirely training-free, plug-and-play, with near-zero extra GPU memory and 3.5% time overhead, it pushes document readability close to saturation. The engineering viability is exceptionally strong. The authors explicitly state this fills the "dense document readability" gap for already deployed systems like Canva and Microsoft Designer.

Limitations & Future Work¶

The authors acknowledge that latent-space masking is only a soft suppression rather than a strict ban. Consequently, text boundaries can still exhibit residual artifacts under dense or irregular layouts, and ARO might fail when overlapping irregular transparent images.
Foreground extraction relies on PyMuPDF/OpenCV; dense mathematical formulas or nested tables can cause missed regions. Although this manifests as a graceful degradation of readability rather than foreground corruption, it remains a fragile link.
While the Recursive Narrative Bank operates with O(1) memory per page regardless of document length, memory saturation occurs once documents exceed ten pages; validating performance on very long documents remains future work. The benchmark was also intentionally fixed to three pages to facilitate human evaluation.
No comparison against an SD-Inpainting baseline was reported. The authors justify this by pointing out the unfairness of configurations under document resolutions (A4, 16:9)—downsampling destroys the very text legibility being evaluated, while tiling introduces seams, a caveat worth noting.
Finer-grained control (e.g., matching section-specific themes, adaptive color schemes) remains an open problem.

vs BAGEL: BAGEL excels at poster-style interactive editing, keeping the rest of the layout stable during local edits. However, when faced with dense multi-page text documents, its background synthesis compromises content fidelity and fails to maintain cross-page consistency. The proposed method directly builds on the BAGEL base, bridging the gap in "readability + multi-page consistency."
vs POSTA / CreatiPoster: These fall under poster generation, taking user-supplied text/assets and freely rearranging the structure—essentially assembling posters from scratch. In contrast, this paper tackles constrained editing, where foreground text, illustrations, and layouts are completely immutable, and only the background layer is modified. The task definitions are fundamentally different.
vs GPT-4o / GPT-5 / Nano Banana 2: When tasked with editing document backgrounds, general-purpose multimodal models tend to rewrite text content or treat input as "scanned photos" to redraw and warp text, violating the foreground preservation constraint. Thus, they are unsuitable as editors. This failure stems from task alignment, not competence.
vs TextDiffuser / SAWNA: The former uses spatial conditioning to generate legible text, while the latter preserves empty negative space, but both assume controllable text rendering or vacant spaces. They do not protect existing foreground content in complex layouts. This paper squarely addresses the gap of "pre-formatted layouts with immutable foregrounds."

Rating¶

Novelty: ⭐⭐⭐⭐ Defining document background editing as layered constrained editing and solving it with latent-space soft attenuation, WCAG readability optimization, and recursive memory is quite novel. However, the individual components are mostly clever combinations of existing ideas.
Experimental Thoroughness: ⭐⭐⭐⭐ Covered nine metrics, ablation studies, a 30-person user trial, and computational overhead analysis. The readability improvements are highly substantial. Points were subtracted as the benchmark was a self-built 3-page synthetic dataset and missed baselines like SD-Inpainting.
Writing Quality: ⭐⭐⭐⭐ Clear motivation; the thesis "doing less rather than more" runs consistently throughout the text. Mathematical formulations and discussions are robust.
Value: ⭐⭐⭐⭐ Being training-free, plug-and-play, and requiring near-zero extra overhead to push document readability close to saturation makes it of direct, practical value for deployed automated design tools.