Text-based Tactile Graphics Generation for the Visually Impaired

Conference: ECCV 2026
Paper: CVF / Official Poster
Code: https://ruihangao.github.io/Text2TactileGraphics/
Area: Image Generation
Keywords: Tactile Graphics, Tactile Texture Synthesis, Braille Generation, 3D Printing Fabrication, Accessibility

TL;DR

This paper introduces the first end-to-end generative graphics pipeline that produces fabrication-ready 2.5D tactile graphics directly from natural language prompts, jointly synthesizing global base geometry, fine-grained tileable tactile textures, and standard-compliant braille for direct SLA 3D printing.

Background & Motivation

Tactile graphics—incorporating raised contours, textured bas-reliefs, and braille annotations—serve as an indispensable physical medium for more than 290 million blind and low-vision (BLV) individuals worldwide to perceive diagrams, spatial maps, and non-textual educational concepts. Traditionally, however, tactile graphic production heavily relies on expert manual curation, swell paper micro-capsule heating, and thermoform embossing. This manual design workflow is labor-intensive, slow, and expensive, severely restricting tactile assets to standardized elementary teaching diagrams while leaving user-driven, customized tactile graphics largely inaccessible.

Although recent generative models have demonstrated remarkable visual realism across 2D image, video, and 3D asset generation, existing pipelines remain almost exclusively optimized for screen-based visual perception. Applying contemporary 3D generative models directly to physical tactile graphics fails fundamentally: standard text-to-3D models focus entirely on coarse, global object hulls while ignoring the micro- and millimeter-scale surface textures discernible by human fingertips; the resulting meshes suffer from severe fabrication defects such as non-manifold boundaries, disconnected floating artifacts, and unstable open bases; and off-the-shelf depth predictors do not meet the sub-millimeter flatness tolerances demanded by touchable braille.

To bridge the gap between data scarcity, visual-haptic modality mismatch, and physical fabrication constraints, the authors decouple tactile graphics into three complementary physical layers: a stable macro base geometry, high-frequency touchable surface textures, and standard-compliant semantic braille. Core idea: integrate template-conditioned relief synthesis with high-frequency tileable normal diffusion, rigid base flattening, and standard braille placement to generate watertight, fabrication-ready 2.5D tactile reliefs optimized for SLA 3D printing.

Method

Overall Architecture

The framework executes an end-to-end fabrication-aware pipeline: given natural language prompts describing an object and its desired tactile materials, the system first synthesizes a template-constrained relief image and lifts it into a 2.5D base mesh using metric monocular depth and surface normal estimation; concurrently, a specialized text-to-tactile-texture diffusion model synthesizes high-resolution normal maps from text or vision-based tactile sensors, which are seamlessly tiled across large areas via Fourier high-pass filtering and intra-tile diffusion inpainting; next, semantic parts identified by SAM3 are displaced along surface normals using the integrated displacement field; finally, a rigid base flattening algorithm projects base vertices onto a strictly horizontal plane, places standard-compliant braille labels onto cleared regions, and closes the mesh into a watertight manifold ready for resin 3D printing on a Formlabs Form 4 SLA printer.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Text Prompt Input<br/>Object description and tactile materials"] --> B["Template-Guided Base Geometry Generation<br/>Plate-constrained synthesis and monocular lifting"]
    A --> C["Tactile Texture Synthesis & Tiling<br/>Specialized texture tuning and intra-tile inpainting"]
    B --> D["Part-Level Displacement Map Integration<br/>SAM3 mask guided normal displacement"]
    C --> D
    D --> E["Fabrication-Aware Base Flattening & Braille<br/>Rigid zero-tilt projection and standard braille"]
    E --> F["SLA 3D Printed Physical Tactile Graphic"]

Key Designs

1. Template-Guided Base Geometry Generation: Eliminating Scale Ambiguity and Global Tilt in Monocular Lifting Unconstrained text-to-image synthesis models produce free-floating 3D objects with arbitrary viewing angles and drifting ground planes, which hinders tabletop stability during two-handed tactile exploration. The authors reformulate base geometry synthesis as an in-context image editing task conditioned on a fixed marble base-plate template \(I_{\mathrm{bp}}\) (using Qwen-Image-Edit). This forces the generated object to rest firmly on a flat, standardized support plate. From the synthesized relief image \(I \in \mathbb{R}^{H \times W \times 3}\), metric depth and surface normals are estimated via MoGE v2 and converted into a 2.5D bas-relief mesh by mapping depth values to height offsets. Conditioning on the plate template stabilizes the average plate height at standard thickness and achieves a 100% plate detection rate, completely preventing plate dropouts and severe warping.

2. Tactile Normal Texture Fine-Tuning and Intra-Tile Diffusion Tiling: Seamless High-Frequency Detail Expansion Standard text-to-image models are inherently object-centric (for example, prompting "avocado skin texture" typically synthesizes a whole spherical fruit rather than a uniform material pattern) and tend to suppress cross-semantic materials (such as "avocado skin on a dolphin"). The authors fine-tune an image diffusion model on a curated texture dataset containing vision-based GelSight tactile sensor captures and synthetic high-frequency patterns, outputting high-resolution normal maps. To tile these normal patches seamlessly over large 3D surfaces without boundary seams, the patch is center-cropped from \(1024^2\) to \(512^2\), and each channel is processed through a Gaussian high-pass filter in the frequency domain: $\(G_{\mathrm{hp}}(u,v) = 1 - \exp\left(-\frac{d(u,v)^2}{2\sigma_f^2}\right)\)$ The flat-normal DC baseline is restored by adding a constant offset to the \(z\)-channel, followed by per-pixel vector renormalization: $\(N^{\mathrm{hp}}(p) = \frac{(\mathrm{HP}(N_x),\, \mathrm{HP}(N_y),\, \mathrm{HP}(N_z) + 1)}{\|(\mathrm{HP}(N_x),\, \mathrm{HP}(N_y),\, \mathrm{HP}(N_z) + 1)\|}\)$ The patch is then divided into four quadrants and point-symmetrically rearranged (Intra-tile arrangement), bringing all exterior seams to the interior center. SDEdit is applied on the seam mask with initial noisy latent \(z_{t_0} = (1 - t_0) \cdot E(I_{\mathrm{swap}}) + t_0 \cdot \epsilon\) at strength \(t_0 = 0.9\). Denoising over 10 Euler steps resolves seam discontinuities in 3.18 seconds—more than three times faster than Tiled Diffusion while eliminating hallucinated semantic distortions.

3. Part-Level Displacement Map Integration: Decoupled Control over Material and Geometry The seamlessly tiled normal map is integrated into a scalar displacement field \(D_{\mathrm{disp}} \in \mathbb{R}^{H_t \times W_t}\). To apply distinct tactile textures onto designated object components, SAM3 provides zero-shot segmentation masks from text queries (with optional interactive click refinement). For any mesh vertex \(v\) located within the masked target region, its 3D position is modulated along the local surface normal \(n\): $\(v' = v + \lambda D_{\mathrm{disp}}(u) n\)$ where \(u\) is the vertex UV texture coordinate and \(\lambda\) controls physical displacement magnitude. This mathematical formulation strictly decouples tactile surface micro-relief from the underlying macroscopic anatomy, allowing fine tactile adjustment without distorting global geometry.

4. Fabrication-Aware Base Flattening and Standard Braille Placement: Ensuring Tactile Legibility and Watertight Manifold Topology Braille reading is sensitive to sub-millimeter surface height discrepancies, where even minor plate warping can impair dot recognition. Because monocular lifting inevitably introduces low-frequency surface tilt across the base, the authors introduce an exact base-flattening step. Base plate vertices \(V_{\mathrm{base}}\) are identified via foreground/background segmentation, and their average elevation is computed as \(h_{\mathrm{base}} = \frac{1}{|V_{\mathrm{base}}|} \sum_{v \in V_{\mathrm{base}}} v_z\). Each base vertex is projected onto the horizontal plane \(v' = (x, y, h_{\mathrm{base}})\), enforcing zero height variation and preserving mesh triangulation. Hemispherical braille dots adhering strictly to US and international accessibility standards are generated on designated flat plate margins free from geometric relief. Finally, bottom boundary contours are extruded downward and capped, producing a watertight manifold STL mesh ready for direct 3D printing.

A Worked Example

Generating "A lamp. The shade has embossed flower texture and the base has woven texture": 1. Input prompt is fed into Qwen-Image-Edit with the marble plate template, synthesizing a 2.5D bas-relief image of the lamp; MoGE v2 estimates metric depth and lifts the image into a 2.5D mesh; 2. Prompts "embossed flower" and "woven texture" are fed into the fine-tuned texture diffusion model, generating \(1024^2\) normal maps; each map is cropped to \(512^2\), filtered via FFT high-pass, quadrant-swapped, and inpainted via 10-step SDEdit; 3. SAM3 segments the "lampshade" and "lamp base" masks; the flower displacement field is applied to the shade and the woven displacement field to the base along local surface normals; 4. Base plate vertices are detected and rigidly flattened to \(h_{\mathrm{base}}\); the word "lamp" is translated to braille dots and placed onto the cleared bottom margin; 5. The mesh base is closed, exported to Formlabs Form 4 SLA resin printing, and fabricated into a physical tactile plate.

Key Experimental Results

Main Results

The authors fabricated 48 physical tactile artifacts across 16 diverse prompts and conducted double-blind in-person perceptual studies at the Library of Accessible Media for Pennsylvanians (LAMP). Fifteen participants—8 blind and low-vision (BLV) individuals and 7 blindfolded sighted participants (BSP)—evaluated tactile quality on a 5-point Likert scale against Hunyuan3D 2.1 and a Naive baseline (monocular lifting without dedicated texture/braille):

User Population	Metric	Ours	Hunyuan3D 2.1	Naive Baseline	Statistical Significance
Blind & Low-Vision (BLV)	Tactile Match Rating (1-5)	3.93 ± 1.32	3.62 ± 1.28	3.62 ± 1.20	\(p = 0.006\) (vs Hunyuan), \(d = 0.16\)
Blindfolded Sighted (BSP)	Tactile Match Rating (1-5)	3.35 ± 1.39	2.97 ± 1.31	3.02 ± 1.24	\(p = 0.008\) (vs Naive), \(d = 0.16\)
BLV Recognition Accuracy	Object ID w/o Braille	~20%	N/A (Failed base)	~20%	Consistent with human haptic limits
BLV Recognition Accuracy	Object ID w/ Braille	Nearly 100%	0% (No braille)	0% (No braille)	Confirms full tactile comprehension

Ablation Study

Ablation evaluations were conducted across base plate stability, texture fine-tuning, and tileability metrics across 50 benchmark prompts:

Module Under Test	Configuration	Metric 1	Metric 2	Latency / Notes
Base Geometry	QIE-4 (w/o Template)	Plate CLIP: 34.82	Mean \(\bar{\mu}\): 0.181, SD \(\sigma_{\mathrm{total}}\): 0.131	Unstable base height, sunken contours
	QIE-4 (w/ Template, Ours)	Plate CLIP: 36.64	Mean \(\bar{\mu}\): 0.337, SD \(\sigma_{\mathrm{total}}\): 0.122	100% plate detection rate, stable base
	Nano Banana Pro (w/o Template)	Plate CLIP: 36.83	Mean \(\bar{\mu}\): 0.467, SD \(\sigma_{\mathrm{total}}\): 0.188	Plate missing in 6 of 50 samples
	Nano Banana Pro (w/ Template)	Plate CLIP: 37.66	Mean \(\bar{\mu}\): 0.365, SD \(\sigma_{\mathrm{total}}\): 0.137	Plate geometry fully recovered
Texture Synthesis	Qwen-Image-4step (Base)	LPIPS Self-Sim↓: 0.496	HF Ratio↑: 0.305	Focuses on global object, lacks micro-patterns
	Ours-4step (Fine-tuned)	LPIPS Self-Sim↓: 0.412	HF Ratio↑: 0.396	High pattern consistency and tactile relief
Normal Tiling	Direct Crop	Border Discont.↓: 26.10	Grad Discont.↓: 52.82	Severe tactile seams at patch boundaries
	Crop + High-Pass Filter	Border Discont.↓: 22.84	Grad Discont.↓: 46.53	Removes DC tilt, visible boundary jumps
	Tiled Diffusion	Border Discont.↓: 4.43	Grad Discont.↓: 9.76	Runtime 10.9s; distorts normal geometry
	Ours-Intra (Quadrant SDEdit)	Border Discont.↓: 1.61	Grad Discont.↓: 3.19	Runtime 3.18s, superior seam continuity

Key Findings

• Fine-grained tactile surface texture is critical for tactile perception: In in-person haptic evaluations, participants expressed a statistically significant preference for Ours over Hunyuan3D and Naive baselines (\(p < 0.01\)), confirming that touch easily discriminates micro-textures like woven fabric, coral pores, and reptile leather.
• Braille provides an indispensable semantic anchor: Passive tactile exploration alone yielded only ~20% object recognition, whereas standard-compliant braille boosted comprehension to nearly 100%, proving that textual annotations are vital for non-visual accessibility.
• Intra-tile rearrangement with SDEdit outperforms Tiled Diffusion: In normal map tiling, Intra-tile inpainting reduced edge discontinuity from 4.43 to 1.61 and gradient discontinuity from 9.76 to 3.19, while cutting computation time by over 70% (3.18s vs 10.9s) and eliminating unwanted semantic hallucinations.

Highlights & Insights

• Formulating fabrication constraints as generative priors: Rather than chasing screen-only photometric rendering, this work enforces 3D printing manifoldness, horizontal base stability, and standard braille sizing as geometric priors, establishing a fabrication-aware generative paradigm.
• Intra-tile quadrant swapping with lightweight SDEdit: Point-symmetrically swapping four quadrants exposes boundary seams directly to the image interior, enabling shallow diffusion inpainting to produce seamless tileable normal maps without large \(3 \times 3\) grid memory overheads.
• Translating generative AI into tangible accessibility tools: By coupling generative software with high-resolution SLA resin printing, this work bridges digital AI synthesis with physical assistive tools for the BLV community.

Limitations & Future Work

• Perceptual cognitive load of complex 2.5D geometry: Touching 2.5D reliefs remains cognitively demanding; overlapping spatial projections and perspective foreshortening can still challenge users despite braille labels.
• User study sample size: Physical fabrication constraints and recruitment timelines limited the in-person study to 15 participants (8 BLV, 7 BSP); larger, geographically distributed cohorts are needed to assess long-term educational utility.
• Multi-material physical feedback: The current system outputs monolithic resin prints; integrating multi-material 3D printing could enable variations in compliance, friction, elasticity, and thermal conductivity.

Related Work & Insights

• vs Pic2Tac & Swell Paper Systems: Prior methods relied on heuristic edge detection or 2D tactile icons printed on swell paper; this approach produces rich, expressive 2.5D bas-reliefs with fine material textures.
• vs Hunyuan3D 2.1 & Trellis 2: General text-to-3D models focus on visual rendering, frequently generating non-manifold faces, floating fragments, and unlevel bases that fail physical fabrication; this system achieves 100% watertight 3D printability via template guidance and base flattening.
• vs TactileDreamFusion: Previous work was restricted to per-material optimization on existing 3D assets; this framework synthesizes open-vocabulary tactile textures and complete 2.5D tactile graphics directly from free-form text.

Rating

• Novelty: ⭐⭐⭐⭐⭐ (First integrated generative pipeline producing fabrication-ready 2.5D tactile graphics with braille directly from text)
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ (Combines rigorous quantitative ablations, physical SLA 3D printing, and blindfolded/BLV in-person perceptual trials)
• Writing Quality: ⭐⭐⭐⭐⭐ (Clear mathematical formulation, disciplined notation, and transparent analysis of fabrication trade-offs)
• Value: ⭐⭐⭐⭐⭐ (Democratizes customized tactile graphics for over 290 million visually impaired people worldwide)