Skip to content

FullFront: Benchmarking MLLMs Across the Full Front-End Engineering Workflow

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/
Area: LLM Reasoning
Keywords: Multimodal LLM, Front-End Engineering, Code Generation, Webpage Perception QA, Perception-Generation Decoupling

TL;DR

FullFront introduces the first comprehensive benchmark assessing MLLMs across the full front-end development workflow—spanning conceptual webpage design, fine-grained visual perception QA, and functional code generation with interaction authoring—revealing critical grounding failures and a striking decoupling between visual perception and code implementation.

Background & Motivation

Front-end engineering represents a foundational pillar of modern interactive software development, demanding an intricate workflow that bridges high-level conceptual design, fine-grained visual comprehension of spatial layouts, and the precise authoring of functional, interactive code. With rapid breakthroughs in Multimodal Large Language Models (MLLMs), researchers and practitioners have increasingly pursued autonomous front-end engineering systems capable of realizing an end-to-end "idea-to-design-to-code" paradigm. However, existing evaluation suites remain fragmented and narrow in scope: benchmarks such as Design2Code, IW-Bench, and WebCode2M focus almost exclusively on translating static screenshots into raw HTML, yet they rely on heavily bloated crawled codebases or oversimplified synthetic templates, while ubiquitously substituting critical visual imagery with uniform dummy placeholders or random images. Conversely, multimodal web QA benchmarks like WebQuest and VisualWebBench emphasize high-level semantic question answering over webpage contents, largely ignoring the fine-grained visual perception of element dimensions, alignment, margins, and hierarchical structuring that constitutes the lifeblood of real-world front-end implementation.

Crucially, these disjointed evaluation regimes foster a systemic illusion regarding model capabilities. In isolated screenshot-to-code tasks, a model that produces aesthetically coherent layouts is frequently assumed to possess robust visual comprehension of the target interface. Yet, this foundational assumption has never been rigorously scrutinized within an integrated, multi-stage engineering pipeline where perception, design, and code authoring can be directly cross-examined. Real-world software engineering requires developers not only to synthesize visual interfaces from abstract textual briefs and decipher micro-level typographic styling, but also to implement dynamic event handlers (such as dropdown toggles, modal dialogs, and hover effects) and perform incremental refactoring on legacy codebases. Without a cohesive benchmark spanning this complete development lifecycle, the acute bottlenecks hindering MLLMs from functioning as dependable front-end engineering assistants remain completely obscured.

To bridge this critical evaluation gap, this paper introduces FullFront, the first systematic benchmark designed to evaluate MLLMs across the end-to-end front-end engineering workflow. FullFront addresses three core stages of the front-end lifecycle: Webpage Design (conceptualization), Webpage Perception QA (visual comprehension), and Webpage Code Generation (implementation). Leveraging an MLLM-assisted two-stage reconstruction pipeline applied to authentic real-world websites, the authors curate a clean, standardized, and copyright-free evaluation corpus equipped with 15 predefined image semantic categories and structured interaction specifications. Core idea: by unifying design conceptualization, fine-grained visual perception QA, and executable code implementation into a standardized evaluation pipeline, FullFront provides a multi-tiered testbed that quantifies the capability gap between current MLLMs and expert human engineers while unmasking a profound decoupling between visual grounding and code generation.

Method

FullFront establishes a multifaceted front-end evaluation pipeline encompassing 3 core developmental phases and 8 specialized subtasks, supported by rigorously validated multi-layer metrics and controlled data curation protocols.

Overall Architecture

The FullFront benchmark maps directly to the three essential phases of front-end software development: 1. Webpage Design (200 tasks): Given rich natural language descriptions of page layouts, text-to-image MLLMs are evaluated on their capacity to conceptualize coherent, visually structured webpage layouts; 2. Webpage Perception QA (1,800 multiple-choice questions): Evaluates fine-grained visual comprehension of spatial layout, element dimensions, alignment, and typographic hierarchy across Real-world QA (1,250 questions on 625 pages), Synthetic QA (400 questions on 200 pages), and Multi-window QA (150 questions across multi-screenshot contexts); 3. Webpage Code Generation (700 coding challenges): Evaluates practical implementation across four distinct settings: Image to Code (200 direct screenshot-to-HTML tasks), Text to Code (200 specification-to-HTML tasks), Code Refinement (200 incremental code optimization tasks updating draft HTML-v1 to match target screenshots), and Interaction Authoring (100 dynamic tasks implementing click/hover events from before-and-after screenshots).

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400, 'subGraphTitleMargin': {'top': 8, 'bottom': 16}}}}%%
flowchart TD
    A["Real-World Webpage Corpus<br/>625 authentic screenshots + privacy filtering"] --> B["Two-Stage Controlled Synthesis<br/>OmniParser extraction + GPT-4o HTML-v1 + Claude 3.7 HTML-v2"]
    B --> C["15-Category Standardized Image Strategy<br/>Semantic URL assignment + explicit CSS sizing requirement"]
    C --> D["Three-Phase Workflow Evaluation"]
    subgraph D["Three-Phase Workflow Evaluation"]
        direction TB
        E["Webpage Design (200 problems)<br/>Text to Design Image"]
        F["Perception QA (1,800 problems)<br/>Real-world / Synthetic / Multi-window"]
        G["Code Generation (700 problems)<br/>Image2Code / Text2Code / Refine / Interaction"]
    end
    D --> H["Bimodal Metric Validation Framework<br/>Code Score + Gemini Visual Score + Blind Expert Evaluation"]
    H --> I["Cognitive Decoupling & Error Diagnostics<br/>4-quadrant confusion analysis & failure mode attribution"]

Key Designs

1. Controlled Two-Stage Data Curation and 15-Category Image Strategy: eliminating crawled code bloat and uninformative placeholders Directly scraped webpage codebases are notoriously plagued by thousands of lines of minified JavaScript, tracking scripts, and deeply nested boilerplate DOM nodes, rendering them unsuitable as ground-truth evaluation references; furthermore, existing benchmarks uniformly resort to blank monochrome placeholders or uncurated random images that destroy visual-semantic coherence. FullFront resolves this via an automated two-stage reconstruction pipeline: OmniParser first extracts fine-grained bounding boxes and OCR textual contents from real webpage screenshots; GPT-4o leverages this structured data to generate an initial clean draft (HTML-v1); subsequently, Claude 3.7 Sonnet meticulously refines the layout, typographic hierarchy, and visual styling against the original high-resolution screenshot to produce the canonical reference (HTML-v2). For visual media, FullFront establishes 15 standardized semantic categories (People, Animal, Food, Plant, Landscape, Icon, Logo, Architecture, Technology, Transportation, Map, Texture, Art, Movie, Other) mapped to fixed royalty-free URLs. When authoring code from screenshots, models must semantically classify each depicted image into its proper category and explicitly declare dimensions and positioning in CSS, thoroughly probing their capacity to manage proportional visual assets without relying on natural asset aspect ratios.

2. Comprehensive Multi-Subtask Lifecycle Coverage: bridging static coding, text briefs, dynamic interactions, and code refinement To transcend the narrow boundaries of static image-to-code conversion, FullFront introduces four coordinated coding environments reflecting real developer workflows: beside standard Image to Code (200 samples), Text to Code (200 samples) evaluates the ability to generate complete web experiences purely from exhaustive textual briefs without visual aids; Code Refinement (200 samples) simulates iterative code maintenance by supplying models with draft HTML-v1 code and target screenshots, requiring precise incremental modifications rather than rewriting from scratch; Interaction Authoring (100 samples) pioneers the systematic evaluation of client-side dynamics. Grounded in paired "before" and "after" interaction screenshots, models must infer the target event from 10 standardized interaction types—comprising 6 click interactions (display dropdown, toggle checkbox, change background color, display modal, display tooltip, reveal text input) and 4 hover interactions (display dropdown, bold text, underline text, display tooltip)—and output unified, self-contained HTML/CSS/JavaScript with designated hook IDs (#InteractionPart) and accessibility bindings.

3. Multi-Faceted DOM Code Score and Fine-Grained Visual Judge: providing reliable proxies aligned with human judgment Traditional string edit distances fail to accommodate diverse yet semantically equivalent CSS implementations (e.g., Flexbox vs. CSS Grid), while raw pixel similarity is fragile against minor rendering offsets. FullFront introduces a composite Code Score combining DOM structural sequence alignment with implementation-penalized content fidelity: $\(Score_{code} = 0.25 \cdot Sim_{struct} + 0.20 \cdot Sim'_{text\_content} + 0.10 \cdot Sim'_{text\_style} + 0.20 \cdot Sim'_{image} + 0.25 \cdot Sim'_{form}\)$ Structural similarity (\(Sim_{struct}\)) computes the Longest Common Subsequence (LCS) ratio over parsed DOM tag sequences. For each content modality \(c \in \{text, image, form\}\), matched element pairs \((e_{gen}, e_{ref})\) are identified via bipartite matching above threshold \(\theta_{match} = 0.9\), and their average similarity is scaled by the category implementation rate \(Rate_c = |M_c| / |E_{ref,c}|\) as a completeness penalty (\(Sim'_c = Sim_c \cdot Rate_c\)). To complement code metrics, Gemini 2.5 Flash is deployed as a 10-dimensional visual evaluator assessing element reproduction, sizing proportions, typographic fidelity, margin/alignment precision, visual hierarchy, and color harmony. Extensive validation against blind human expert rankings demonstrates exceptional Spearman rank correlations for both Gemini Visual Score (\(\rho = 0.9364\)) and Code Score (\(\rho = 0.9152\)).

4. Four-Quadrant Decoupling Analysis: unmasking grounding failures and design prior dependence To rigorously investigate whether accurate code generation stems from true visual perception or superficial pattern matching, FullFront introduces an aligned cross-task diagnostic framework. By pairing 100 questions from Synthetic QA directly with the corresponding elements in Image to Code generation outputs, model behaviors are mapped into four distinct quadrants: \(p_\checkmark c_\checkmark\) (accurate perception and faithful implementation), \(p_\checkmark c_\times\) (accurate perception bottlenecked by coding execution), \(p_\times c_\checkmark\) (faulty perception paired with correct code implementation, termed "Grounding Failure"), and \(p_\times c_\times\) (complete failure across modalities). This taxonomy provides a sharp diagnostic tool to uncover whether frontier models truly ground visual inputs or merely exploit strong canonical layout priors.

Benchmark & Evaluation Protocol

The benchmark evaluates 12 leading multimodal and text-only models, including proprietary frontiers (Claude 3.7 Sonnet, GPT-5, GPT-4.1, o4-mini, Gemini 2.5 Pro/Flash, GPT-4o) and prominent open-source models (Qwen2.5-VL-72B-Instruct, InternVL3-78B, InternVL2.5-78B, LLaVA-OneVision-72B). Experiments were conducted on a high-performance cluster utilizing 4 NVIDIA A100-SXM4-80GB GPUs with inference temperature fixed at 0.0. A curated FullFront-mini subset (comprising 350 QA items, 200 coding problems, and 50 design tasks) was independently evaluated by 5 professional front-end engineers to establish a human performance upper bound.

Key Experimental Results

Main Results

Evaluation results across the three Webpage Perception QA subtasks (Table 2) and four Webpage Code Generation subtasks (Table 3 / Table 8) highlight significant capability stratifications:

Model Perception: Real (Acc) Perception: Syn (Acc) Perception: Multi (Acc) Code: Refine (Code/Vis) Code: Img2Code (Code/Vis) Code: Inter (Code/Vis) Code: Text2Code (Code/Vis)
Human Expert* 0.9700 0.9600 0.9400 — — — —
Claude 3.7 Sonnet 0.5464 0.5325 0.4533 0.63 / 8.36 0.64 / 8.93 0.55 / 9.18 0.64 / 8.14
GPT-5 0.5816 0.5775 0.5467 0.58 / 8.84 0.56 / 8.58 0.53 / 9.00 0.59 / 8.48
GPT-4.1 0.4672 0.4650 0.3733 0.67 / 9.03 0.61 / 8.89 0.55 / 9.13 0.55 / 8.36
Gemini 2.5 Flash 0.4800 0.4250 0.3867 0.70 / 8.65 0.63 / 8.64 0.52 / 8.07 0.59 / 8.01
o4-mini 0.4976 0.5300 0.4400 0.62 / 8.16 0.57 / 8.47 0.55 / 8.84 0.51 / 6.87
GPT-4o 0.4448 0.4675 0.3733 0.43 / 6.53 0.34 / 5.91 0.36 / 5.81 0.49 / 6.14
InternVL3-78B 0.4816 0.5375 0.4600 0.49 / 5.87 0.42 / 4.47 0.38 / 4.48 0.50 / 4.91
Qwen2.5-VL-72B 0.4696 0.4950 0.4267 0.53 / 6.21 0.40 / 4.48 0.40 / 6.22 0.49 / 5.18
InternVL2.5-78B 0.4696 0.5050 0.4267 0.36 / 5.03 0.33 / 4.01 0.30 / 3.51 0.50 / 5.44
LLaVA-OneVision-72B 0.3296 0.3275 0.2733 0.33 / 4.90 0.14 / 1.89 0.06 / 0.45 0.41 / 5.13

(Notes: Vis denotes Gemini Visual Score [scale 0-10], Code denotes Code Score. Rows marked with * were evaluated on FullFront-mini).

Interaction Performance and Decoupling Quadrant Distribution

Analysis of interaction success rates (Table 9) alongside the 4-quadrant breakdown of Perception accuracy (\(p\)) vs. Code Implementation accuracy (\(c\)) across 100 aligned Synthetic QA samples (Table 5 / Table 7) reveals fundamental cognitive divides:

Model Overall Interaction Success (%) \(p_\checkmark c_\checkmark\) (Aligned Success) \(p_\checkmark c_\times\) (Coding Bottleneck) \(p_\times c_\checkmark\) (Grounding Failure) \(p_\times c_\times\) (Dual Failure) Fatal Code Errors: Size / Blank Page
o4-mini 93.00 — — — — 22 / 1
GPT-5 92.00 48 6 31 15 3 / 1
Claude 3.7 Sonnet 78.00 44 3 36 17 6 / 0
GPT-4.1 78.00 — — — — 7 / 6
Gemini 2.5 Flash 70.00 39 5 25 31 10 / 1
GPT-4o 80.00 32 13 21 34 47 / 0
Qwen2.5-VL-72B 57.00 22 28 10 40 195 / 17
InternVL3-78B 48.00 25 23 8 44 93 / 23
LLaVA-OneVision-72B 16.00 — — — — 252 / 39

Key Findings

  • Pervasive Perception-Generation Decoupling and "Grounding Failures": In leading proprietary models, a substantial fraction of test cases fall into the \(p_\times c_\checkmark\) quadrant—36% in Claude 3.7 Sonnet and 31% in GPT-5. In these instances, the model fails to correctly answer subtle multiple-choice questions concerning element alignment or relative positioning (e.g., misjudging a tag's vertical placement), yet generates perfectly centered and properly aligned HTML/CSS code matching standard web conventions. Macroscopic generation scores remain virtually unchanged regardless of whether models answer perceptual questions correctly (Table 6). This statistical invariance demonstrates that high visual fidelity in MLLM-generated code is largely sustained by powerful pre-trained canonical layout priors rather than genuine visual grounding of input pixels.
  • Severe Coding Bottlenecks in Open-Source MLLMs: While open-source architectures like InternVL3-78B achieve competitive perception accuracy on Synthetic QA (53.75%, surpassing Claude 3.7 Sonnet's 53.25%), their implementation capacity collapses when translating spatial insights into executable code. Open-source models exhibit a heavy concentration in the \(p_\checkmark c_\times\) quadrant (23%~28%) and suffer catastrophic failure rates in Interaction Authoring (LLaVA-OneVision-72B drops to 16% success). Furthermore, open-source models frequently trigger fatal runtime anomalies, including massive Abnormal Image Sizes that disrupt layout flow (195-252 occurrences) and infinite generation loops producing empty/malformed pages (17-39 Blank Pages).
  • Textual Specifications Effectively Rival Visual Inputs: Comparative analysis across input modalities reveals that Text to Code achieves parity with Image to Code for premier models (Claude 3.7 Sonnet scores 0.64 Code Score on both). Supplementary benchmarking of frontier text-only models (Table 11) demonstrates that DeepSeek-R1 (0.6090) and DeepSeek-V3.1 (0.6050) surpass every open-source multimodal model and closely trail top proprietary MLLMs on identical prompts. This suggests that for latency- and compute-constrained agent workflows, transforming visual inputs into dense structured textual briefs before code synthesis represents a highly viable architectural alternative.

Highlights & Insights

  • Holistic Front-End Workflow Unification: FullFront establishes the first closed-loop benchmark unifying the entire software engineering lifecycle—from conceptual UI design and micro-level layout perception to executable code authoring and dynamic event handling.
  • Empirical Deconstruction of Visual Grounding in Code: The benchmark provides concrete empirical proof that visual code fidelity does not imply accurate visual perception, offering a vital methodological blueprint for diagnosing cognitive priors versus actual perception in multimodal agents.
  • Copyright-Free Controlled Data Generation Pipeline: The synergy of OmniParser screen deconstruction, dual-stage LLM code refinement, and 15-category standardized asset mapping provides a robust, reusable paradigm for constructing clean GUI benchmark data across web, desktop, and mobile platforms.

Limitations & Future Work

  • Single-Page Vanilla Tech Stack Scope: The evaluation environment is currently constrained to standalone, desktop-rendered single-page HTML/CSS/JavaScript, omitting modern componentized enterprise frameworks (React, Vue, Tailwind CSS), state management libraries, multi-page routing, asynchronous API mocking, and responsive cross-viewport adaptation.
  • Restricted Interaction Complexity: Interaction Authoring evaluates 10 atomic click/hover micro-interactions, lacking multi-step form validation, drag-and-drop mechanics, canvas manipulation, or long-horizon stateful user workflows; likewise, Perception QA is formulated as multiple-choice questions rather than interactive exploration or visual debugging.
  • Absence of End-to-End Error Propagation Analysis: While FullFront coordinates three workflow stages, models are currently evaluated on each stage independently; future extensions should evaluate end-to-end multi-agent pipelines where artifacts flow sequentially from conceptual design sketches to code synthesis to assess cascade error accumulation.
  • vs Design2Code / WebCode2M / IW-Bench: Prior screenshot-to-code benchmarks rely on raw scraped code with external dependencies and dead links, while replacing images with generic unstyled placeholders. FullFront introduces 15 semantic image categories and mandates explicit CSS sizing, enforcing realistic visual styling and preventing layout collapse.
  • vs WebQuest / VisualWebBench: Existing webpage QA benchmarks primarily measure high-level semantic reading comprehension and knowledge retrieval. FullFront focuses specifically on spatial topology, typographic weights, button proportions, and margin alignments directly tied to front-end engineering execution.
  • vs Interaction2Code: While Interaction2Code explored preliminary interactive synthesis, FullFront defines a standardized 10-type interaction taxonomy with rigorous single-file automated validation protocols (e.g., #InteractionPart scoping and aria-expanded state verification), establishing a significantly more reliable benchmark for dynamic web code.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Establishes the first unified benchmark covering the full front-end engineering lifecycle, providing transformative insights into multimodal perception-generation decoupling.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Encompasses 12 frontier models plus human expert baselines across 2,700 benchmark tasks, enriched with multi-metric correlation analyses and cognitive confusion matrices.
  • Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally clear narrative progression, comprehensive tabular presentations, rigorous failure mode taxonomies, and exhaustive appendices.
  • Value: ⭐⭐⭐⭐⭐ Delivers an indispensable evaluation infrastructure and foundational diagnostic insights for the advancement of future Generalist Front-End Engineering Agents.