Skip to content

VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback?

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/vis-nlp/VisEditBench
Area: Multimodal Vision-Language Models
Keywords: Data Visualization Code Editing, Multimodal Feedback, Vision-Language Models, Render-Grounded Agent, Benchmark Evaluation

TL;DR

VisEditBench establishes a human-annotated benchmark of 1,395 multimodal visualization code-editing tasks across feedback-guided repair and reference-guided restyling, uncovering a severe divergence between code executability and visual fidelity in frontier VLMs, and demonstrates that the closed-loop VisEditAgent boosts overall editing pass rate by 12.24%.

Background & Motivation

Data visualizations are ubiquitous across data science, business analytics, journalistic storytelling, and scientific discovery. While recent advancements in vision-language models (VLMs) have catalyzed rapid progress in text-to-visualization code generation and chart-to-code inverse rendering, real-world visualization authoring is fundamentally iterative. Practitioners almost never ship the initial one-shot visualization: even when models generate syntactically valid and runnable visualization scripts, the resulting charts frequently exhibit overlapping labels, clipped annotations, illegible legends, poor visual balance, or misalignment with publication style guidelines. Consequently, human authors constantly inspect rendered charts, diagnose visual defects, edit the underlying code, and re-execute the program through repeated cycles of trial and error.

Existing benchmarks fail to capture this dominant paradigm, focusing almost exclusively on synthesis from scratch or unconstrained image-to-code reconstruction. In real workflows, developers already possess functioning base code and must perform surgical, localized modifications informed by multimodal inputsโ€”such as a defective chart image marked with handwritten circles and arrows, or an exemplary target chart exhibiting desired aesthetic choices. Crucially, visualization code editing demands multimodal grounding: the model must cross-reference visual issues with specific code snippets, modify styling and layout properties, and ensure that underlying data mappings, scales, and analytical semantics remain strictly preserved. Single-pass code generators completely lack the visual perceptual feedback loop necessary to evaluate whether code modifications actually rectify visual defects without introducing collateral distortions.

To bridge this fundamental gap and provide an empirical foundation for interactive visualization systems, this paper systematically explores visualization code editing from multimodal feedback. Core idea: VisEditBench introduces the first 1,395-task benchmark spanning feedback-guided repair and reference-guided restyling across eight editing intents, complemented by VisEditAgentโ€”a render-grounded baseline that iteratively plans, generates, executes, validates, and refines edits against rendered visual evidence to overcome the "executable yet visually flawed" bottleneck of zero-shot VLMs.

Method

Overall Architecture

VisEditBench models multimodal visualization code editing as a formal revision task: given an initial visualization script \(c_i\), a rendered chart image \(I_i\) (exhibiting a bug, human markup, or a target reference design), and a natural language instruction \(u_i\), the system must produce revised executable code \(c'_i\) that renders the intended visual output while preserving the underlying data semantics. Tasks are divided into two distinct real-world regimes: Feedback-guided Repair, where the model repairs layout, readability, or encoding flaws based on buggy/annotated charts and textual complaints; and Reference-guided Restyling, where the model adapts visual aesthetics (color palettes, typography, gridlines, geometry) to visually match a target chart while maintaining original data values.

To solve this task reliably, the authors propose VisEditAgent, an iterative render-grounded agent framework that mirrors human exploratory visualization workflows. Moving beyond blind single-turn code emission, VisEditAgent orchestrates a multi-stage execution loop: it plans targeted modifications, synthesizes diverse code candidates, executes and renders them into physical image artifacts in isolated execution backends, visually validates candidates using a strict multi-metric multimodal evaluator, and performs targeted feedback-guided refinement on the strongest candidate.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: Base Code + Chart Image + Edit Instruction"] --> B["Edit Planning<br/>Joint Visual Diagnosis & Code Localization"]
    B --> C["Candidate Generation & Rendering<br/>Parallel Sampling & Headless Execution"]
    C --> D["Visual Validation & Selection<br/>Multimodal Rubric-Based Ranking"]
    D --> E["Feedback-Based Refinement<br/>Iterative Polishing via Diagnostic Signals"]
    E --> F["Output: Revised High-Fidelity Visualization Code"]

Key Designs

1. Edit Planning: Joint Visual Diagnosis & Code Localization Zero-shot models frequently fail because they jump directly into code rewriting without diagnosing where the visual fault lies or which parameters govern it, often accidentally mutating data processing routines. Edit Planning addresses this premature commitment problem by compelling the agent to conduct explicit multimodal cross-reasoning before touching the codebase. Given the base code \(c_i\), the rendered/annotated image \(I_i\), the instruction \(u_i\), and task metadata (editing intent, chart type, visualization library), the model reasons across three explicit questions: what specific visual defect or styling target is present in the rendered image; which functions, arguments, or layout declarations in \(c_i\) are responsible for this visual behavior; and what minimal-impact editing strategy should be executed. This step bounds the scope of edits to layout, styling, and annotations, preventing destructive drift in underlying data aggregations.

2. Candidate Generation & Rendering: Parallel Sampling & Headless Execution Because visualization adjustments often involve multiple syntactically valid yet visually divergent configurations (such as rotating tick labels versus increasing figure size versus staggering labels across two rows), relying on a single deterministic prediction yields low success rates. Candidate Generation & Rendering overcomes this by prompting the model to sample \(K\) diverse implementation variants (\(K=3\) by default) conditioned on the edit plan. Each candidate code is immediately dispatched to an isolated headless execution environment: Matplotlib scripts are executed using a non-interactive Agg backend, while declarative Vega-Lite specifications are rendered via vl-convert-python. This step grounds abstract code tokens into concrete pixel bitmaps, transforming symbolic programming search into an observable physical visual space.

3. Visual Validation & Selection: Multimodal Rubric-Based Ranking Since Python interpreters only verify whether code runs without throwing runtime exceptions, code execution alone provides zero guarantee of visual satisfaction. Visual Validation & Selection introduces an independent VLM judge (GPT-4o by default; dynamically routed to Gemini 2.5 Pro when evaluating GPT-4o outputs to eliminate self-evaluation bias) that evaluates rendered candidates against the input instruction and reference/target image. The judge scores candidates across five fine-grained criteria: Code Execution \(E \in \{0, 1\}\), Task Accuracy (0โ€“5, assessing whether requested changes are fulfilled while retaining analytical semantics), Readability and Clarity (0โ€“5, evaluating label collision, occlusion, and axis scaling), Visual Quality (0โ€“5, rating aesthetic polish and color palette suitability), and Visual Similarity (0โ€“100, checking alignment with the corrected or reference chart). The framework ranks rendered options and selects the candidate that balances semantic preservation with high visual fidelity.

4. Feedback-Based Refinement: Iterative Polishing via Diagnostic Signals Even the top-ranked candidate may retain residual imperfections, such as an annotation box slightly overlapping a bar top or gridlines remaining too prominent. Feedback-Based Refinement closes the final performance gap by extracting diagnostic feedbackโ€”including runtime tracebacks (in the event of execution failure) or visual critique discrepancies returned by the validatorโ€”and reinjecting them into the model's context for a targeted refinement turn. Guided by precise error localization ("the legend is still positioned inside the plot area overlapping data points"), the model applies surgical adjustments to the winning code candidate and re-renders the chart. This loop resolves cascading layout bugs within an economical budget (averaging approximately 5 API calls, ~30 seconds of latency, and CAD \(0.08โ€“\)0.10 per task).

Loss & Training

VisEditBench is an empirical benchmark and VisEditAgent is an entirely training-free prompting and execution framework. All 24 evaluated models (proprietary and open-source) are assessed in a standardized zero-shot setting using consistent decoding parameters: high-detail image inputs, greedy/low-temperature decoding at temperature = 0.2, top-p = 1.0, maximum generation length of 4096 tokens, and strict prompts instructing the models to return only complete, runnable visualization code without conversational boilerplate. The automated VLM judge operates at temperature = 0.0 with a 1200 token budget. A sample achieves an official Pass if and only if all stringent criteria are simultaneously satisfied: \(E = 1\), Task Accuracy \(\ge 4.5\), Readability & Clarity \(\ge 4.0\), Visual Quality \(\ge 4.0\), and Visual Similarity \(\ge 90\).

Key Experimental Results

Main Results

The primary evaluation assesses 24 frontier closed-source and open-source VLMs across the full 1,395 VisEditBench tasks. Table 1 reports the final pass rates (in %) broken down by the eight distinct editing intents:

Model Correctness Repair Quality Improv. Robustness Gen. Style Adapt. Constraint Satis. Consistency Harmon. Refactor / Transform. Style-Aware Repair Overall Pass Rate
Claude-Opus-4.6 69.90 79.10 84.50 80.00 79.10 59.30 79.40 77.60 76.00
Claude-4.6-Sonnet 70.34 79.73 86.20 55.71 80.76 64.40 85.71 61.22 74.46
Claude-Opus-4.1 64.50 73.50 80.10 48.60 73.10 49.20 74.60 46.90 67.30
GPT-5 61.60 75.45 56.89 57.14 59.61 66.10 69.84 51.02 65.68
Gemini-2.5-Pro 61.20 70.90 73.30 38.60 69.20 52.50 57.10 40.80 63.40
GPT-5-Mini 55.51 64.64 71.55 47.14 67.30 55.93 58.73 51.02 60.00
Claude-Haiku-4.5 58.40 66.80 67.20 31.40 65.30 49.20 41.20 36.70 58.80
GPT-4o (Zero-shot) 56.08 67.34 57.76 10.00 63.46 47.46 42.86 30.61 55.75
Qwen3-VL-32B 54.95 59.00 50.00 21.42 48.10 54.23 36.51 16.32 51.72
Qwen3-VL-30B-A3B 49.80 53.38 44.83 15.71 38.46 42.37 47.61 20.41 47.05
Gemma-3-27B 49.24 56.76 23.28 0.00 26.92 42.37 12.70 12.25 42.95
Qwen3-VL-8B 49.04 52.02 28.44 0.00 5.70 52.44 26.98 14.29 42.09
Qwen3-VL-4B 51.11 50.90 1.72 1.42 7.69 37.28 4.76 16.32 38.84
Pixtral-12B 44.68 47.07 5.17 0.00 15.38 30.51 6.35 10.20 35.32
Gemma-3-12B 42.97 47.97 6.89 1.43 13.46 28.81 7.94 10.20 34.96
InternVL-3.5-14B 41.65 49.78 3.45 0.00 3.85 37.29 7.94 10.21 34.83
InternVL-3.5-8B 42.96 41.22 7.75 0.00 11.54 32.20 12.70 10.20 33.17
Gemma-3-4B 30.79 25.00 1.00 0.00 3.85 16.95 3.18 1.00 21.29
Qwen3-VL-2B 24.52 20.49 1.00 0.00 5.77 16.95 7.94 6.12 17.55
Gemini-3.0-Flash 17.96 19.28 9.48 35.71 5.77 15.25 9.52 0.00 17.06
InternVL-3.5-1B 11.41 7.88 0.00 0.00 1.92 8.47 1.59 2.04 7.41

Ablation Study

Table 2 details the performance gains unlocked by VisEditAgent across multiple foundation backbones, alongside module ablations isolating the contributions of candidate generation and iterative refinement (values in %):

Configuration / Model Correctness Repair Quality Improv. Robustness Gen. Style Adapt. Constraint Satis. Consistency Harmon. Refactor / Transform. Style-Aware Repair Overall Pass Rate
GPT-4o Zero-shot 56.08 67.34 57.76 10.00 63.46 47.46 42.86 30.61 55.75
GPT-4o + VisEditAgent (Full) 64.64 78.37 61.22 62.85 65.38 64.41 60.32 44.89 67.99
w/o Multi-candidate Gen. 61.44 74.44 56.03 28.57 59.62 55.93 52.38 38.78 61.94
w/o Refinement 58.79 71.52 52.59 41.43 57.69 54.24 49.21 34.70 59.86
Qwen3-VL-4B Zero-shot 51.11 50.90 1.72 1.42 7.69 37.28 4.76 16.32 38.84
Qwen3-VL-4B + VisEditAgent 63.14 63.00 42.24 35.71 44.23 54.23 42.85 18.36 44.98
Qwen3-VL-8B Zero-shot 49.04 52.02 28.44 0.00 5.70 52.44 26.98 14.29 42.09
Qwen3-VL-8B + VisEditAgent 71.08 67.71 1.80 24.29 13.46 49.15 17.46 16.36 54.53

To decouple render-grounded feedback from brute-force token compute, the authors evaluated compute-matched baselines on a stratified 500-task subset: GPT-4o zero-shot achieved 55.0%, Best-of-3 reached 58.2%, Best-of-5 reached 59.4%, and 3-candidate code-only refinement (without rendering images) achieved 62.8%, whereas full VisEditAgent attained 68.0%. Moreover, in the visual-input ablation, stripping the chart image while keeping the instruction and code caused GPT-4o's pass rate to collapse from 59% to 42%, establishing that visual evidence is indispensable for localizing subtle layout and presentation bugs.

Key Findings

  • High Code Executability Masking Widespread Visual Failure: Frontier models achieve near-perfect code execution rates (Claude Opus 4.6 at 96.42%, Claude-4.6-Sonnet at 96.19%, and GPT-4o at 93.31%), yet their overall pass rates drop to 76.00%, 74.46%, and 55.75% due to lower visual similarity and task accuracy. Qualitative error analysis over 500 failure instances reveals that 46.7% are "executable but visually incorrect", 28.4% stem from weak visual feedback localization, and 14.7% fail in reference-style matching.
  • Style Adaptation as a Universal Zero-Shot Blind Spot: Zero-shot models struggle drastically when tasked with transferring styling properties from a reference chart image without modifying underlying data points. All open-source models under 14B parameters scored near 0.00% on Style Adaptation, and GPT-4o attained merely 10.00%. VisEditAgent dramatically alleviates this deficit, boosting GPT-4o's Style Adaptation pass rate to 62.85% (+52.85 percentage points) via multi-candidate visual selection.
  • Task Difficulty and Multi-cause Complexity: 57.1% of benchmark tasks involve multi-cause defects requiring simultaneous coordinated edits across geometry, labels, and encodings. As difficulty escalates from Easy to Hard, pass rates degrade precipitously across all systems: GPT-4o drops from 69.9% to 37.8%, while Claude-4.6-Sonnet drops from 86.2% to 56.2%, highlighting that complex spatial interdependencies remain fundamentally challenging for current VLMs.

Highlights & Insights

  • Authentic Engineering Workflows over Synthetic Prompts: By harvesting real issues from Stack Overflow, GitHub repositories, and diverse empirical charts, VisEditBench mirrors the practical realities of data visualization debugging, replacing ungrounded scratch-generation with realistic multimodal iterative maintenance.
  • The "Render-as-Feedback" Principle: VisEditAgent demonstrates that multimodal code agents must not treat code generation as an open-loop text prediction task. Integrating headless renderers to ground candidate programs into image artifacts enables visual perceptual feedback to steer program synthesis effectively.
  • High Human-Evaluator Alignment: Comprehensive manual validation across 2,790 model outputs established strong Pearson correlations (\(0.8338\) to \(0.8700\)) between automated rubric scores and human judgments, alongside 81.0% pass/fail concordance, demonstrating the reliability of the automated evaluation framework.

Limitations & Future Work

  • Ecosystem Breadth: VisEditBench currently benchmarks Matplotlib (82.9%, imperative Python) and Vega-Lite (17.1%, declarative JSON-like specification). It does not yet incorporate web-native interactive visualization libraries like D3.js or Plotly, nor statistical staples like R's ggplot2, where reactivity and grammar-of-graphics primitives present distinct editing challenges.
  • Inference Latency and Cost Overhead: VisEditAgent requires approximately five LLM calls, ~30 seconds of processing time, and roughly CAD \(0.08โ€“\)0.10 per task. Streamlining candidate pruning through lighter visual critic models or direct visual diff representations represents a key avenue for real-time interactive user interfaces.
  • vs Text2Vis / VisEval: These efforts evaluate generation from text queries and raw data tables from scratch, offering no pre-existing code or visual failure feedback; VisEditBench addresses surgical in-place editing of flawed visualizations.
  • vs ChartMimic / Plot2Code: Chart-to-code benchmarks evaluate inverse image-to-code reconstruction without base code or interactive user instructions; VisEditBench tests controlled editing that preserves data semantics while modifying targeted visual facets.
  • vs SWE-bench Multimodal: SWE-bench Multimodal assesses repository-level software bugs with sparse visual signals across broad software stacks; VisEditBench zeroes in on domain-specific visualization challenges including scale calibration, label clutter, chart readability, and reference-guided aesthetic transfer.

Rating

  • Novelty: โญโญโญโญโญ Formulates the first dedicated multimodal visualization code editing benchmark and introduces a rigorous render-grounded agent framework.
  • Experimental Thoroughness: โญโญโญโญโญ Comprehensive zero-shot evaluation across 24 models, coupled with extensive ablation studies, compute-matched controls, error analyses, and full human cross-evaluation.
  • Writing Quality: โญโญโญโญโญ Exceptionally structured presentation, rigorous mathematical/algorithmic task formulation, and rich empirical insights.
  • Value: โญโญโญโญโญ Provides an indispensable benchmark and architectural blueprint for next-generation automated data analysis agents, IDE visualization copilots, and multimodal code reasoning systems.