Self-Evolving MCP-GUI Agents via Automated Environment Generation and Experience Learning

Conference: ECCV 2026
Paper: Official page · PDF
Code: https://github.com/Tiantian-H/EE-MCP
Area: Agents / Computer Use / Multimodal Tool Calling
Keywords: MCP-GUI hybrid policy, automatic environment generation, capability profiling, trajectory distillation, experience bank

TL;DR

The paper improves MCP-GUI agents through a loop of diagnosing weaknesses, generating executable tasks, collecting trajectories, and updating either the model or its experience, raising Chrome pass rate from 60.0% to 77.8% while finding that GUI-intensive VS Code benefits more from training-free experience augmentation than from directly imitating expert trajectories.

Background & Motivation

Computer-use agents can inspect screenshots and manipulate interfaces, or invoke structured tools through the Model Context Protocol (MCP). Each route has blind spots: APIs can directly handle bookmarks, navigation, and file operations, but may not cover every dialog or setting; GUIs offer broad coverage but suffer from coordinate errors, layout changes, and lengthy exploration. MCPWorld and OSWorld-MCP already establish the value of combining them. Giving a model access to both interfaces, however, does not automatically teach it when to call a tool and when to switch to the keyboard or mouse.

Standard expert-demonstration fine-tuning asks a student to imitate successful trajectories, yet an expert's successful visual actions may be difficult for a smaller student to reproduce. Optimizing only aggregate success also obscures whether failures come from tool-name syntax, visual grounding, or dependencies across a sequence of tool calls. This paper asks how these different weaknesses can become executable practice tasks, and whether the resulting knowledge should enter model parameters or remain as inference-time strategic advice. Generating instructions alone is insufficient: browser tabs, spreadsheet contents, and open files must actually be initialized before trajectories and evaluations are meaningful.

Thus, “self-evolution” here is not repeated training without external supervision. It is an automated improvement process supported by an expert model, an LLM judge, tool interfaces, and environment validation. Core idea: use multidimensional capability profiles to decide what to practice, environment generation to make that practice executable, trajectory distillation to learn action execution, and an experience bank to learn transferable strategies; the appropriate mechanism depends on the application's MCP-GUI composition and tool-chain complexity.

Method

Overall Architecture

The inputs are seed tasks, a tool-capable base vision-language model, and an execution environment that can initialize desktop applications. There are two outputs: a fine-tuned policy and an experience bank usable on its own. The agent's state includes a screenshot, available MCP tool descriptions, and action history. Its actions range from structured tool calls to clicks, typing, scrolling, and keyboard shortcuts. The paper casts this as unified hybrid-policy decision making, but does not introduce a separately trained modality-routing network.

The process first collects Claude expert trajectories on seed tasks, optionally performs initial fine-tuning, and constructs an initial experience bank. It then evaluates the current agent to build a capability profile, uses that profile to generate tasks and environments, and feeds new expert trajectories and eligible student successes into the improvement process. The experience bank extracts rules from trajectory differences for the next iteration. The diagram shows the full mode; experience-only augmentation can skip parameter updates.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Seed["Seed tasks and initial policy"] --> Profile["Multidimensional<br/>Capability Profiling"]
    Profile --> Generate["Task and Environment<br/>Generation"]
    Generate --> Distill["Trajectory Filtering<br/>and Distillation"]
    Distill --> Bank["Experience Bank<br/>Augmentation"]
    Bank --> Output["Updated policy and experience bank"]
    Output -->|Next evaluation round| Profile

Key Designs

1. Multidimensional Capability Profiling: locate failures in skills that can be practiced

Knowing only that a task failed does not adequately guide a generator. Claude Sonnet 4 reads the goal, initial and final screenshots, and complete trajectory, returning a binary verdict and a continuous quality score. The system aggregates evidence across five dimensions: modality usage, difficulty, skill category, format quality, and execution efficiency. The skills are data retrieval, data manipulation, search and query, execution and automation, navigation and browsing, and configuration and settings. Format quality covers tool-call formatting, parsing, and argument issues, while efficiency tracks steps, completion, and timeouts. Incorrect call syntax and difficulty with complex configuration tasks therefore produce different profiles.

Gap analysis compares MCP usage and success rates by difficulty and skill against configurable targets, then biases generated tasks toward deficits. In Chrome's second iteration, it identified underused tools, notably bookmark_page and delete_browsing_data, and generated tasks targeting those functions. This changes data collection from simply obtaining more examples to obtaining examples that expose current weaknesses. The thresholds are not universal optima, however, and more MCP calls are not inherently better. The main text does not supply enough detail to reproduce every profile calculation or task-sampling weight.

2. Task and Environment Generation: turn an instruction into an executable initial state

After a task is generated from the profile, an LLM generates its environment setup script. Browser tasks may require particular existing tabs, spreadsheet tasks need populated cells, and editor tasks need prepared files. Once the script runs, the system checks whether the target application reached the expected initial state. Failures are returned to the generator as feedback for another attempt. Environment validation thus sits between task generation and trajectory collection, preventing missing files or other setup failures from being mistaken for weaknesses in the agent.

Only validated environments support the next round of expert demonstrations and student attempts. The design brings manual application-state preparation into the automated loop rather than merely producing more textual instructions. There is an important implementation boundary: the main text describes generation, checking, and failure retries, but delegates the actual validators, retry limits, reset behavior, and isolation implementation to supplementary material. That supplement is absent from the local full-text cache, so the particular validation API or sandbox mechanism cannot be established here.

3. Trajectory Filtering and Distillation: accumulate data but restart learning from base weights

The expert, Claude Sonnet 4, attempts tasks multiple times, and the system retains the trajectory with the highest judge score. Each step records the screenshot, reasoning text, MCP or GUI action, and execution feedback, then standardizes the trajectory to the student's native <tool_call> format. Training supervises reasoning and actions rather than optimizing online reinforcement-learning rewards. For Chrome tasks that map to explicit tool calls, this can directly teach names, arguments, and invocation timing. Whether pixel-level actions are reproducible still depends on the student's visual-grounding ability.

Data accumulate across iterations, with previous successful trajectories replayed alongside new ones. Crucially, each fine-tuning round restarts from base-model weights instead of continuing from the previous fine-tuned checkpoint. Retained knowledge is therefore carried largely by the accumulated dataset, a choice intended to reduce compounding errors. The paper also describes a student self-improvement variant: attempt each task at most 3 times, accept trajectories with continuous scores above 0.5, and retain the best eligible attempt. This rejection sampling uses actions the student can already perform to mitigate expert-student capability mismatch; it does not make all failed trajectories supervision targets.

The algorithm description and experimental attribution should be distinguished. The main algorithm emphasizes expert-trajectory accumulation, while Section 4.5 and the experimental narrative also introduce successful student trajectories. The main text does not fully specify the proportions of these sources in every experimental configuration. Consequently, reducing distribution shift through self-generated data is a supported methodological rationale, but a particular pass-rate improvement cannot be attributed solely to rejection sampling.

4. Experience Bank Augmentation: retain strategic differences between success and failure, not lengthy raw trajectories

Parameter updates are not the only place to store learning. The bank groups successful Claude and Qwen trajectories by skill category, then uses an LLM to compare them with failures and extract concise, transferable rules. The prose identifies four knowledge types: task strategies, environment knowledge, tool-use patterns, and error recovery. Examples include checking the current view with a screenshot, using reliable keyboard shortcuts, respecting tool naming conventions, and continuing with GUI actions when a tool merely opens a settings page. The rules explain why one execution route is more reliable instead of mechanically replaying expert click coordinates.

Each skill category has a capacity limit to prevent unbounded prompt growth; an LLM merges and summarizes entries when that limit is exceeded. Application-type filtering prevents rules from another application from contaminating the prompt. At inference time, the agent receives experience relevant to the current task's skills. The central relation in the paper's Equation (10) is:

\[ \mathrm{prompt}_k=\mathrm{prompt}_{\mathrm{base}}\oplus\mathrm{Experience}(\mathcal{E}_{k-1},c_{\mathrm{task}}). \]

Here, \(\oplus\) denotes prompt concatenation, \(\mathcal{E}_{k-1}\) is the previous iteration's bank, and \(c_{\mathrm{task}}\) is the task's skill category. This mode requires no weight updates, but generating, maintaining, and reading experience still incurs model-call costs. The prose lists four knowledge types, whereas the formal storage tuple lists only strategies, environment knowledge, and tool patterns. This note preserves that distinction rather than assuming error recovery is a separate implementation field.

A Worked Example

The Chrome print-dialog example illustrates the difference between the two learning mechanisms. The original policy first invoked the nonexistent or incorrectly named google_chrome.print, then repeatedly clicked around. Experience extraction identified that the goal was merely to open the print dialog, which Ctrl+P can do directly. On the next inference pass, the relevant rule enabled a single shortcut action to complete the task. The student need not learn a fragile sequence of click coordinates: experience changes its choice of execution route.

Within the full loop, the incorrect tool name and ineffective GUI exploration enter the capability profile, new tasks probe related weaknesses, and successful-versus-failed trajectory comparison turns the shortcut into a reusable rule. One evidence boundary matters: the paper describes the failed example as taking 17 steps, although the standard evaluation budget is 10 steps per attempt. It does not explain whether these use different settings, so the example is not a clean, matched-budget step-count comparison.

Loss & Training

The main student is Qwen3-VL-8B, fine-tuned with LoRA rank 8 and learning rate \(2\times10^{-5}\), for \(K\in\{1,3\}\) evolution iterations. Serving uses vLLM, tensor parallelism 1, fp16, and temperature 0.2. Each task receives at most 3 attempts, with a student-trajectory acceptance threshold of 0.5. The judge supplies the binary pass verdict independently, so the continuous-score threshold must not be treated as the definition of pass rate.

SFT uses accumulated trajectories to maximize the conditional likelihood of reasoning and actions given the task and state history. Equations (3) through (5) have corrupted symbol extraction in the cache; this note therefore states the supported training semantics without presenting reconstructed conditioning terms as original equations. The paper reports a training pool growing from 44 to 79 samples, but the main text does not fully disclose epochs, batch size, LoRA target layers, data-source proportions, or total teacher-call cost.

Key Experimental Results

Main Results

The benchmark contains 45 Chrome tasks, 30 VS Code tasks, and 45 LibreOffice Calc tasks, totaling 120 seed tasks. They expose 11, 9, and 16 MCP tools respectively, totaling 36. Each attempt has a 10-step budget at \(1024\times768\) screenshot resolution. Pass Rate is the fraction of binary judge successes; LLM Score measures completion quality in \([0,1]\); MCP Ratio is the fraction of all actions that use MCP. Improvements below are percentage points.

The following results come from the paper's Table 6, with Qwen3-VL-8B in all three applications. They convey the central conclusion more clearly than Chrome's best result alone.

Strategy	Chrome pass rate (%)	VS Code pass rate (%)	Calc pass rate (%)
Baseline	60.0	53.3	44.4
Experience bank only	62.2	63.3	46.7
Distillation + experience bank	77.8	43.3	42.2

Distillation plus experience improves Chrome by 17.8 percentage points over baseline but reduces VS Code performance by 10.0 points. The bank alone improves the three applications by 2.2, 10.0, and 2.3 points respectively, suggesting that strategy transfer suits some applications better than action imitation. Calc already has 61.5% baseline MCP usage, which falls to 47.8% after SFT. High tool usage therefore does not guarantee effective expert distillation: multi-step tool composition still matters.

Ablation Study

The paper's Table 4 compares action spaces and peer agents under the same 45-task Chrome protocol. Modality ablations are kept distinct from the full method so that gains from interface availability are not conflated with gains from learning.

Configuration / method	Pass rate (%)	Successful tasks
Hybrid, distillation + experience bank	77.8	35/45
Hybrid, pre-evolution baseline	60.0	27/45
MCP-only ablation	51.1	23/45
GUI-only ablation	48.9	22/45
UI-TARS-1.5-7B	24.4	11/45
Aguvis-7B-720p	15.6	7/45

The hybrid baseline exceeds MCP-only by 8.9 points and GUI-only by 11.1 points, supporting complementary action channels. The full method adds another 17.8 points over the hybrid baseline. However, without a separate distillation-only row, that increment is not a fully isolated causal effect of distillation. The main text also provides no numerical ablations removing capability profiling, environment validation, or bank capacity limits.

Key Findings

The paper's Table 7 introduces three Chrome hold-out designs with 12 tasks each. They retain the same tool coverage but change websites, navigation paths, and targets, and are excluded from evolution. Means and standard deviations below are reported as in the source.

Method	HO-1	HO-2	HO-3	Mean pass rate ± standard deviation (%)
Hybrid	8/12	7/12	9/12	66.7 ± 8.3
MCP-only	7/12	5/12	7/12	52.8 ± 9.6
GUI-only	4/12	5/12	7/12	44.4 ± 12.7
Aguvis-7B-720p	4/12	1/12	3/12	22.2 ± 12.7
UI-TARS-1.5-7B	2/12	2/12	3/12	19.4 ± 4.8

• Hybrid leads all three hold-out designs, but MCP-only and GUI-only tie on HO-2 and HO-3. Their mean-plus/minus-standard-deviation intervals also overlap. The paper's claim that all adjacent methods have non-overlapping intervals is unsupported by the table, and standard deviation is not itself a significance test.
• Chrome's best distillation-plus-experience pass rate, 77.8%, occurs at iteration 1; experience-only peaks at 64.4% in iteration 2. The iteration-3 MCP ratio of 55.8% and action reduction from 243 to 163 must not be combined with the iteration-1 peak as if achieved simultaneously.
• In iteration 1, the full method's Chrome MCP ratio decreases from 43.7% to 36.3% while pass rate improves. The objective is effective switching between action routes, not maximizing tool-call share.
• Qwen2.5-VL-7B starts at 48.9% pass rate and loses 8.9 points with experience augmentation. Its MCP ratio is 6.6%, compared with Qwen3's 43.7%. This supports basic tool competence as an important prerequisite, but the models differ in several ways, so this is not a single-variable ablation of native tool-call formatting.

Highlights & Insights

• Improve execution and route selection separately. Parameter learning can correct tool-calling behavior, while experience rules can recommend more reliable routes. The opposite effects of the same distillation approach across applications make this distinction practically relevant.
• Initial environment state is part of the training data. Automatic task creation expands interaction data only when paired with a verifiable environment. This design transfers naturally to agents that depend on files, databases, or application state.
• Compressed strategies transfer more readily than coordinates. Shortcuts and tool-chain rules preserve an action's purpose while discarding incidental interface positions. Capacity control and application filtering, however, remain primarily design claims without isolated quantitative validation.

Limitations & Future Work

• An author-acknowledged capability prerequisite. The automated process depends on a base model with native tool-calling competence; experience cannot create missing action capabilities from nothing. Process rewards and improved GUI distillation are possible extensions, not validated solutions in this paper.
• Small scale and primarily fixed-pool evolution. The main benchmark contains only 120 tasks, and profiles from seed tasks repeatedly guide evolution. This measures adaptation to a target pool, not broad desktop generalization. Hold-out evaluation covers Chrome with the same tool coverage, not new applications or tools.
• The teacher and judge share a model. Claude Sonnet 4 both demonstrates and evaluates, potentially introducing stylistic or judgment biases. Its 93% repeated-judgment agreement measures stability, not agreement with humans or programmatic completion checks.
• Incomplete automation and cost details. The cache lacks the cited supplement, preventing verification of environment validators, numerical bank capacities, isolated rejection-sampling gains, and end-to-end cost. “No fine-tuning” must not be equated with “no computation.”
• Reporting conventions require caution. Strict ordering claims for the hold-out table, the 17-step failure example versus the 10-step budget, and mixing peak metrics from different iterations limit strong conclusions. The main text also lacks a complete definition of its efficiency metric, so this note does not use it as a core reproducible comparison.

Related Work & Insights

• Compared with MCPWorld / OSWorld-MCP: These primarily measure the capabilities of API, GUI, and hybrid interaction; this paper uses capability profiles to generate targeted practice. It supplies an improvement process but not comprehensive gains across large established benchmarks.
• Compared with Agent Workflow Memory: Both reuse historical experience. This paper emphasizes skill-organized rules extracted from success-failure comparisons, application filtering, capacity merging, and an automatic environment-generation loop. It is not the first proposal for agent memory.
• Compared with STaR / Self-Instruct / ReST: It brings bootstrapping, synthesized tasks, and successful-sample selection into stateful desktop interaction. The additional challenge is ensuring executable environments, not merely plausible textual outputs.
• Compared with DigiRL: The primary mechanisms here are offline SFT and inference-time prompt augmentation, not online reinforcement learning directly optimizing environment returns. Selecting a learning mechanism by error type is a useful direction, but reliable automatic selection of the best mechanism still requires separate evaluation.

Rating

• Novelty: 4/5. Combining automatic environment generation, profile-guided practice, and experience learning has systems value, although most individual components have precedents.
• Experimental Thoroughness: 3/5. Cross-application tests, modality ablations, and hold-outs address key questions, but scale, isolated mechanism ablations, and cost accounting remain limited.
• Writing Quality: 3/5. The central argument is clear and negative results are useful; some statistical claims, step-budget conventions, and formal component descriptions are inconsistent.
• Value: 4/5. The paper offers useful deployment lessons for hybrid tool agents, especially the warning that expert distillation is not a universally effective upgrade.