Skip to content

OpenSkill: Open-World Self-Evolution for LLM Agents

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/OpenLAIR/OpenSkill
Area: LLM Reasoning
Keywords: Self-Evolving Agents, Open-World Knowledge Acquisition, Supervision-Free Verification, Skill Synthesis, Cross-Model Transfer

TL;DR

Addressing the absence of curated skills and target-task feedback in realistic deployments, OpenSkill decouples open-world retrieval into domain knowledge acquisition and deterministic verification anchoring, synthesizes verifiable proxy pytest environments without target answers, and iteratively refines modular skill artifacts that transfer zero-shot across model families while attaining state-of-the-art pass rates.

Background & Motivation

Large language model (LLM) agents are increasingly tasked with solving open-ended, multi-step interactive challenges across dynamic software and physical environments. To sustain performance without periodic full-model retraining, self-evolving agents seek to acquire, refine, and retain executable knowledge artifacts after initial deployment. However, existing self-evolution frameworks operate under a pervasive and fragile idealization: they presuppose the presence of an intact learning loop, relying upon human-curated initial skill libraries, pre-existing collections of successful execution trajectories, or instant scalar feedback from ground-truth task verifiers and rewards. In realistic open-world deployment settings, an autonomous agent typically encounters nothing beyond a bare task instruction prompt—devoid of demonstrations, reference traces, or external evaluation oracles to gauge behavioral improvement.

This reliance on curated closed-world signals exposes two fundamental bottlenecks in real-world deployments. First, skill construction remains intrinsically constrained: models restricted to parametric internal knowledge inevitably hallucinate when confronted with newly released software APIs, project-specific coding idioms, or niche domain constraints; concurrently, methods that distill skills from successful trajectories fail catastrophically when an agent cannot generate a single passing trajectory on challenging tasks. Second, verification construction is severely compromised: common self-reflection and verbal reinforcement paradigms depend either on target-task reward feedback or introspective critique. Without gold test cases, target-task evaluation is unavailable during learning, while ungrounded self-critique suffers from severe confirmation bias, and querying the evaluation environment directly risks critical test suite data leakage.

To address the core challenge of whether an LLM agent can achieve autonomous self-evolution in open environments without target-task supervision, this work reframes the open world as both the repository of knowledge to be acquired and the foundation for constructing an independent practice environment. Core idea: Decouple agent self-evolution into dual-pathway open-world retrieval, deterministic virtual test suite construction, and diagnosis-guided refinement loops, enforcing a strict leakage barrier that reserves ground-truth supervision exclusively for final evaluation while producing explicit, interpretable skill artifacts that transfer zero-shot across model architectures.

Method

Overall Architecture

The OpenSkill pipeline formalizes the supervision-free self-evolution protocol through three sequential stages: Open-World Knowledge Acquisition, Leakage-Free Skill Evolution, and Zero-Shot Target Evaluation. Given only a task instruction \(I_i\) and execution environment \(E_i\), the system outputs a frozen skill set \(\hat{S}_i^*\) without inspecting the hidden ground-truth test suite \(T_i^{GT}\). The pipeline initiates with two orthogonal search routines retrieving task background documentation \(k_i\) and verification anchors \(k_i^v\). An isolated verifier agent synthesizes a deterministic virtual test suite \(\tilde{T}_i\) based on \(k_i^v\). Inside a containerized sandbox, an executor agent drafts and iteratively updates modular skill scripts across at most \(J=3\) cycles, guided by a diagnosis classifier that differentiates implementation bugs from external knowledge gaps. Upon achieving a virtual pass rate of 1.0 or exhausting iterations, the finalized skills are frozen and deployed zero-shot to downstream target agents.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Task Inputs (Instruction I_i + Environment E_i)"] --> B["Dual-Pathway Open-World Retrieval<br/>Gemini Deep Research (k_i) + Verification Anchors (k_i^v)"]
    B --> C["Skill Planning & Initial Generation<br/>Modular decomposition to skill drafts S_i^(0)"]
    C --> D["Virtual Test Suite Construction<br/>Independent verifier generates deterministic pytest assertions (T~_i)"]
    D --> E["Sandbox Execution & Pass Rate Scoring<br/>Supervision-free evaluation r~^(j)"]
    E -->|Virtual Pass Rate < 1.0| F["Diagnostic Failure Classification & Targeted Retrieval<br/>SELF-FIXABLE vs NEEDS-DR Classifier"]
    F -->|Knowledge Injection / Bug Fixing| C
    E -->|Virtual Pass Rate = 1.0 or Max J=3 reached| G["Final Skill Set Freeze S_i*"]
    G --> H["Zero-Shot Target Evaluation<br/>Cross-model deployment against hidden T_i^GT"]

Key Designs

1. Dual-Pathway Open-World Retrieval and Leakage Filtering: Resolving Parametric Knowledge Limits without Benchmark Contamination

To ensure comprehensive knowledge acquisition while preventing benchmark data contamination, OpenSkill segregates external queries into two functionally orthogonal pathways: general knowledge acquisition \(\mathcal{D}\) and verification knowledge retrieval \(\mathcal{D}_v\). An LLM Agent Planner inspects \(I_i\) and environment file metadata to construct search prompts focused strictly on documentation, library API signatures, parameter constraints, and known failure pitfalls, explicitly barring the generation of task-specific solution code. Before dispatch, all queries pass through a sanitize_query module that filters benchmark names and task ID variations to prevent indexing official evaluation repositories. The primary pathway leverages Gemini Deep Research (deep-research-pro-preview-12-2025) to generate a grounded background document \(k_i\), which a Skill Planner decomposes into 1–4 modular skill specifications \(p_i\). Concurrently, the verification pathway \(\mathcal{D}_v\) employs search-grounded Gemini 3.1 Flash Lite—treating the first 4,000 characters of \(k_i\) as an exclusion block—to mine four categories of independently verifiable external anchors \(k_i^v\): reference values computable from public small inputs, statistical invariants (e.g., column counts, probability distributions summing to one), cross-validation procedures via alternative software tools, and standard output format schemas.

2. Independent Verifier and Deterministic Virtual Test Generation: Mitigating Confirmation Bias in Supervision-Free Feedback

To evaluate synthesized skills without ground-truth oracles, OpenSkill deploys an Independent Verifier agent \(g(I_i, E_i, k_i^v)\) to construct a virtual test suite \(\tilde{T}_i = \{\tilde{t}_{i,1}, \dots, \tilde{t}_{i,K}\}\). To prevent self-confirmation bias, the verifier operates in a distinct LLM session completely decoupled from the skill creator's conversation context, reasoning traces, and code implementations, sharing only the containerized execution outputs. Using the verified anchors \(k_i^v\) as an external primary oracle, the verifier writes a deterministic pytest test suite capped at 60 assertions to emphasize rigorous correctness over volume. Each assertion \(\tilde{t}_{i,k} \in \{0, 1\}\) evaluates a deterministic invariant (e.g., exact equality of row counts or data types). A non-LLM parser executes the test suite to calculate the virtual pass rate:

\[\tilde{r}^{(j)} = \frac{1}{|\tilde{T}_i|} \sum_{k=1}^{|\tilde{T}_i|} \tilde{t}_{i,k}\left(\pi_\theta(I_i, \hat{S}_i^{(j)}, E_i)\right)\]

This proxy reward \(\tilde{r}^{(j)}\) guides evolutionary updates without requiring access to target ground-truth signals.

3. Diagnostic Failure Classification and Targeted Retrieval Loops: Decoupling Implementation Bugs from Domain Knowledge Deficits

When an intermediate skill fails virtual tests (\(\tilde{r}^{(j)} < 1.0\)), unguided prompt revision frequently produces repetitive, non-convergent edits. OpenSkill incorporates a lightweight diagnostic classifier (Gemini 3.1 Flash Lite, temperature 0.1) that ingests the failed assertion report, execution trace, and prior background documentation to categorize errors into SELF-FIXABLE or NEEDS-DR. If labeled SELF-FIXABLE (e.g., variable misnomers, syntax discrepancies, or off-by-one errors resolvable from current context), the host returns structured error traces and instructs the agent to edit skill code in place. If classified as NEEDS-DR (indicating missing mathematical formulas, undocumented API behaviors, or niche parameter requirements), the pipeline activates targeted search \(k_i^{(gap)} = \mathcal{D}(F^{(j)}, \mathcal{K})\), capped at 3 searches per task, injecting newly retrieved documentation into the refinement prompt. The evolution executes for at most \(J=3\) rounds, halting immediately if \(\tilde{r}=1.0\), thereby safeguarding against overfitting to virtual test suites.

Benchmark & Evaluation Protocol

The framework is evaluated across three diverse benchmarks: SkillsBench (spanning 11 domains including Software, Cybersecurity, Health, Energy, and Finance across 84 complex tasks where skill quality is the primary performance bottleneck), SocialMaze (social reasoning encompassing 6 subtasks: FTS, HRD, REFT, RDP, SGA, UPI), and ScienceWorld (interactive text-based science experiments). Experiments compare OpenSkill against seven closed-world automated baselines: No Skill, single-pass Self-Gen, five-step CoT, Anthropic Skill Creator, AutoSkill, Memento, and SkillNet (on ScienceWorld), alongside an expert Human-Curated upper bound. Target agents include Claude Opus 4.6 (Claude Code) and GPT 5.2 (Codex). All tasks run inside isolated Docker environments across \(n_{eval}=5\) independent zero-shot runs evaluated against the hidden ground-truth test suite \(T_i^{GT}\).

Key Experimental Results

Main Results

OpenSkill establishes new state-of-the-art automated performance across both primary target models on all three benchmarks, outperforming closed-world baselines and approaching human expert upper bounds.

Benchmark / Dataset Target Agent No Skill Strongest Automated Baseline OpenSkill (Ours) Human Reference Gain vs Strongest Baseline
SkillsBench (Overall) Opus 4.6 25.5% 34.7% (Skill Creator) 43.6% 44.5% +8.9%
SkillsBench (Overall) GPT 5.2 25.0% 33.3% (CoT) 42.1% 44.8% +8.8%
SocialMaze (Avg) Opus 4.6 81.6% 81.0% (Skill Creator) 82.7% - +1.1%
SocialMaze (Avg) GPT 5.2 66.6% 69.8% (Skill Creator) 70.7% - +0.9%
ScienceWorld (Avg) Opus 4.6 87.1% 88.7% (SkillNet) 90.0% - +1.3%
ScienceWorld (Avg) GPT 5.2 78.0% 83.1% (Skill Creator) 85.3% - +2.2%

In domain-specific breakdowns on SkillsBench, OpenSkill demonstrates marked superiority in knowledge-intensive domains: on Opus 4.6, Health reaches 69.6% (vs 24.8% No Skill, 31.2% Skill Creator) and Software attains 59.9% (exceeding Human at 38.8%); on GPT 5.2, Energy reaches 80.0% (exceeding Human at 53.3%) and Cybersecurity achieves 52.5% (exceeding Human at 42.5%).

Ablation Study

Ablations on SocialMaze using Opus 4.6 isolate the impact of individual architectural components and refinement iteration bounds:

Configuration Description Average Reward (%) Analysis & Relative Impact
Full Model (J=3) Open-world retrieval (DR) + Virtual Verifier (VV), 3 iterations 82.7% Optimal convergence point
Ablation: +VV Only Parametric knowledge with virtual verifier refinement 80.8% +6.3% over parametric baseline
Ablation: +DR Only Open-world retrieval without virtual verifier refinement 80.6% +6.1% over parametric baseline
Parametric Baseline (None) Pure parametric generation without retrieval or verifier 74.5% Lower performance floor
Iteration Sweep: J = 1 Single-cycle generation and test verification 81.2% Resolves initial syntactic faults
Iteration Sweep: J = 5 Extended to 5 refinement iterations 79.9% Slight degradation due to proxy overfitting
Iteration Sweep: J = 10 Extended to 10 refinement iterations 78.0% Severe overfitting to virtual assertion boundaries

Key Findings

  • High Cross-Model Transferability: Skills generated by Opus 4.6 transfer zero-shot to four distinct open and proprietary models (Haiku 4.5, Qwen 3 Coder, DeepSeek V3, and Mistral Large 3), delivering consistent performance improvements of +5.5% to +14.8% over the respective no-skill baselines without any model-specific recalibration. In contrast, skills generated by AutoSkill suffer severe negative transfer, demonstrating that OpenSkill produces model-agnostic, modular procedural knowledge.
  • Proxy Verifier Validity and Intent Coverage: Semantic analysis on randomly sampled ground-truth test suites reveals that the virtual verifier captures 88.9% (120/135) of human-authored test intents. Across all 84 SkillsBench tasks, proxy outcomes demonstrate a 60.7% overall agreement rate with hidden ground-truth evaluations (Fisher's exact test OR = 2.97, \(p = 0.035\), point-biserial \(r = 0.242, p = 0.027\)), exhibiting 56.9% precision and 80.5% recall.
  • Diminishing Returns and Overfitting in Self-Evolution: Expanding refinement rounds beyond \(J=3\) leads to monotonic performance degradation (82.7% at \(J=3 \rightarrow 79.9\%\) at \(J=5 \rightarrow 78.0\%\) at \(J=10\)). This decline stems from agents tailoring code strictly to idiosyncratic proxy assertions at the expense of generalizable target task execution logic.

Highlights & Insights

  • The Open World as a Dual-Purpose Substrate: Unlike standard RAG systems that query external documents purely for contextual question answering, OpenSkill repurposes the web and code repositories into both a curriculum source and an unsupervised testbed, unlocking autonomous self-improvement.
  • Deterministic Assertion Gating: Replacing subjective LLM-as-a-judge scorecards with executable pytest suites anchored in statistical invariants enforces binary, reproducible quality control that circumvents sycophantic self-critique loops.
  • Decoupled Artifact Portability: Preserving learned capabilities as human-readable Markdown and Python skill artifacts avoids opaque, model-bound parameter updates via RL, enabling one-time skill creation costs to amortize across multiple downstream client models.

Limitations & Future Work

  • Vulnerability to Noisy External Evidence: The framework's efficacy is bounded by retrieval precision; contradictory, deprecated, or malformed online documentation risks simultaneously corrupting both skill synthesis and virtual test construction.
  • Semantic Verification Boundaries: While the virtual verifier excels at syntactic, formatting, and structural sanity checks, it exhibits false positives on deep domain semantics (such as chemical graph isomorphism and complex algorithmic equivalence).
  • Future Directions: Developing multi-source cross-verification protocols to filter noisy documentation, extending virtual testing to multi-agent distributed systems, and implementing global cross-task skill deduplication to reduce token usage and API latency.
  • vs Voyager & Reflexion: Voyager relies on native simulation rewards in Minecraft, and Reflexion requires target task execution feedback; OpenSkill eliminates dependencies on target-task reward oracles by bootstrapping verification entirely from external documentation.
  • vs Skill Creator & AutoSkill: Anthropic Skill Creator operates within closed parametric bounds, while AutoSkill abstracts skills from conversational dialogue histories; OpenSkill integrates live open-world deep research and orthogonal proxy testing, driving superior generalization across complex technical domains.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Establishes a pioneering, supervision-free self-evolution framework for autonomous open-world agents.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive cross-domain evaluations spanning three benchmarks, two commercial backbones, and four transfer models.
  • Writing Quality: ⭐⭐⭐⭐⭐ Clear mathematical formulation, meticulous information isolation audits, and transparent failure mode dissections.
  • Value: ⭐⭐⭐⭐⭐ Offers an impactful, highly practical engineering blueprint for lifelong autonomous agent adaptation in production environments.