Skip to content

OpenSkill: Open-World Self-Evolution for LLM Agents

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/OpenLAIR/OpenSkill
Area: LLM Agent
Keywords: Agent Self-Evolution, Open-World Knowledge Acquisition, Virtual Task Verifier, Unsupervised Skill Refinement, Cross-Model Skill Transfer

TL;DR

OpenSkill establishes the first target-task-supervision-free framework for open-world LLM agent self-evolution by decoupling external knowledge retrieval into implementation guidance and objective verification anchors, generating deterministic virtual unit test suites to iteratively diagnose and refine modular skills that achieve state-of-the-art automated pass rates and transfer seamlessly across distinct model families.

Background & Motivation

Autonomous agents powered by Large Language Models (LLMs) are increasingly deployed in long-horizon, real-world software engineering, scientific research, and complex tool-use workflows. Because deployment environments diverge dynamically from training distributions, self-evolving agents aim to continuously adapt after initial release by discovering, curating, and accumulating reusable behavioral skills. However, current self-improvement paradigms universally presuppose a functional, pre-existing learning loop: they rely on human-authored skill libraries, expert-demonstrated successful trajectories, or environment-exposed task rewards and verifier signals (such as Reflexion, Skill Creator, and AutoSkill). In realistic open-world deployments, an agent is presented with nothing more than an initial seed task prompt and an empty sandbox, completely lacking initial skills, reference answers, or ground-truth reward evaluators.

This reality exposes two fatal architectural bottlenecks in contemporary agent self-evolution. First is the bounded ceiling of closed-world skill synthesis: approaches that generate skills in a single pass from parametric weights or reflective interaction traces can only reconfigure what the base model already knows, failing entirely when tasks require cutting-edge library APIs, private project conventions, or niche domain constraints. Second is the pervasive risk of supervision leakage in verification: tuning skills against target-task feedback risks overfitting or reverse-engineering evaluation suites, whereas ungrounded verbal self-critique suffers from severe LLM hallucinations and confirmation bias, rendering the feedback loop uncalibrated. Agents critically need a mechanism capable of acquiring external knowledge while concurrently bootstrapping a supervision-independent practice environment from scratch.

To overcome this supervision impasse, OpenSkill turns the open world into both the source of task knowledge and the substrate for objective verification, strictly walling off target-task ground truth until final deployment. Core idea: decouple open-world external resources into domain knowledge for modular skill drafting and objective verification anchors for deterministic virtual testing, orchestrating a leakage-free refinement loop that drives autonomous skill evolution and zero-shot cross-model transfer without any target-task supervision.

Method

OpenSkill formalizes the open-world self-evolution problem where, given only natural language instructions \(I_i\), execution environment \(E_i\), and external open-world resources \(\mathcal{K}\) (documentation, repositories, research papers, and web tutorials), the system synthesizes an explicit, inspectable skill set \(\hat{S}_i = \{\hat{s}_{i,1}, \dots, \hat{s}_{i,m}\}\) without updating model parameters \(\theta\). The entire pipeline guarantees that hidden target-task test suites \(T_i^{GT}\) remain strictly isolated behind an information barrier until final evaluation.

Overall Architecture

The framework executes across three distinct stages: Stage 1 performs dual-channel open-world knowledge acquisition to retrieve task domain knowledge alongside orthogonal verification anchors, structuring them into a modular skill plan; Stage 2 executes a leakage-free iterative evolution loop inside a sandboxed container, where an independent verifier runs deterministic assertions against agent outputs, diagnosing failures into code bugs or knowledge gaps to trigger in-place edits or targeted retrievals; Stage 3 freezes the evolved skill set \(\hat{S}_i^*\) and deploys it zero-shot to a target agent executing on the hidden ground-truth evaluation suite.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: Task Prompt & Environment<br/>(Ii, Ei)"] --> B["Dual-Channel Open-World Retrieval<br/>Domain Knowledge ki & Verification Anchors kiv"]
    B --> C["Skill Planning & Initial Drafting<br/>Structured Plan pi & Initial Skill Set Si(0)"]
    C --> D["Sandboxed Virtual Task Execution & Verification<br/>Independent Verifier emits deterministic pytest assertions"]
    D --> E{"Virtual Pass Rate<br/>r~(j) == 1.0 ?"}
    E -- "Fail (r~ < 1)" --> F["Diagnostic Refinement & Categorized Repair<br/>Classify Bug (SELF-FIXABLE) vs Gap (NEEDS-DR)"]
    F -->|Targeted Retrieval / In-Place Code Patch| D
    E -- "Pass or Max Iterations J=3" --> G["Zero-Shot Target Deployment & Evaluation<br/>Frozen Skill Si* injected into Target Agent ฯ€ฮธ'"]

Key Designs

The architecture is structured around four interlocking, contribution-level designs that govern knowledge intake, modularization, objective validation, and diagnostic iteration:

1. Dual-Channel Open-World Retrieval: Decoupled Knowledge Acquisition and Verification Anchoring

Standard retrieval-augmented pipelines perform monolithic document queries that conflate operational implementation recipes with evaluation standards, creating noisy prompts and reinforcing model confirmation bias. OpenSkill resolves this by introducing a dual-channel orthogonal retrieval mechanism. The primary retriever \(\mathcal{D}\) queries the open world using an asynchronous deep-research agent to collect background concepts, official API signatures, parameter defaults, and usage pitfalls into a domain knowledge artifact \(k_i \subset \mathcal{K}\). Concurrently, an orthogonal verification retriever \(\mathcal{D}_v\) executes a search-grounded call strictly constrained to identify external, independently verifiable "verification anchors" \(k_i^v\). These anchors are restricted to four invariant categories: hand-computable reference values for miniature inputs, dataset-level statistical invariants (row counts, sum-to-one constraints, monotonicity, valid metric boundaries), published library cross-validation procedures, and canonical input-output benchmark pairs. Crucially, all queries through \(\mathcal{D}\) and \(\mathcal{D}_v\) pass through a Leakage Filtering sanitizer that strips benchmark names and task identifiers, preventing test-suite memorization and ensuring authentic grounding.

2. Skill Planning & Initial Drafting: Modular Architecture Synthesis and Context Slicing

Attempting to generate an monolithic, all-encompassing skill script for intricate long-horizon tasks frequently overwhelms model context windows, precipitating repetitive tool calls and code hallucination. OpenSkill deploys an LLM Skill Planner that inspects the instruction \(I_i\), environment preview \(E_i\), and retrieved domain knowledge \(k_i\) to decompose the objective into a modular skill plan \(p_i\) comprising \(m\) distinct skills (\(1 \le m \le 4\)). Each skill specification delineates an explicit operational boundary, primary function signatures, and targeted dependencies on specific sections of \(k_i\). The planner performs fuzzy header matching to slice \(k_i\) into localized reference packets for each component, enabling the base agent \(\pi_\theta\) to draft an initial modular skill set \(\hat{S}_i^{(0)} = \{\hat{s}_{i,1}^{(0)}, \dots, \hat{s}_{i,m}^{(0)}\}\). This decoupled modular design reduces syntactic complexity and allows subsequent iterative repairs to target isolated functions without destabilizing adjacent logic.

3. Sandboxed Virtual Task Execution & Verification: Isolated Deterministic Assertion Testing

Evaluating newly synthesized code through ungrounded LLM self-scoring induces pervasive optimism, lenient grading, and confirmation bias. OpenSkill constructs an Independent Verifier \(g(I_i, E_i, k_i^v)\) running in an isolated session within the task's Docker container. The verifier shares filesystem access to the execution output artifacts generated by the skill creator but possesses zero visibility into the creator's conversational context, reasoning traces, or implementation code. Treating the retrieved verification knowledge \(k_i^v\) as its primary truth oracle, the verifier synthesizes a deterministic pytest test suite \(\tilde{T}_i = \{\tilde{t}_{i,1}, \dots, \tilde{t}_{i,K}\}\) (capped at 60 assertions) composed exclusively of hard equality and relational checks (assert output == expected). The proxy quality metric is computed deterministically as the virtual pass rate: $\(\tilde{r}^{(j)} = \frac{1}{|\tilde{T}_i|} \sum_{k=1}^{K} \tilde{t}_{i,k}(\pi_\theta(I_i, \hat{S}_i^{(j)}, E_i))\)$ The refinement loop exits immediately with surrogate_pass if \(\tilde{r}^{(j)} = 1.0\), replacing subjective qualitative heuristics with reproducible empirical verification.

4. Diagnostic Refinement & Categorized Repair: Root-Cause Classification and Targeted Gap Retrieval

When virtual execution fails (\(\tilde{r}^{(j)} < 1\)), naive multi-turn prompting that re-injects entire failure traces causes the agent to cycle endlessly on missing domain knowledge or inadvertently break working sub-components. OpenSkill introduces a structured failure diagnostic \(F^{(j)}\) coupled with a binary root-cause classifier. The classifier reads failed assertions, traceback logs, and prior domain knowledge to categorize the error into either SELF-FIXABLE (implementation bugs such as typos, off-by-one errors, formatting mismatches, or type discrepancies) or NEEDS-DR (fundamental knowledge gaps requiring external domain information, such as unknown parameter ranges or specialized algorithm usage). Under SELF-FIXABLE, the system provides targeted assertion tracebacks, instructing the creator to patch the skill script in place without direct output editing. Under NEEDS-DR, the system invokes a targeted retrieval pass \(k_i^{(gap)} = \mathcal{D}(F^{(j)}, \mathcal{K})\) (budgeted at at most 3 searches per task) to fetch the precise missing documentation and inject it into the refinement prompt. This diagnostic cycle iterates for up to \(J=3\) rounds, bounded by stale-episode and token-budget safeguards to ensure rapid convergence.

A Worked Example

Consider an intricate biomedical data processing task from SkillsBench requiring gene sequence alignment and statistical feature matrix extraction: 1. Knowledge & Anchor Retrieval: The primary retriever gathers Biopython API usage guides and scoring parameters (\(k_i\)). The orthogonal verifier retriever fetches invariant properties (\(k_i^v\)): output similarity matrices must be symmetric, diagonal values must equal 1.0, and a known 4-base test sequence must produce an exact alignment score of 12. 2. Planning & Drafting: The planner structures two modular skills: sequence_preprocessor and alignment_matrix_calculator, slicing relevant documentation headers to prompt initial drafting. 3. Isolated Virtual Testing: The independent verifier authors 18 deterministic pytest assertions. On execution, 16 assertions pass but 2 fail (\(\tilde{r}^{(0)} = 16/18 \approx 0.889\)): matrix symmetry is violated on odd-length sequences due to an index boundary error. 4. Root-Cause Diagnosis & In-Place Repair: The diagnostic classifier marks the error as SELF-FIXABLE. The creator agent receives the failing assertion traceback and modifies the indexing logic in place inside the container sandbox. 5. Convergence & Deployment: Re-running pytest achieves a 100% virtual pass rate (\(\tilde{r}^{(1)} = 1.0\)). The evolution loop terminates immediately, freezing the finalized skill \(\hat{S}_i^*\) for zero-shot deployment on the target agent.

Key Experimental Results

Main Results

The primary evaluation spans SkillsBench (Li et al., 2026), encompassing 11 domain categories where skill quality is the dominant performance bottleneck. Evaluations were conducted across two premier LLM target backbones: Claude Opus 4.6 (Claude Code) and GPT 5.2 (Codex), reporting average reward (pass rate, %) over hidden test suites under 5 independent runs.

Domain Target Agent No Skill Self-Gen CoT Skill-Creator AutoSkill Memento OpenSkill (Ours) Human (Upper Bound)
Software Opus 4.6 32.6 37.9 34.9 51.3 36.0 34.4 59.9 38.8
Office Opus 4.6 17.0 16.7 17.1 21.4 25.7 31.4 50.0 50.0
Cybersecurity Opus 4.6 17.8 18.8 20.4 24.6 16.6 28.8 44.1 55.0
Health Opus 4.6 24.8 19.8 19.2 31.2 14.5 25.0 69.6 80.0
Energy GPT 5.2 0.0 33.3 55.6 22.2 11.0 22.3 80.0 53.3
Cybersecurity GPT 5.2 25.0 20.8 34.7 36.8 4.1 12.5 52.5 42.5
Overall (11 Domains) Opus 4.6 25.5 23.9 23.9 34.7 24.7 30.1 43.6 44.5
Overall (11 Domains) GPT 5.2 25.0 32.2 33.3 29.2 11.2 15.6 42.1 44.8

Across distinct benchmark archetypes (average reward %): - SocialMaze (Social Reasoning): Opus 4.6 reaches 82.7% (surpassing No Skill 81.6% and Skill Creator 81.0%); GPT 5.2 reaches 70.7% (substantially outperforming Skill Creator 69.8% and Self-Gen 67.6%). - ScienceWorld (Interactive Scientific Experiments): Opus 4.6 achieves 90.0% (outperforming SkillNet 88.7% and Skill Creator 88.3%); GPT 5.2 achieves 85.3% (outperforming Skill Creator 83.1% and CoT 80.0%).

Ablation Study

Component contributions and iteration dynamics evaluated on SocialMaze using Claude Opus 4.6 (Figure 4 in the paper):

Configuration / Variable Average Reward (%) Net Gain over Base Description / Mechanism
Parametric-Only Baseline (No OW / No VV) 74.5 - Single-pass drafting relying solely on frozen parametric weights
+ Open-World Query (OW only) 80.6 +6.1% External API documentation and factual grounding added
+ Virtual Verifier (VV only) 80.8 +6.3% Deterministic proxy pytest assertions driving code repair
+ Both (Full OpenSkill Pipeline) 82.7 +8.2% Synergistic combination of open knowledge and verification
Refinement Iterations \(J=1\) 81.2 - Single round refinement, residual bugs remain
Refinement Iterations \(J=3\) (Default) 82.7 - Optimal empirical balance point
Refinement Iterations \(J=5\) 79.9 -2.8% Performance degradation due to virtual test overfitting
Refinement Iterations \(J=10\) 78.0 -4.7% Over-specialization on imperfect proxy assertions

Key Findings

  • Unprecedented Automated Pass Rates Near Human Ceilings: OpenSkill achieves the highest automated pass rate across all three benchmarks and both primary agent families, scoring 43.6% on Opus 4.6 (+8.9% over Skill-Creator) and 42.1% on GPT 5.2 (+8.8% over CoT), trailing expert human-curated skills by only 0.9โ€“2.7 percentage points without observing any target supervision.
  • Robust Model-Agnostic Zero-Shot Transfer: Skills synthesized by Opus 4.6 transfer directly onto four distinct, significantly smaller target models on SkillsBench without any model-specific prompt adaptation (Figure 3): Haiku 4.5 improves from 10.4% to 20.7% (+10.3%), Qwen 3 Coder rises from 19.4% to 26.6% (+7.2%), DeepSeek V3 jumps from 8.4% to 23.2% (+14.8%), and Mistral Large 3 increases from 11.7% to 17.9% (+6.2%). In stark contrast, AutoSkill-generated skills collapsed below the no-skill baseline across all four transferred architectures.
  • Statistically Significant Verifier Alignment: Across 84 sampled tasks, the Virtual Verifier demonstrated a 60.7% agreement rate with ground-truth outcomes (Fisher's exact test odds ratio \(OR = 2.97, p = 0.035\); point-biserial \(r = 0.242, p = 0.027\)). In semantic intent audits across 15 tasks, virtual test suites covered 88.9% (120 of 135) of human-authored test intents while generating 3.4ร— more defensive boundary assertions.

Highlights & Insights

  • Decoupled Knowledge vs. Verification Anchoring: Rather than executing an undifferentiated web search, separating operational library documentation from mathematical and invariant verification anchors ensures that agents construct an objective self-checking harness rather than an echo chamber.
  • Executable Deterministic Assertions over LLM Grading: Replacing subjective verbal reflection with sandboxed pytest suites ensures that every iteration is governed by hard, reproducible equality constraints, eliminating evaluation drift and scoring inflation.
  • One-Time Synthesis for Multi-Target Deployment: Treating skills as inspectable, modular code artifacts decouples the synthesis cost from downstream execution, enabling expensive deep-research reasoning to be amortized across diverse, cost-effective inference models.

Limitations & Future Work

  • Author-Acknowledged Limitations: Open-world web sources can be noisy, obsolete, or contradictory, requiring robust provenance verification; the virtual verifier experiences a ~29.8% false positive rate (Proxy Pass, GT Fail) due to missing subtle edge cases or anti-cheat harness checks, and excessive iterations (\(J > 3\)) induce overfitting to surrogate metrics.
  • Identified Failure Regimes: In physical/manufacturing domains relying on domain-specific simulation mechanics (Manufacturing recorded 0.0% across all automated baselines), purely textual web search and general container sandboxes fail to synthesize functional procedural actions.
  • Future Directions: Integrating multimodal and environmental execution state tracing to infer causal invariants; developing multi-agent adversarial test-generation debates to harden virtual test suites against false positives; deploying provenance-weighted source aggregation to suppress noisy web data.
  • vs. Reflexion (Shinn et al., 2023): Reflexion requires real-time scalar reward feedback or execution status directly emitted by the target benchmark; OpenSkill operates strictly under zero target-task supervision, synthesizing both the practice tasks and evaluation metrics autonomously from the open world.
  • vs. Voyager (Wang et al., 2023) / AutoSkill (Yang et al., 2026): Voyager and AutoSkill rely on historical execution traces and internal parametric priors, resulting in skills tightly coupled to the originating model; OpenSkill acquires non-parametric external knowledge, yielding modular skills that generalize across completely different model families.
  • vs. CoEvoSkills (Zhang et al., 2026a): CoEvoSkills co-evolves skills and verifiers from shared internal model priors, incurring risks of collusion and mutual hallucination; OpenSkill grounds its verifier in external, invariant anchors discovered from the open world to ensure objective evaluation.

Rating

  • Novelty: โญโญโญโญโญ Formulates the supervision-free open-world agent self-evolution paradigm and introduces an elegant dual-channel knowledge/verification decoupling mechanism.
  • Experimental Thoroughness: โญโญโญโญโญ Rigorous validation across 11 SkillsBench domains, two additional benchmark suites, two target model families, 4 zero-shot transfer architectures, and extensive statistical confidence/bootstrap testing.
  • Writing Quality: โญโญโญโญโญ Clear problem formalization, rigorous multi-layer information isolation auditing, and deep failure-mode diagnostic analysis.
  • Value: โญโญโญโญโญ Establishes a highly practical engineering blueprint for deploying autonomous self-improving agents in uncurated real-world software and scientific environments.