Skip to content

SkillPyramid: A Hierarchical Skill Consolidation Framework for Self-Evolving Agents

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/xiongyuaay/skill_pyramid
Area: LLM Agent
Keywords: self-evolving agents, skill consolidation, hierarchical skill library, atomic extraction, abstract induction

TL;DR

SkillPyramid introduces a hierarchical skill consolidation and self-evolution framework that reorganizes flat skill libraries into a cohesive pyramidal topology featuring reusable low-level atomic operations and high-level abstract problem-solving patterns, enabling agents to compose, validate, and incrementally absorb new skills during execution to achieve a 38.0% average reward gain and a 27.7% step reduction across ALFWorld, WebShop, and ScienceWorld.

Background & Motivation

Recent developments in large language model (LLM) agents have enabled autonomous systems to tackle long-horizon, sequential decision-making tasks across interactive environments. In this operational paradigm, skillsโ€”encapsulated as callable units of capabilityโ€”serve as a vital substrate for bridging high-level intent with environment-specific actions. However, existing agent systems predominantly maintain skill collections as static, isolated, and flat repositories. As agents operate in open-ended deployment environments, curated skill sets inevitably grow stale, and agents struggle to accumulate experience into sustainable, transferrable assets.

This flat organizational design creates two severe architectural limitations. First, existing skills are typically reused as rigid, monolithic wholes, leaving the latent transferable sub-operations and structured problem-solving patterns embedded within them underexploited. Second, when confronted with unseen tasks that lack an exact one-to-one match in the repository, agents must explore from scratch, frequently failing due to sample inefficiency and error accumulation. Furthermore, prior exploration-based skill creation strategies rely either on ungrounded from-scratch generation or naive append-only logging, which quickly bloats the repository with redundant, noisy, and conflicting routines without establishing cross-task compositionality.

To overcome these bottlenecks, the skill collection must be reimagined from a passive storage pool into an active, self-evolving capability system. Complex interactive tasks routinely share underlying atomic execution primitives alongside recurring procedural schemas. By organizing skills into a multi-tier topological hierarchy, an agent can explicitly decouple fine-grained operational primitives from broad structural guidance. Core idea: organize agent skills into a hierarchical pyramid through bidirectional relation modelingโ€”extracting reusable atomic capabilities downward and inducing abstract schemas upwardโ€”while using task-driven two-stage composition and incremental self-evolution to turn static repositories into continually evolving cognitive systems.

Method

Overall Architecture

SkillPyramid restructures a flat skill library \(S=\{s_1, \dots, s_N\}\) into a multi-layer hierarchical pyramid \(\mathcal{P}=\{L_0, L_1, \dots, L_K\}\), where layer index \(\ell\) reflects abstraction: the bottom layer \(L_0\) captures fine-grained executable atomic skills, while the top layer \(L_K\) summarizes high-level procedural strategies. When a new task arrives, a dedicated creator agent retrieves relevant pyramid components to assemble a candidate skill via top-down composition. After real-world execution, validated skills are refined and incrementally integrated back into the pyramid using local relation modeling without requiring costly global reconstruction.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: Flat Skill Library S & Task Stream"] --> B["Coarse-to-Fine Relation Analysis (Relation Analyzer)"]
    B --> C["Bidirectional Relation Construction (Relation Builder)"]
    C --> D["Two-Stage Skill Composition (Skill Creator)"]
    D --> E["Task Execution & Closed-Loop Validation"]
    E -->|Validated Candidate| F["Incremental Topological Incorporation (Self-Evolution)"]
    E -->|Failed Trajectory| G["Discard Candidate Skill"]
    F --> H["Output: Continually Evolving Skill Pyramid P"]

Key Designs

1. Coarse-to-Fine Relation Analysis (Relation Analyzer): preventing computational explosion and semantic noise in all-to-all skill comparison Evaluating pairwise relations across the entire skill library simultaneously creates prohibitive context overhead and introduces severe noise from spurious surface correlations. The Relation Analyzer addresses this by employing a progressive two-stage screening process. First, a coarse-grained analyzer \(A_c\) inspects solely the skill names \(n_i\) and short descriptions \(d_i\), using embedding similarity and lexical heuristics to partition the repository into coarse candidate clusters \(G_0 = \{G_{0,1}, \dots, G_{0,M}\}\). Next, a fine-grained analyzer \(A_f\) processes each coarse group in parallel over multi-turn inspection rounds, analyzing the full structured natural language program (including prerequisites, operational workflows, and assertions) to produce definitive relation assignments \((g_k, \tau_k, e_k)\) equipped with concrete evidence and extraction guidance.

2. Bidirectional Relation Construction (Relation Builder): resolving skill isolation and decoupling execution primitives from abstract schemas Guided by the assignments dispatched from the analyzer, the Relation Builder constructs explicit topological links across two complementary operational axes: - Downward Atomic Extraction: For a subset \(g_k\) sharing concrete subroutines, the builder extracts their common sub-capability as an atomic skill \(a_k = B_{\text{atom}}(g_k)\) and positions it one level below the source skills (\(\ell(a_k) = \min_{s\in g_k}\ell(s) - 1\)). The source skills are rewritten as \(s'_i = \text{Rewrite}(s_i, \rho(a_k))\) by appending structured reuse references \(\rho(a_k) = (n_k, q_k, w_k, o_k)\) (denoting name, identifier, trigger conditions, and provided capability) to their dependency fields \(\delta\). This covers three concrete relation modes: SHARED_PART (extracting shared sub-steps), SUBSET (delegating to a strict subskill), and MERGE (canonicalizing near-identical implementations). - Upward Abstract Induction: For a subset \(g_k\) exhibiting shared procedural workflows, the builder induces an abstract skill \(u_k = B_{\text{abs}}(g_k)\) and places it one tier above (\(\ell(u_k) = \max_{s\in g_k}\ell(s) + 1\)). Rather than detailing low-level physical manipulations, abstract skills define reusable organizational patterns, subgoals, and decision gates, while referencing lower-level skills in their dependency fields to provide strategic top-level guidance.

3. Two-Stage Skill Composition (Skill Creator): mitigating hallucination and procedural drift in ungrounded skill synthesis When presented with a novel task instruction \(x\), generating executable routines from scratch frequently introduces hallucinations and invalid action sequences. The Skill Creator mitigates this by retrieving relevant skills \(R(x) \subseteq S\) from the pyramid and composing a targeted skill through a strictly disciplined two-stage protocol: - Framework Construction: High-level abstract skills are prioritized to delineate task categories, ordered subgoals, decision branches, and verifiable success criteria; - Detail Instantiation: Lower-level atomic skills are prioritized to instantiate executable environment actions, state validation routines, failure fallbacks, and explicit reuse references. If an existing skill fully satisfies the task requirement, it is directly reused; otherwise, the agent outputs a structured composite skill \(\hat{s} = C_{\text{create}}(x, R(x), \mathcal{P})\).

4. Incremental Topological Incorporation (Self-Evolution): avoiding catastrophic repository redundancy and global reconstruction latency To sustain adaptation in continuous deployment without rebuilding the entire pyramid upon every new task, SkillPyramid incorporates a closed-loop self-evolution mechanism. Generated candidate skills \(\hat{s}\) are gated strictly by task completion: failed attempts are immediately pruned, while successful candidates are trajectory-refined. To integrate \(\hat{s}\) into \(\mathcal{P}_t\), the system treats \(\hat{s}\) as a targeted query to retrieve only a localized subset of related existing skills. The Relation Builder then derives new bidirectional edges \(\Delta\mathcal{P}_t\) and runs structural integrity checks (enforcing top-down directed reuse and cycle avoidance) to yield an updated pyramid \(\mathcal{P}_{t+1} = \text{Update}(\mathcal{P}_t, \hat{s}, \Delta\mathcal{P}_t)\) at negligible computational overhead.

Loss & Training

SkillPyramid operates entirely in a training-free, in-context learning paradigm. The offline pyramid construction pipeline employs Qwen3.5-35B-A3B as the relation extraction and builder backbone, utilizing Qwen3-Embedding-4B for semantic vector retrieval during candidate grouping. The outer Analyzer-Builder iteration is capped at \(T=3\) rounds, while fine-grained group analysis is constrained to 5 rounds. For all downstream evaluations, backbone LLMs run deterministically at temperature 0, and newly generated skills are admitted into the active library only upon satisfying external environment reward gates.

Key Experimental Results

Main Results

SkillPyramid was comprehensively evaluated across three standard agent environments: embodied household manipulation (ALFWorld), scientific experimentation (ScienceWorld), and real-world web navigation (WebShop), powered by four diverse backbone models (DeepSeek-V3.2, GPT-4.1, Gemini 2.5 Pro, and Qwen3-235B). The table below summarizes task reward/success rate (R) and average interaction steps (S):

Backbone Model Method ALFWorld (Seen)
Rโ†‘ / Sโ†“
ALFWorld (Unseen)
Rโ†‘ / Sโ†“
WebShop
Rโ†‘ / Sโ†“
ScienceWorld (Seen)
Rโ†‘ / Sโ†“
ScienceWorld (Unseen)
Rโ†‘ / Sโ†“
DeepSeek-V3.2 ReAct 66.4 / 19.5 69.4 / 19.3 31.6 / 24.1 69.9 / 17.6 64.7 / 19.3
Reflexion 70.1 / 18.2 73.6 / 18.0 35.3 / 22.4 72.4 / 16.3 67.8 / 17.9
ExpeL 67.9 / 18.9 76.1 / 17.4 29.2 / 24.0 74.9 / 16.0 74.1 / 17.5
ReAct + Skills (Flat) 75.7 / 15.6 80.6 / 14.7 27.0 / 25.1 71.6 / 15.9 68.5 / 16.6
SkillPyramid (Ours) 81.4 / 10.7 84.8 / 10.6 32.0 / 23.9 76.3 / 16.7 79.1 / 17.0
GPT-4.1 ReAct 58.6 / 20.1 56.1 / 20.4 30.4 / 21.8 57.4 / 19.1 52.3 / 20.2
Reflexion 64.2 / 18.5 62.8 / 18.6 34.6 / 20.1 66.8 / 17.0 61.6 / 18.2
ExpeL 65.7 / 17.6 64.8 / 17.4 32.6 / 18.7 72.1 / 14.4 66.8 / 15.6
ReAct + Skills (Flat) 70.7 / 16.4 75.8 / 16.2 38.5 / 22.1 78.6 / 17.6 71.8 / 18.6
SkillPyramid (Ours) 77.1 / 15.1 85.1 / 13.1 42.5 / 15.3 86.6 / 15.2 88.6 / 14.2
Qwen3-235B ReAct 55.4 / 20.5 53.8 / 21.0 28.6 / 22.4 54.8 / 19.6 50.6 / 20.8
Reflexion 61.8 / 19.0 60.2 / 19.5 32.4 / 20.6 63.6 / 17.7 59.4 / 18.9
ExpeL 63.2 / 18.2 62.4 / 17.8 30.8 / 19.2 69.4 / 15.3 63.8 / 16.2
ReAct + Skills (Flat) 71.4 / 17.3 68.6 / 18.3 31.0 / 22.2 74.2 / 15.5 73.0 / 15.9
SkillPyramid (Ours) 77.1 / 13.7 91.0 / 11.0 38.0 / 18.4 85.6 / 14.0 86.3 / 14.3

On the ALFWorld unseen benchmark, SkillPyramid also outperforms dedicated skill libraries like SkillNet (83.6 / 14.8) and SkillX (79.9 / 16.6), reaching 84.8 / 10.6 under DeepSeek-V3.2. When coupled with GPT-5.5 Codex, SkillPyramid pushes performance from 88.1 (14.5 steps) to 91.8 (12.3 steps).

Ablation Study

Ablation experiments conducted on ALFWorld and ScienceWorld using DeepSeek-V3.2 demonstrate the critical role of each architectural component:

Configuration ALFWorld (Seen) Reward ScienceWorld (Seen) Reward Description
SkillPyramid (Full Model) 81.4 76.3 Full hierarchical organization and self-evolution
w/o Atomic Skills 78.6 75.3 Disables downward extraction (-2.8 on ALFWorld)
w/o Abstract Skills 80.0 71.1 Disables upward induction (-5.2 on ScienceWorld)
w/o Self-evolution 76.4 73.7 Freezes pyramid after initial construction
w/ Scratch-generated Skills 64.3 70.1 Generates skills without pyramid grounding; collapses below flat baseline
ReAct w/ flat skills (Flat Baseline) 75.7 71.6 Same initial skills without hierarchical consolidation
ReAct (Vanilla Baseline) 66.4 69.9 No external skill repository

Action-distribution task grouping further highlights component asymmetry: for atomic-heavy tasks (\(N=184\), high primitive action density), disabling atomic extraction causes performance to tumble from 82.6 to 76.1 (-6.5); conversely, for abstract-heavy tasks (\(N=147\), deep dependency chains), removing abstract induction causes a steep drop from 78.9 to 70.9 (-8.0).

Key Findings

  • Hierarchical consolidation outperforms quantity accumulation: Given identical initial skills, SkillPyramid attains an average reward of 73.7, distinctly surpassing ReAct+Skills (65.8). This confirms that performance leaps stem from topological restructuring and explicit reuse rather than expanded initial skill volume.
  • Superior out-of-distribution generalization: On unseen task splits across ALFWorld and ScienceWorld, SkillPyramid reduces interaction trajectory lengths by 27.7% while boosting rewards by up to 15 points over ReAct, demonstrating that abstract procedural schemas steer agents away from trial-and-error wandering.
  • Ungrounded generation injects severe noise: Generating skills completely from scratch drops ALFWorld performance to 64.3, falling significantly below the flat skill baseline (75.7). Grounding new skills in the verified pyramid structure is vital for reliable skill synthesis.

Highlights & Insights

  • Bidirectional Decoupling Paradigm: Decomposing monolithic skills into downward atomic primitives and upward abstract schemas resolves the long-standing trade-off between execution fidelity and cross-domain transferability.
  • Query-Centric Incremental Evolution: By using newly validated skills as query anchors to execute localized edge updates, SkillPyramid circumvents prohibitive all-to-all clustering and sustains long-term lifelong learning.
  • Robustness on Noisy Web-Mined Repositories: When applied to 2,271 filtered web-mined skills on GAIA-Lite, SkillPyramid elevates accuracy from 52.0% (no skills) and 56.0% (flat web skills) to 62.7% (4.8 steps), showcasing high practical value for noisy, real-world skill mining.

Limitations & Future Work

  • Initial Construction Footprint: The one-time bootstrap analysis requires substantial upfront API compute, and construction quality relies on moderately capable models (small 3B models lose significant performance during extraction).
  • Absence of Hard Rollback Mechanisms: Skill admission depends on immediate task reward signals; if an erroneous or suboptimal skill slips through a lucky success, the system lacks an automatic pruning mechanism to purge corrupted dependencies.
  • Scope and Modality Constraints: Current evaluations are centered on textual and symbolic environments; extending the hierarchical schema to continuous vision-language-action (VLA) robotics and modeling complex temporal/causal graphs represent key future horizons.
  • vs ReAct / Reflexion / ExpeL: ReAct provides step-by-step reasoning without persistent memory; Reflexion and ExpeL distill experiential insights into flat textual reflections. SkillPyramid transforms experience into structured programs connected via a directed dependency hierarchy.
  • vs Voyager / SkillNet / SkillX: Voyager builds flat executable skill libraries for single-sandbox domains (Minecraft); SkillNet and SkillX introduce unified management but lack bidirectional topological decomposition. SkillPyramid is the first to introduce upward abstraction and downward extraction with closed-loop incremental evolution.

Rating

  • Novelty: โญโญโญโญโญ [Pioneers a bidirectional hierarchical topology for agent skills with downward atomic extraction and upward abstract induction]
  • Experimental Thoroughness: โญโญโญโญโญ [Extensively tested across 4 major LLM backbones, 3 diverse interactive benchmarks, plus ablation, robustness, and web-mined skill transfer]
  • Writing Quality: โญโญโญโญโญ [Exemplary clarity and rigorous formalization, complete with reproducible prompt schemas and granular error analysis]
  • Value: โญโญโญโญโญ [Provides a practical, modular blueprint for scalable continual capability accumulation in self-evolving autonomous agents]