Skip to content

SkillPyramid: A Hierarchical Skill Consolidation Framework for Self-Evolving LLM Agents

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/xiongyuaay/skill_pyramid
Area: LLM Reasoning
Keywords: Agent Skills, Hierarchical Consolidation, Self-Evolving Agents, Embodied Reasoning, Skill Reuse

TL;DR

Addressing the inability of flat skill libraries to support structured reuse and cross-task generalization in LLM agents, SkillPyramid proposes a bidirectional hierarchical consolidation mechanism (downward atomic extraction and upward abstract induction) paired with decoupled synthesis and task-validated incremental self-evolution, improving average reward by 38.0% while reducing execution steps by 27.7%.

Background & Motivation

Recent breakthroughs in large language models (LLMs) have driven a paradigm shift in autonomous agent systems from passive conversational responders to proactive problem solvers in complex physical simulators and web environments. Within this operational shift, skills—structured, callable capability units encapsulating decision procedures, environmental preconditions, and executable action sequences—serve as an indispensable foundation for tackling long-horizon and open-ended problems. Whether through external tool augmentation or experiential trajectory summarization, maintaining an expressive repository of reusable skills has become standard practice for amortizing exploration costs across tasks. However, existing skill-management architectures predominantly store skills as flat, isolated collections, creating fundamental scalability barriers during lifelong agent deployment.

The critical limitation of flat skill libraries lies in their inability to systematically consolidate, organize, and transfer experiential knowledge across tasks. Because existing systems treat skills as isolated black-box blocks to be retrieved and injected wholesale, the rich, transferable operational knowledge buried inside concrete skill bodies remains underexploited. When an agent confronts an out-of-distribution or novel task that lacks an exact keyword or semantic match in its repository, a flat library offers no partial match guidance, forcing the agent to revert to trial-and-error exploration from scratch in vast action spaces, which frequently culminates in execution failure. Concurrently, agents redundantly synthesize similar low-level primitives across separate task requests, yielding library bloat without crystallizing high-level generalizable problem-solving schemas.

This limitation stems from a fundamental structural tension: concrete execution demands fine-grained, executable micro-actions, whereas high-level multi-step planning requires abstracted procedural schemas, and a single-layer flat repository cannot satisfy both demands simultaneously. The angle of attack proposed in this paper is to dismantle the isolated boundaries of stored skills, factorizing shared execution primitives into reusable atomic building blocks while inducing recurring high-level workflows into abstract schemas. Core idea: construct a hierarchical skill topology, SkillPyramid, by downward atomic extraction of minimal reusable capabilities and upward abstract induction of generalizable task schemas, enabling decoupled task-driven skill synthesis and closed-loop lifelong self-evolution via execution validation.

Method

Overall Architecture

SkillPyramid transforms a static, flat skill library into a dynamic, hierarchically structured system capable of continuous adaptation. Given an initial skill collection \(S=\{s_1, s_2, \dots, s_N\}\), the framework employs a collaborative multi-agent architecture to construct a \((K+1)\)-level skill pyramid \(P=\{L_0, L_1, \dots, L_K\}\), where layers span from fine-grained atomic primitives at \(L_0\) to abstract task patterns at \(L_K\). When faced with an incoming stream of task instructions, the system retrieves relevant sub-pyramids, synthesizes a candidate skill via a decoupled two-stage strategy, and—upon passing real environmental verification—incrementally integrates the validated skill and its dependencies into the topological hierarchy without costly full-library reconstruction.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Task Instruction x & Initial Flat Library S0"] --> B["Bidirectional Hierarchical Consolidation<br/>Atomic Extraction ak & Abstract Induction uk"]
    B --> C["Decoupled Hierarchical Skill Creation<br/>Framework Construction & Detail Instantiation"]
    C --> D["Environment Interaction & Task Outcome Validation"]
    D -->|Task Success & New Skill Generated| E["Task-Driven Incremental Self-Evolution<br/>Trajectory Refinement & Topology Insertion"]
    D -->|Task Failure| F["Discard Candidate Skill & Backtrack Action"]
    E --> G["Updated Skill Pyramid Pt+1"]

Key Designs

1. Bidirectional Hierarchical Consolidation: Bottom-Up Induction and Top-Down Extraction

To resolve the issue where flat skill repositories duplicate redundant sub-actions and fail to expose structural patterns, the system deploys a Relation Analyzer and a Relation Builder operating in a coarse-to-fine sequence. The Relation Analyzer first groups skills into candidate subsets \(G_0=\{G_{0,1}, \dots, G_{0,M}\}\) using only high-level names and descriptions, followed by fine-grained inspection that classifies relationships into shared sub-capabilities (SHARED_PART), subset specialization (SUBSET), or canonical near-equivalences (MERGE). In Downward Atomic Extraction, the Relation Builder extracts the common executable operation shared by group \(g_k\) to form an atomic skill \(a_k = B_{\text{atom}}(g_k)\), placing it at a lower tier \(\ell(a_k) = \min_{s \in g_k} \ell(s) - 1\) and rewriting the original source skills to delegate that sub-action via an explicit reuse reference \(\rho(a_k)\). In Upward Abstract Induction, the builder synthesizes the shared solving strategy across \(g_k\) into an abstract skill \(u_k = B_{\text{abs}}(g_k)\) positioned at an elevated tier \(\ell(u_k) = \max_{s \in g_k} \ell(s) + 1\). By augmenting each skill representation with an explicit dependency field \(\delta\), this bidirectional structuring unifies micro-level execution fidelity with macro-level task decomposition.

2. Decoupled Hierarchical Skill Creation: Framework Construction and Detail Instantiation

Synthesizing an executable skill from scratch for an unseen task instruction \(x\) often triggers hallucinated actions or brittle execution chains. The Skill Creator resolves this by executing a two-stage decoupled synthesis conditioned on the retrieved pyramid context \(R(x) \subseteq S\). In the first stage—Framework Construction—the retrieval mechanism prioritizes higher-level abstract skills to derive the general problem-solving blueprint, defining the task class, ordered subgoals, decision bifurcations, and termination conditions without committing to concrete low-level parameters. In the second stage—Detail Instantiation—the retrieval focuses on bottom-level atomic skills to populate the skeletal framework with concrete environmental actions, required contextual inputs, state-check assertions, and explicit delegation references formatted as [reuse skill: ... | when: ... | provides: ...]. If an existing skill in \(P\) already completely covers the task, the agent directly reuses it; otherwise, it composes a grounded candidate skill \(\hat{s} = C_{\text{create}}(x, R(x), P)\), effectively transforming open-ended synthesis into constrained multi-level assembly.

3. Task-Driven Incremental Self-Evolution: Trajectory-Gated Refinement and Local Topological Insertion

To allow the skill repository to continually adapt across open-ended task streams while safeguarding against knowledge corruption, the framework implements a strict trajectory-validated self-evolution loop. When an agent deploys a newly synthesized candidate skill \(\hat{s}\) in the environment, the candidate is admitted into the permanent knowledge base only if the task successfully achieves a positive reward signal; upon success, the skill definition is refined and consolidated from the actual execution trace \(\tau\), whereas failed candidates are immediately discarded. During admission, rather than re-indexing the global library, the Relation Analyzer treats \(\hat{s}\) as a targeted query to retrieve nearby topological candidates, prompting the Relation Builder to infer local reuse relations. Structural validation constraints enforce strict acyclicity (preventing reverse cross-tier dependencies and self-loops) and permit skill merges only under functional equivalence of triggers and parameters. The incremental operator \(P_{t+1} = \text{Update}(P_t, \hat{s}, \Delta P_t)\) guarantees that knowledge expansion incurs low local overhead while continuously enriching the pyramid.

Benchmark & Evaluation Protocol

The evaluation is conducted across three diverse benchmarks: ALFWorld (140 in-distribution seen tasks, 134 out-of-distribution unseen tasks spanning long-horizon household chores), ScienceWorld (194 seen and 211 unseen variations across 24 elementary science curricula requiring complex instrument manipulation and multi-step state tracking), and WebShop (200 real-world e-commerce product search and purchase scenarios). Experiments evaluate four representative backbone LLMs: DeepSeek-V3.2, GPT-4.1, Gemini 2.5 Pro, and Qwen3-235B, all evaluated at temperature 0 to ensure deterministic and reproducible outputs. In addition, the auxiliary construction agents utilize Qwen3.5-35B-A3B alongside Qwen3-Embedding-4B for coarse clustering, validating that the hierarchical consolidation pipeline does not rely on proprietary frontier LLMs to organize the knowledge base.

Key Experimental Results

Main Results

The primary benchmark results compare ReAct, Reflexion, ExpeL, ReAct+Skills (flat initial library), and SkillPyramid across the three environments and four LLM backbones, reporting task reward / success rate (\(R \uparrow\)) and average interaction steps (\(S \downarrow\)).

Backbone LLM Method ALFWorld (Seen)
\(R \uparrow\) / \(S \downarrow\)
ALFWorld (Unseen)
\(R \uparrow\) / \(S \downarrow\)
WebShop (Seen)
\(R \uparrow\) / \(S \downarrow\)
ScienceWorld (Seen)
\(R \uparrow\) / \(S \downarrow\)
ScienceWorld (Unseen)
\(R \uparrow\) / \(S \downarrow\)
DeepSeek-V3.2 ReAct 66.4 / 19.5 69.4 / 19.3 31.6 / 24.1 69.9 / 17.6 64.7 / 19.3
Reflexion 70.1 / 18.2 73.6 / 18.0 35.3 / 22.4 72.4 / 16.3 67.8 / 17.9
ExpeL 67.9 / 18.9 76.1 / 17.4 29.2 / 24.0 74.9 / 16.0 74.1 / 17.5
ReAct + Skills 75.7 / 15.6 80.6 / 14.7 27.0 / 25.1 71.6 / 15.9 68.5 / 16.6
SkillPyramid (Ours) 81.4 / 10.7 84.8 / 10.6 32.0 / 23.9 76.3 / 16.7 79.1 / 17.0
GPT-4.1 ReAct 58.6 / 20.1 56.1 / 20.4 30.4 / 21.8 57.4 / 19.1 52.3 / 20.2
Reflexion 64.2 / 18.5 62.8 / 18.6 34.6 / 20.1 66.8 / 17.0 61.6 / 18.2
ExpeL 65.7 / 17.6 64.8 / 17.4 32.6 / 18.7 72.1 / 14.4 66.8 / 15.6
ReAct + Skills 70.7 / 16.4 75.8 / 16.2 38.5 / 22.1 78.6 / 17.6 71.8 / 18.6
SkillPyramid (Ours) 77.1 / 15.1 85.1 / 13.1 42.5 / 15.3 86.6 / 15.2 88.6 / 14.2
Gemini 2.5 Pro ReAct 60.0 / 18.7 61.9 / 19.2 31.7 / 22.1 58.2 / 18.4 56.1 / 19.1
Reflexion 67.2 / 17.3 69.4 / 17.7 38.2 / 19.9 66.8 / 16.9 63.2 / 17.5
ExpeL 68.6 / 17.9 70.2 / 17.0 33.1 / 19.3 72.8 / 15.0 67.4 / 14.9
ReAct + Skills 75.0 / 17.0 73.9 / 16.7 33.0 / 18.7 77.3 / 16.7 79.1 / 17.0
SkillPyramid (Ours) 73.6 / 16.0 80.6 / 11.9 35.0 / 18.2 89.7 / 9.0 82.9 / 14.3
Qwen3-235B ReAct 55.4 / 20.5 53.8 / 21.0 28.6 / 22.4 54.8 / 19.6 50.6 / 20.8
Reflexion 61.8 / 19.0 60.2 / 19.5 32.4 / 20.6 63.6 / 17.7 59.4 / 18.9
ExpeL 63.2 / 18.2 62.4 / 17.8 30.8 / 19.2 69.4 / 15.3 63.8 / 16.2
ReAct + Skills 71.4 / 17.3 68.6 / 18.3 31.0 / 22.2 74.2 / 15.5 73.0 / 15.9
SkillPyramid (Ours) 77.1 / 13.7 91.0 / 11.0 38.0 / 18.4 85.6 / 14.0 86.3 / 14.3

On the ALFWorld unseen split, direct comparisons against concurrent skill systems using DeepSeek-V3.2 show: SkillNet scores \(R=83.6, S=14.8\); SkillX reaches \(R=79.9, S=16.6\); while SkillPyramid secures \(R=\mathbf{84.8}, S=\mathbf{10.6}\), achieving superior task performance with 4.2 to 6.0 fewer environment interaction turns.

Ablation Study & Empirical Analysis

Ablation experiments conducted on DeepSeek-V3.2 evaluate the specific necessity of atomic extraction, abstract induction, and self-evolution on ALFWorld and ScienceWorld.

Model Variant ALFWorld Reward (\(R\)) ScienceWorld Reward (\(R\)) Ablation Description & Impact
Full SkillPyramid 81.4 76.3 Complete framework with bidirectional hierarchy and self-evolution
w/o Atomic Skills 78.6 75.3 Disables downward extraction; source skills retain verbose subroutines
w/o Abstract Skills 80.0 71.1 Disables upward induction; procedural reasoning in ScienceWorld drops heavily
w/o Self-evolution 76.4 73.7 Freezes pyramid after initial construction; fails to adapt to task progression
w/ Scratch-generated 64.3 70.1 Generates new skills without pyramid grounding; degrades below flat baseline
ReAct w/ flat skills 75.7 71.6 Flat baseline equipped with identical initial skill repository
ReAct (Vanilla) 66.4 69.9 Standard text-action prompting without external skill integration

To reveal why atomic and abstract skills exhibit asymmetric contributions across benchmarks, tasks were partitioned prior to ablation using oracle trajectory statistics: low-level action mass \(m_{\text{low}}\) (primitive execution ratio) and dependency chain length \(\ell_{\text{dep}}\):

Task Grouping # Tasks \(m_{\text{low}}\) \(\ell_{\text{dep}}\) Full w/o Atomic (Drop) w/o Abstract (Drop)
Atomic-heavy Tasks 184 0.93 1.96 82.6 76.1 (-6.5) 80.2 (-2.4)
Abstract-heavy Tasks 147 0.43 5.14 78.9 76.8 (-2.1) 70.9 (-8.0)

Key Findings

  • Asymmetric Component Utility: Downward atomic extraction and upward abstract induction serve distinct functional roles. Atomic skills drive performance in manipulation-heavy domains with high primitive density (dropping 6.5 points on atomic-heavy tasks when removed), whereas abstract skills are indispensable for deep procedural synthesis across multi-stage experiments (dropping 8.0 points on abstract-heavy tasks).
  • Hazard of Ungrounded Skill Synthesis: Generating skills from scratch without anchoring in the pyramid degrades ALFWorld reward to 64.3—substantially below the flat-skill baseline of 75.7. Unconstrained LLM skill synthesis introduces ungrounded actions and hallucinatory preconditions, confirming that hierarchical grounding is mandatory for reliable capability generation.
  • Robustness on Noisy Web-Mined Data: When applied to 2,271 filtered skills mined from 150K raw web candidates evaluated on GAIA-Lite (75 diverse assistant tasks), the unaugmented baseline scores 52.0% accuracy (5.2 steps), flat web skills reach 56.0% (5.9 steps), and SkillPyramid achieves 62.7% (4.8 steps). This demonstrates that hierarchical consolidation successfully filters and structures noisy real-world skill data beyond synthetic benchmarks.

Highlights & Insights

  • Transitioning from a Static Repository to an Active Topology: By formalizing explicit reuse references \(\rho(a_k)\) within a multi-tiered hierarchy, SkillPyramid eliminates redundant sub-action implementation across skills and enables modular composition analogous to software library linking.
  • Two-Stage Decoupled Skill Synthesis: Disentangling framework outline construction (driven by abstract schemas) from detail instantiation (driven by executable atomic primitives) prevents cognitive overload in the planning LLM, ensuring both macro-level strategic coherence and micro-level execution validity.
  • Local Topology Insertion for Continual Self-Evolution: Rather than relying on global re-clustering or brute-force flat appending, treating validated candidate skills as query probes for localized subgraph updates ensures lifelong adaptation with bounded computational overhead.

Limitations & Future Work

  • Initial Construction Overhead: The cold-start phase requires a comprehensive multi-round analysis over the entire initial skill library (consuming 8.4K API calls and 24.7M tokens in this work); scaling to millions of skills will necessitate incremental streaming initialization techniques.
  • Absence of Hard Semantic Rollback: Admission relies on binary task-level execution success, which can inadvertently admit sub-optimal or over-specialized skills that happen to pass an easy instance; without an explicit deprecation or rollback mechanism, flawed skills may propagate through subsequent composition chains.
  • Restricted Relation Semantics: The current framework models only hierarchy, subset, and abstraction axes, leaving complex temporal resource conflicts and non-hierarchical causal dependencies unaddressed.
  • vs Flat Memory and Reflection (Reflexion, ExpeL): Reflexion and ExpeL accumulate natural language insights or critique summaries within an unstructured flat memory; as trials scale, context bloat and retrieval dilution inevitably hurt decision-making. SkillPyramid compiles experience into callable, modular programs embedded in a structured dependency graph.
  • vs Code-Based Skill Banks (Voyager, Code as Policies): Voyager maintains executable code libraries for embodied exploration but relies on single-level program synthesis without explicit high-level task schema abstraction. SkillPyramid introduces a bidirectional architecture that simultaneously abstracts reusable procedural logic and modularizes low-level action primitives.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Proposes the first bidirectional hierarchical skill consolidation topology with decoupled generation and lifelong self-evolution for LLM agents.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluations across three distinct interactive benchmarks, four frontier LLM backbones, detailed task-type ablations, and noisy web-scale transfer.
  • Writing Quality: ⭐⭐⭐⭐⭐ Clear mathematical formalization, structured narrative exposition, and transparent disclosure of system prompt templates and algorithmic procedures.
  • Value: ⭐⭐⭐⭐⭐ Provides a rigorous, modular, and scalable paradigm for lifelong agent learning and structured capability accumulation.