SkillMe: Evolving Agent Skills for Recurring Personal Workflows¶
Conference: EMNLP 2026
Paper: https://aclanthology.org/2026.emnlp-main.564/
Code: https://github.com/deerlove-lxq/SkillMe
Area: LLM Agent
Keywords: Agent Skill Evolution, Recurring Personal Workflows, Evidence-Grounded Maintenance, Task-Scoped Skills, Agent-Native Evaluation
TL;DR¶
Addressing the situated mismatch of generic skills under recurring personal workflows, SkillMe establishes a workflow-situated skill evolution framework where a high-capability Publisher induces task-scoped skills as temporary workspace files, heterogeneous Executors provide trace-grounded feedback, and evidence attribution drives closed-loop revision, yielding a +25.7 points mean gain and significantly lowered execution volatility.
Background & Motivation¶
Large language model (LLM) agents are increasingly entrusted with automating recurring real-world personal workflows, spanning inbox triage, meeting minutes extraction, data report generation, and multi-file code refactoring. To prevent agents from inefficiently rediscovering procedural knowledge from scratch in repetitive tasks, existing research has extensively developed agent skill libraries, extracting reusable procedures from human demonstrations or agent execution rollouts into hierarchical, shareable skill packages. This broad-reuse-oriented paradigm inherently presumes that procedural guidance that transfers broadly across benchmarks will naturally succeed when applied to an individual user's repeated tasks.
However, in real-world personal workflows, broad transferability does not guarantee situated workflow fit. Generic public skills frequently assume cloud-native REST APIs, standard OAuth2 flows, and uniform environment variables. When executed in concrete personal environments, agents routinely encounter unstated local contracts, such as decoy binaries, port startup delays on local mock servers, non-standard parameter passing schemas, or strict formatting assumptions. Storing such localized guidance permanently within the agent's persistent global skill directory causes context bloat and triggers misleading tool conflicts in unrelated tasks; meanwhile, injecting it wholesale into every prompt robs the agent of flexible planning. Thus, the core tension is that recurring workflows urgently demand precise adaptation to local execution contracts, yet static generic libraries and one-off distillation fail to capture subtle environment obstacles and model-specific execution failure modes.
The key insight of this work is to maintain an on-demand, task-scoped evolution layer that coexists with native skill libraries: a strong model explores and establishes initial procedural guidance, while ordinary user-side runs convert execution traces into grounded empirical evidence for continuous refinement. Core idea: SkillMe introduces a workflow-situated skill evolution framework that induces task-scoped skills with verifiable evidence links, activates them selectively as session-local workspace artifacts, collects four-way structured feedback grounded in execution traces, and performs evidence-attributed batch revision, achieving cost-effective and low-variance workflow adaptation.
Method¶
Overall Architecture¶
SkillMe comprises a user-side agent execution harness and a platform-side evolution engine. On the user side, the agent retains its persistent native skill library. When a recurring workflow is matched, the corresponding task-scoped skill is mounted as temporary workspace files, allowing the executor to inspect and adopt guidance progressively, and is completely cleaned up post-session. On the platform side, an asymmetric division of labor between a high-capability Publisher and lightweight Executors governs the lifecycle of skill induction, situated execution, feedback collection, and evidence-grounded maintenance.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Workflow Trigger & Matching<br/>Task instruction + isolated workspace"] --> B["Publisher Read-Before-Write Induction<br/>Full trace audit & evidence mapping"]
B --> C["Temporary Workspace On-Demand Mounting<br/>Task-scoped skill activated & cleaned"]
C --> D["Four-Way Structured Feedback Collection<br/>Execution trace anchoring & typed categorization"]
D --> E["Evidence-Grounded Attribution & Batch Synthesis<br/>Workflow conflict revision vs executor clarification"]
E -->|Update skill pool| C
Key Designs¶
1. Publisher Read-Before-Write Induction: Eliminating Groundless Assumptions and Hallucinated Guidance Existing skill generation methods often prompt strong models to summarize guidance freely from rollouts, inadvertently injecting generic pre-training priors (such as assuming public cloud APIs or uninstalled third-party libraries) that conflict with local environments. SkillMe enforces a strict read-before-write protocol during Publisher induction: before generating any guidance, the Publisher must sequentially inspect task metadata, execution summary, complete tool-call arguments, outputs, and extracted code artifacts, constructing an explicit evidence map linking candidate claims to concrete transcript spans. The resulting skill body systematically covers six standard sections: Methodology, Key Steps, Reusable Artifacts, Pitfalls, Verification Checklist, and Assumptions & Transfer Notes. Any unsupported claim is discarded, ensuring every piece of guidance is anchored in validated execution facts.
2. Temporary Workspace On-Demand Mounting: Balancing Non-Invasive Isolation with Progressive Inspection Persistently installing workflow-specific guidance into an agent's global skill directory causes context clutter and triggers unintended tool invocations across unrelated tasks, while forcing the entire skill into the system prompt inflates context costs and restricts autonomous reasoning. SkillMe adopts a session-local file mounting mechanism: upon workflow match, the versioned skill is written directly into the current execution workspace as standard files. The Executor follows a progressive reading protocolโfirst assessing whether it can solve the task independently, and reading the full skill or specific sections (e.g., troubleshooting pitfalls, parameterized command templates) only when uncertainty or obstacles arise. Once the session concludes, the temporary skill files are removed, preserving native environment hygiene while providing on-demand guidance.
3. Four-Way Structured Feedback Collection: Transforming Execution Traces into Verifiable Evidence
Traditional agent self-reflection and multi-turn feedback rely on open-ended text commentary or scalar rewards, making it difficult to attribute root causes to specific procedural steps. SkillMe introduces a four-way typed feedback schema: ack (confirming specific steps or artifacts that directly produced success), bug (reporting erroneous guidance that caused failure), style (flagging valid but redundant, overly verbose, or partially relevant instructions), and help (signaling unresolved environment blockers or schema gaps). Executors independently inspect their post-execution transcripts, mandating that every feedback item specify the target skill section, a concise claim, and exact transcript event IDs or line ranges, thereby creating an auditable evidence chain.
4. Evidence-Grounded Attribution and Batch Synthesis: Preventing Negative Transfer and Ensuring Coherent Updates When aggregating feedback across heterogeneous Executors, naively incorporating all suggestions causes negative transfer due to differences in tool-calling conventions and model-specific capabilities. The Publisher executes a two-phase maintenance process: in per-item attribution, each feedback claim is cross-checked against the original evidence map and execution trace. Confirmed environment-level obstacles (e.g., hidden service warm-up delays, non-standard CLI arguments) are flagged for revision, whereas issues stemming from incomplete reading, tool misuse, or executor-specific reasoning failures are recorded as clarification memories without modifying the shared skill. In batch synthesis, validated modifications within the same section are consolidated into cohesive updates via targeted patches or multi-section rewrites.
A Worked Example¶
Consider the Google Workspace email triage workflow (task_gws_triage), which features hidden execution constraints: a decoy gws binary in the workspace root crashes with a Java exception, while the real Go binary is hidden under .bin_real/gws, requiring a local fws mock server and non-standard --params '{...}' JSON arguments.
1. No-Skill Condition: Executors invoke gws --help, hit the Java exception, fail to locate the hidden binary, and terminate without reading any emails (mean score: 10.0).
2. Generic Public Skill: Equipped with a standard open-source Gmail skill, Executors attempt OAuth2 authentication against non-existent cloud endpoints, failing completely (mean score: 16.4).
3. Initial Publisher Skill (R0): The Publisher documents .bin_real/gws and the fws startup flow, lifting the 8-Executor mean to 58.4; however, slower models fail because they query the mock server before its startup completes, returning bug and help traces.
4. Evolved Skill: Incorporating trace feedback, the Publisher patches the skill with an explicit fws port health check and RFC 2822 MIME templates, boosting the mean score to 87.9 and successfully stabilizing the fragile workflow contract.
Key Experimental Results¶
Main Results¶
Evaluated on the 71-task skill-needed subset of PinchBench using OpenClaw, SkillMe improves the overall mean score across all eight heterogeneous Executors by an average of +25.7 points over their paired no-skill baselines (with deepseek-v4-pro as Publisher).
| Task Category (71 Tasks) | gpt-oss-120b | gemma-4-31b-it | doubao-seed-2.0-lite | gpt-5.4-nano | gpt-oss-20b | doubao-seed-2.0-mini | gemini-3.1-flash-lite | qwen3.5-flash | SkillMe Mean |
|---|---|---|---|---|---|---|---|---|---|
| Productivity | 92.5 | 96.3 | 94.4 | 89.8 | 89.0 | 97.4 | 94.9 | 93.0 | 93.4 |
| Research | 63.0 | 89.0 | 96.4 | 86.2 | 75.8 | 87.6 | 80.2 | 86.6 | 83.1 |
| Writing | 6.2 | 99.0 | 100.0 | 100.0 | 91.0 | 100.0 | 100.0 | 100.0 | 87.0 |
| Coding | 96.1 | 98.5 | 90.1 | 90.1 | 90.1 | 88.4 | 88.5 | 89.3 | 91.4 |
| Analysis | 83.9 | 96.0 | 98.0 | 93.3 | 84.7 | 94.9 | 94.8 | 94.9 | 92.6 |
| CSV Analysis | 74.8 | 90.9 | 95.6 | 90.9 | 83.9 | 90.3 | 91.2 | 93.5 | 88.9 |
| Log Analysis | 70.7 | 98.8 | 99.2 | 99.7 | 91.9 | 97.9 | 98.2 | 99.2 | 94.4 |
| Meeting Analysis | 52.5 | 87.1 | 96.0 | 85.9 | 63.7 | 89.7 | 83.2 | 94.1 | 81.5 |
| Memory | 100.0 | 100.0 | 90.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 98.8 |
| Skills | 100.0 | 100.0 | 100.0 | 99.0 | 100.0 | 100.0 | 100.0 | 100.0 | 99.9 |
| Integrations | 12.5 | 100.0 | 100.0 | 100.0 | 8.3 | 95.8 | 100.0 | 95.8 | 76.6 |
| No-Skill Baseline Mean | 20.8 | 63.6 | 78.2 | 65.6 | 47.9 | 73.2 | 73.4 | 80.8 | 62.9 |
| SkillMe Mean | 69.5 | 93.0 | 96.6 | 91.6 | 79.8 | 93.0 | 90.8 | 94.6 | 88.6 |
| Gain (ฮ) | +48.7 | +29.4 | +18.4 | +26.0 | +31.9 | +19.8 | +17.4 | +13.8 | +25.7 |
Ablation Study¶
The evaluation contrasts generic public skills against SkillMe on 11 category-representative tasks, and examines the necessity of evidence-attributed feedback memory on 8 Publisher-revised tasks across two distinct Executor models.
| Experiment Suite | Evaluation Condition | 8-Executor Mean | Key Observations & Performance Implications |
|---|---|---|---|
| Generic Skills vs. SkillMe (11 Representative Tasks) | No-Skill Baseline | 48.6 | Struggles heavily with unstated environment setups |
| Generic Public Skills | 66.3 | Improves overall mean but hurts in 13/88 pairs due to mismatched API assumptions | |
| SkillMe Evolved Skills | 95.0 | Beats generic skills in 52/88 pairs, ties in 34, loses in only 2 | |
| Feedback Attribution Ablation (8 Tasks ร 2 Executors) | Shared Skill Only (w/o Feedback Memory) | 40.8 | Baseline performance on challenging tasks where R0 was initially insufficient |
| Direct Cross-Executor Feedback Injection | 47.8 | Incompatible tool-invocation patterns introduce noise and degrade stability | |
| Matched Evidence-Grounded Feedback | 73.4 | Attributed feedback drives +32.6 points gain by directly resolving shared bottlenecks |
Key Findings¶
- Variance Reduction Trumps Monotonic Peak Gain: Across 7 recurrences on the 21 core tasks, evolved skills reduced recurrence-to-recurrence variance from \(\sigma=2.0\) (static R0) to \(\sigma=1.2\), raising mean score from 81.7 to 84.0 by eliminating recurring misunderstandings and fragile environment assumptions.
- Substantial Cost Amortization: Over \(n=30\) recurrences, the amortized cost per task drops to $0.019 (45.5% of the Publisher per-run cost), with six Executors achieving Publisher avoidance rates exceeding 74%.
- Self-Selected Progressive Reading: Executor skill reading rates ranged from 14.3% (qwen3.5-flash) to 61.9% (gemini-flash-lite), showing that stronger models autonomously bypass guidance on familiar tasks while weaker models actively rely on it.
Highlights & Insights¶
- Task-Scoped Ephemeral Activation Eliminates Library Bloat: Mounting skills as temporary workspace files resolves the fundamental tension between workflow specialization and cross-task context interference.
- Trace-Grounded Feedback Replaces Subjective Critique: Anchoring reflections to explicit transcript events and structured tags enables precise bug attribution rather than ambiguous natural language patching.
- Asymmetric Capability Arbitrage: Leveraging a high-tier Publisher for one-time exploration while amortizing subsequent runs across cost-effective Executors offers a practical blueprint for enterprise and personal agent deployment.
Limitations & Future Work¶
- Simulated Workflow Assumptions: Evaluation relies on OpenClaw and PinchBench with exact task ID matching; open-world deployments require robust semantic routing and fuzzy workflow disambiguation.
- Optimistic Best-Recurrence Metric: The benchmark utility aggregates best scores across recurrences; integrating run-time circuit breakers to catch first-attempt failures in real time represents an important future step.
Related Work & Insights¶
- vs Voyager / SkillX: Voyager and SkillX distill broadly transferable Python code snippets for open-ended discovery; SkillMe focuses on post-deployment adaptation for recurring personal workflows where local contracts dominate.
- vs SWE-Skills-Bench: While SWE-Skills-Bench observed that generic skills frequently degrade performance in complex software engineering, SkillMe shows that task-scoped scoping and trace-based attribution prevent such negative transfer.
Rating¶
- Novelty: โญโญโญโญโญ Establishes the first systematic workflow-situated skill evolution framework for personal agents.
- Experimental Thoroughness: โญโญโญโญโญ Rigorous evaluation across 8 models, 71 workflows, 3,064 audited feedback comments, and full cost amortization.
- Writing Quality: โญโญโญโญโญ Exceptionally clear framing, rigorous methodology, and illuminating concrete case studies.
- Value: โญโญโญโญโญ Delivers high practical value for deploying reliable, low-cost personal LLM agents in recurring automation.