when search becomes memory accelerating robot design discovery with self evolvin
content_hash: 2ec883128e756147 ---# When Search Becomes Memory: Accelerating Robot Design Discovery with Self-Evolving Skills
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/wangyf9/Auto-Robotist
Area: Robotics & Embodied AI
Keywords: robot morphology design, evolutionary search, LLM agents, skill memory library, cross-scale transfer
TL;DR¶
Addressing the memoryless limitation of traditional robot morphology search where costly simulation traces are discarded after each iteration, this paper introduces AUTO-ROBOTIST, a self-evolving LLM agent that distills evaluation traces into an explicit natural-language skill library (featuring archetypes, positive/negative rules, and empirical observations), accelerating cold-start discovery and enabling effective cross-scale transfer from \(5\times 5\) to \(10\times 10\) design spaces.
Background & Motivation¶
In automated modular soft-robot design, co-optimizing physical morphology and control policies relies heavily on computationally expensive physical simulation evaluations. In benchmarks such as Evolution Gym (EVOGYM), evaluating a single candidate robot morphology requires not only verifying voxel 4-connectivity and actuator feasibility constraints, but also training a Proximal Policy Optimization (PPO) reinforcement learning controller for up to \(5.12\times 10^5\) environment steps. When scaling up robot dimensions from a \(5\times 5\) grid to a \(10\times 10\) grid, the discrete voxel selection space explodes combinatorially, spanning more than \(5^{100} > 10^{69}\) possible body configurations, which makes unguided search intractable.
However, existing evolutionary search algorithms, quality-diversity methods, and recent systems employing large language models as proposal generators remain fundamentally memoryless. Every physical evaluation is an empirical experiment concerning load support, ground contact stability, actuator stress transmission, and locomotion failure modes. Yet standard evolutionary algorithms preserve only the best-performing elite artifacts or updated population distributions, completely discarding the valuable experimental evidence behind why certain structures failed or succeeded. Furthermore, naively resizing elite bodies across resolutions via \(2\times 2\) voxel tiling consistently breaks functional integrity due to altered moment arms, sudden shifts in contact mechanics, and imbalanced stress distributions.
Human designers do not merely rescale isolated blueprints when encountering a new design challenge; instead, they distill transferable, auditable structural principles from previous engineering trials. While LLMs exhibit strong architectural priors and natural language summarization capabilities, without an explicit memory grounded in physical evidence, they risk degenerating into unconstrained hallucination engines. This paper addresses this gap by converting costly physical simulation traces into a persistent, revisable structural design theory. Core idea: construct a three-level self-evolving skill library decoupling structural archetypes (L1), positive and negative rules (L2), and physical observations (L3), guiding LLM-based mutations alongside genetic algorithm exploration via evidence-weighted utility and continuously rewriting design memory through Add, Diagnose, and Merge operations.
Method¶
Overall Architecture¶
The AUTO-ROBOTIST pipeline couples design evolution with continuous skill evolution. In each search generation, the agent retrieves task- and scale-relevant skills from its persistent memory library, computes evidence-weighted utility sampling scores, and guides a frozen LLM to propose skill-conditioned mutations over current elite parents (Path A). In parallel, standard genetic algorithm mutations (Path B) are maintained to preserve structural coverage and provide robust fallbacks. Proposed child morphologies undergo topological validity filtering before entering the EVOGYM simulator for PPO controller training. The evaluated fitness and structural differentials form observation records, which are then integrated back into the library through three atomic operations: ADD, DIAGNOSE, and MERGE, transforming ephemeral simulation trials into persistent, auditable design knowledge.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Elite Morphology Pool + Target Task Environment"] --> B["Three-Level Skill Library Retrieval & Utility Sampling"]
B --> C["Dual-Path Candidate Proposal & Validity Filtering"]
C --> D["EVOGYM Physical Simulation & PPO Controller Training"]
D --> E["Generative Self-Evolving Library Maintenance<br/>ADD / DIAGNOSE / MERGE"]
E -->|Update Elite Pool & Domain Design Theory| A
Key Designs¶
1. Three-Level Explicit Skill Representation: Decoupling Archetypes, Rules, and Audit Evidence
Standard agent architectures often retain past reflections as unstructured free-form context prompts or raw execution logs, which quickly suffer from context bloat and ungrounded speculation over long search horizons. AUTO-ROBOTIST structures each skill \(s \in \mathcal{S}\) into an explicit, three-level self-contained record: Level 1 (L1 Archetype) defines a compact structural topology with an applicability condition (e.g., portal_frame for load-bearing arches enclosing actuators); Level 2 (L2 Rules) contains actionable positive rules (beneficial sub-patterns) and negative rules (grounded failure modes), where each rule stores a qualitative structural description, supporting observation IDs, and a running mean of parent-relative fitness gain \(\bar{g}_\ell\); Level 3 (L3 Observations) maintains an auditable trace of child and parent voxel grids, task identity, scale, fitness gain \(g_{s,i}\), and proposal attribution. By expressing rules through relative morphological relations (e.g., "support a central actuator column with a rigid side frame") rather than absolute voxel coordinates, the representations generalize naturally across grid resolutions while preserving rigorous accountability.
2. Dual-Path Candidate Proposal: Skill-Conditioned Mutation Paired with GA Exploration
Relying entirely on LLM generation can narrow exploration due to model induction bias, whereas pure genetic algorithm (GA) search lacks targeted hypothesis guidance. In each generation, AUTO-ROBOTIST allocates its evaluation budget of 25 designs into 15 skill-conditioned slots (Path A) and 10 random GA mutation slots (Path B). For Path A, skills are sampled based on a smoothed, saturation-clipped usefulness score: $\(w_s = \frac{1 + \sum_{i=1}^{n_s} \text{clip}(g_{s,i} / \delta_{\max}, 0, 1)}{2 + n_s}\)$ where \(g_{s,i} = f(d_{\text{child}}, \mathcal{T}) - f(d_{\text{parent}}, \mathcal{T})\) represents parent-relative fitness gain, \(n_s\) is the number of attributed observations, and \(\delta_{\max}=2.0\) bounds outlier effects. The sampled L1 archetype and L2 rules are formatted into prompts directing the LLM to mutate selected elite parents (modifying 1–3 voxels at \(5\times 5\), or 1–10 voxels at \(10\times 10\)). Before simulation, every proposed candidate is strictly validated: voxels must belong to legal robot material classes, non-empty cells must form a single 4-connected component, and at least one actuator must be present. Irreparable invalid outputs automatically default to Path B GA mutations, ensuring zero wasted simulation budget on malformed bodies.
3. Generative Self-Evolving Library Maintenance: Closed-Loop ADD, DIAGNOSE, and MERGE
To enable continuous self-refinement as physical evidence accumulates, the agent executes three serial operations after each generation's evaluations. In ADD, unassigned high-fitness observations that do not match existing skills are gathered; the LLM creates at most one new L1 archetype per generation only when a coherent, recurring load-bearing structure emerges, initializing with empty L2/L3 lists to prevent one-skill-per-anecdote bloat. In DIAGNOSE, skills receiving new evidence update their rules: targeted rules update their running empirical mean via: $\(\bar{g}_\ell \leftarrow \bar{g}_\ell + \frac{g_i - \bar{g}_\ell}{m_\ell}\)$ where \(m_\ell\) is the updated sample count. Rule-free observations are categorized by the LLM into positive rules (lenient criterion: gain > 0 and reusable structural pattern) or negative rules (strict criterion: significantly negative gain and obvious failure mode), turning failure into first-class protective constraints. In MERGE, the LLM identifies redundant L1 archetypes describing identical structural principles, consolidating conceptual duplicates while preserving all underlying L2 rules and L3 observations to retain a complete audit trail without collapsing into a single template.
A Worked Example¶
On the challenging payload transport task Carrier, the search trace illustrates how explicit memory evolves: at generation \(R_0\), the agent starts from a weak random seed with a baseline fitness of 2.92; initial evaluations reveal that pairing base actuators with rigid foundations provides reliable forward traction, triggering an ADD operation that registers an active base archetype; as search progresses, the agent observes that open top cavities drop the payload prematurely, leading DIAGNOSE to instantiate the negative rule blocked_lower_frame alongside the positive rule upper_bridge_pair (symmetric rigid overhead beams); by generation \(R_6\), the LLM retrieves these rules to mutate an elite parent by reinforcing central vertical spine supports and closing top bridges, discovering a load-bearing carrier structure that reaches a fitness of 7.14 (and an eventual multi-seed mean of 8.19), markedly outperforming the traditional GA baseline endpoint of 6.40.
Key Experimental Results¶
Main Results¶
Evaluations span seven diverse EVOGYM tasks (flat-ground locomotion Walker, uneven traversal BridgeWalker, balance maintenance Balancer, payload transport Carrier, vertical climbing Climber, vertical jumping Jumper, and object pushing Pusher), covering \(5\times 5\) cold-start discovery (starting from an empty library) and \(5\times 5 \rightarrow 10\times 10\) cross-scale transfer. Metrics include final maximal fitness, convergence speedup \(S\) (speed ratio to reach GA's final endpoint, where \(S > 1\) denotes faster convergence), and lead fraction \(L\) (fraction of evaluation budget where AUTO-ROBOTIST strictly leads GA).
Table 1: \(5\times 5\) cold-start skill discovery performance (mean \(\pm\) standard deviation across seeds 0, 9, 42).
| Task | GA Baseline | AUTO-ROBOTIST | Gain (\(\Delta\)) | Speedup (\(S\)) | Lead Fraction (\(L\)) |
|---|---|---|---|---|---|
| Walker | 9.54 | 9.55 \(\pm\) 0.01 | +0.01 | 1.30 | 0.67 |
| BridgeWalker | 3.65 | 3.69 \(\pm\) 0.05 | +0.04 | 1.28 | 0.53 |
| Balancer | 0.13 | 0.15 \(\pm\) 0.01 | +0.02 | 1.13 | 0.45 |
| Carrier | 6.40 | 8.19 \(\pm\) 1.21 | +1.79 | 2.76 | 0.87 |
| Climber | 0.57 | 0.61 \(\pm\) 0.06 | +0.04 | 1.24 | 0.65 |
| Jumper | 5.58 | 6.67 \(\pm\) 0.33 | +1.09 | 2.16 | 0.83 |
| Pusher | 8.45 | 8.92 \(\pm\) 1.36 | +0.47 | 1.74 | 0.71 |
Table 2: \(10\times 10\) cross-scale transfer performance (1,000 evaluation budget; w/ ref includes source \(5\times 5\) elite exemplar in prompt; w/o ref isolates rule transfer without reference body).
| Task | GA Upsample (\(2\times 2\)) | Ours Upsample (\(2\times 2\)) | GA Full Search | Ours (w/o ref) | Ours (w/ ref) | Gain vs GA (\(\Delta\)) | Speedup (\(S\)) | Lead Fraction (\(L\)) |
|---|---|---|---|---|---|---|---|---|
| Walker | 4.67 | 4.20 | 9.31 | 9.72 | 10.98 \(\pm\) 1.45 | +1.67 | 3.27 | 0.75 |
| BridgeWalker | 1.24 | 0.42 | 2.46 | 2.33 | 3.69 \(\pm\) 0.33 | +1.23 | 1.47 | 0.66 |
| Balancer | -1.44 | -0.14 | 0.09 | 0.08 | 0.09 \(\pm\) 0.04 | +0.00 | 1.81 | 0.52 |
| Carrier | 3.41 | 0.16 | 1.38 | 5.69 | 7.22 \(\pm\) 1.28 | +5.84 | 8.36 | 0.95 |
| Climber | 0.34 | 0.29 | 0.63 | 0.54 | 0.77 \(\pm\) 0.03 | +0.13 | 3.11 | 0.93 |
| Jumper | 1.55 | 2.71 | 5.94 | 7.24 | 6.83 \(\pm\) 0.14 | +0.89 | 1.19 | 0.87 |
| Pusher | 5.85 | 0.76 | 8.47 | 8.36 | 8.83 \(\pm\) 0.80 | +0.36 | 0.97 | 0.74 |
Ablation Study¶
Ablation analysis under the \(5\times 5\) cold-start protocol evaluating component removals on Pusher and Walker:
| Configuration | Pusher Fitness | Pusher Relative Drop | Walker Fitness | Note |
|---|---|---|---|---|
| Full AUTO-ROBOTIST | 9.87 | baseline | 9.56 | complete self-evolving memory loop |
| (A1) No Diagnose | 7.74 | -21.6% | 9.16 | skips DIAGNOSE; L2 rules never update |
| (A2) No Merge | 8.49 | -14.0% | 9.44 | skips MERGE; redundant L1 archetypes accumulate |
| (A3) Pure LLM | 7.51 | -24.0% | 9.35 | removes GA mutations; all 25 slots from LLM |
| (A4) No L2+L3 | 7.52 | -23.8% | 9.54 | strips L2 rules & L3 observations from prompts |
Key Findings¶
- Rule priors alone outperform standard evolution: In \(10\times 10\) transfer without providing any source visual reference body (w/o ref), AUTO-ROBOTIST still surpasses the fully trained GA baseline on Walker (9.72 vs 9.31), Carrier (5.69 vs 1.38), and Jumper (7.24 vs 5.94). This proves the library encodes functional relational invariants (e.g., surrounding actuators with rigid frames) rather than superficial coordinate templates.
- Synergy between geometric anchoring and abstract rules: On contact-sensitive tasks like BridgeWalker and Climber, supplying the source reference body alongside rules provides initial structural anchors, elevating fitness from 2.33 to 3.69 on BridgeWalker and from 0.54 to 0.77 on Climber.
- Exploration balance is essential: In the contact-heavy Pusher task, removing GA mutations (Pure LLM) causes a 24.0% performance drop, highlighting that random discrete structural mutations remain indispensable to prevent the LLM from getting trapped in early design biases.
Highlights & Insights¶
- First-class negative rules for failure prevention: Rather than tracking only successes, AUTO-ROBOTIST captures recurring failure modes (e.g., lower gantry blockage, overlong unbraced rails) as explicit negative constraints, filtering unviable structures before simulation.
- Natural language as a resolution-agnostic topology carrier: Direct voxel upsampling via \(2\times 2\) tiling degrades performance catastrophically across all tasks (e.g., dropping Carrier to 0.16); in contrast, natural-language structural relations transfer seamlessly across varying grid resolutions.
- High sample efficiency with minimal API overhead: In 1,000-evaluation \(10\times 10\) runs, total gpt-5.5 API cost averages only \(51–\)62 per task, while achieving up to \(8.36\times\) convergence speedups, demonstrating that semantic memory delivers substantial gains relative to RL simulation costs.
Limitations & Future Work¶
- Evaluation restricted to 2D voxel simulation: Experiments are currently conducted within EVOGYM 2D environments and have not been validated on 3D continuous morphology or hardware platforms.
- Fixed discrete material alphabet: The grammar relies on five predefined voxel types; adapting the framework to continuous topologies or composite soft materials will require extending the structural vocabulary.
- Heuristic rule conflict resolution: When multiple high-utility skills suggest competing modifications (such as rigid vs active central columns), resolution relies entirely on the LLM prompt; future work should incorporate calibrated uncertainty estimation and formal conflict adjudication.
Related Work & Insights¶
- vs LASeR (Song et al., 2025): While LASeR utilizes LLMs for diversity reflection, its insights remain confined to transient prompt contexts; AUTO-ROBOTIST constructs an explicit, persistent, multi-level memory library enabling cross-generational and cross-scale reuse.
- vs RoboMore (Fang et al., 2025) / BodyGen (Lu et al., 2025): These works focus on joint morphology-reward optimization or topology-aware neural network representations; AUTO-ROBOTIST orthogonally tackles long-term design memory, showing that symbolic skill libraries significantly boost sample efficiency.
- vs Voyager (Wang et al., 2023): While Voyager distills executable code skills for open-ended exploration in Minecraft, AUTO-ROBOTIST establishes an evidence-linked structural memory grounded in the physical mechanics of soft robots.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Pioneering framework transforming expensive robot morphology search traces into an explicit, self-evolving, auditable natural language skill library.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive coverage across all seven EVOGYM tasks, evaluating both cold-start discovery and cross-scale transfer with rigorous component ablations.
- Writing Quality: ⭐⭐⭐⭐⭐ Clear exposition, elegant conceptual formulations, and thorough appendices detailing prompts, hyperparameters, and responsible AI disclosures.
- Value: ⭐⭐⭐⭐⭐ Offers an inspiring paradigm for combining LLM agent memory architectures with expensive physical and engineering simulations.