Skip to content

ToolLoop: Closed-Loop Tool-Use Data Synthesis via Decomposed Generation and Dynamic Self-Feedback

Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning / Tool Calling & Agents
Keywords: Tool-use data synthesis, Function calling, Dynamic self-feedback, Decomposed generation, Closed-loop refinement

TL;DR

ToolLoop decomposes tool-use data synthesis into a three-stage pipeline (ground-truth function combination sampling, backward query derivation, and forward tool call instantiation) augmented with stage-specific dynamic self-feedback, achieving 86.40% non-reasoning accuracy on BFCL with only 11K synthetic examples.

Background & Motivation

Large language models are rapidly evolving from static text predictors into autonomous agents capable of perceiving environment states and invoking external software tools and APIs. However, mastering tool use requires high-quality training datasets that accurately reflect the complex grounding between natural user requests and formal API execution schemas. Prevailing data synthesis methodologies predominantly rely on a conventional "generate-then-filter" paradigm, wherein a generative model produces full instruction-tool pairs in a single forward pass, followed by post-hoc static filtering using heuristic rules or external verifier models.

Despite its simplicity, this one-pass open-loop framework suffers from three severe structural bottlenecks. First, when faced with large, realistic tool candidate repositories featuring overlapping capabilities, directly selecting the correct function and properly binding arguments in a single generation step imposes an overwhelming cognitive load. Second, static post-hoc filtering acts as a binary accept/reject barrier; by discarding invalid instances outright without providing corrective feedback, it disproportionately purges challenging multi-tool and fine-grained queries, leaving behind an impoverished dataset biased toward trivial syntax and simplified schemas. Third, the total absence of intermediate milestone supervision prevents verification of logical consistency across stages, allowing subtle semantic disconnectsโ€”such as queries missing mandatory parameter entitiesโ€”to slip into training corpora.

To break through these bottlenecks, data synthesis must transition from passive terminal filtering toward targeted, iterative self-correction. Core idea: decompose tool-use synthesis into three progressive stages (ground truth function sequence sampling \(\to\) backward user query derivation \(\to\) forward tool call instantiation) and embed an iterative dynamic self-feedback loop combining semantic auditing, deterministic rules, and AST syntax parsing at each stage, transitioning the paradigm from generate-then-filter to generate-verify-refine.

Method

Overall Architecture

ToolLoop abandons monolithic end-to-end generation in favor of a decomposed three-stage assembly line that enforces logical causality: Stage 1 samples an executable combination of function names representing the target ground truth; Stage 2 applies backward derivation to construct a realistic user query that strictly specifies all parameters demanded by the ground-truth functions; Stage 3 executes forward derivation to populate standardized tool calls matching the schema. Crucially, each stage is guarded by a multi-tiered verifier that detects semantic and structural violations. When an error is identified, a tailored refinement promptโ€”containing the negative attempt and actionable error diagnosticsโ€”guides the model to regenerate the intermediate representation, tolerating up to three retries before discarding.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["API Pool<br/>5,281 Executable Functions"] --> B["Candidate Function Construction<br/>Qwen3-Embedding + K-Means Clustering"]
    B --> C["Ground Truth Sampling<br/>Stage 1: Parallelizability & Coherence"]
    C -->|Validation Failure ยท Refine Retry| C
    C -->|Pass| D["Backward Derivation of Query<br/>Stage 2: Strict Argument Grounding"]
    D -->|Semantic Drift / Missing Args ยท Refine Retry| D
    D -->|Pass| E["Forward Derivation of Tool Calls<br/>Stage 3: Schema & Type Instantiation"]
    E -->|AST / Schema Error ยท Refine Retry| E
    E -->|Pass| F["High-Quality Synthetic Dataset<br/>11K Examples Across 4 Scenarios"]

Key Designs

1. Candidate Function Construction: Partitioning the Functional Tool Space via Semantic Embeddings

Randomly sampling tool candidates from huge tool repositories fails to simulate meaningful operational contexts, while presenting the entire tool catalog exceeds the effective context window of LLM generators. ToolLoop converts the docstrings of 5,281 executable APIs into dense semantic representations using Qwen3-Embedding-8B and applies K-means clustering (\(K=26\), with approximately 200 functions per cluster across domains like financial intelligence, geographic services, media processing, and scientific computing). Sampling candidates strictly from within the same semantic cluster ensures that candidate sets present realistic semantic competition and complementarity while keeping prompt token lengths compact and computationally tractable.

2. Three-Stage Decomposed Generation: Reversing Causality for Airtight Semantic Alignment

Rather than asking an LLM to hallucinate a user query, a target tool, and parameter values simultaneously, ToolLoop decomposes synthesis into three causally ordered stages: - Stage 1 (Ground Truth Sampling): For Simple and Multiple scenarios, single functions are chosen; for Parallel and Parallel Multiple scenarios, an LLM selects 2โ€“4 functions that can execute simultaneously at time zero without sequential data dependencies, while maintaining logical scenario coherence. - Stage 2 (Backward Derivation of User Query): With the ground truth function sequence fixed, the model reasons backward to formulate a natural user instruction. The prompt enforces strict 1-to-1 intent alignment, natural colloquial phrasing, and explicit embedding of all concrete argument values required by the schema (forbidding abstract placeholders or missing parameters). - Stage 3 (Forward Derivation of Tool Calls): Conditioned on the query and candidate functions, the model instantiates concrete tool calls formatted in the standard OpenAI function-calling schema, strictly matching argument names, data types, and values extracted from the user text.

3. Dynamic Self-Feedback: Multi-Tiered Verification and Actionable Refinement Loops

To prevent the loss of diverse and complex data instances that plagues static filtering, ToolLoop embeds stage-specific dynamic feedback mechanisms composed of three orthogonal verifiers: (1) deterministic rule checks for data types and structural integrity; (2) AST parsing to detect unclosed brackets, improper escapes, or syntax violations that would cause execution crashes; (3) an LLM judge (Qwen-Max) to evaluate semantic relevance, intent-tool mapping, and completeness. When a check fails, the framework constructs a refinement prompt pairing the original task with the failed sample and explicit corrective instructions (e.g., detailing the specific parameter type mismatch or unclosed AST bracket). Up to three iterative regeneration cycles are allowed, enabling the system to repair most recoverable flaws without discarding valuable high-complexity training instances.

Loss & Training

The 11,024 synthesized data instances (40.4% Simple, 33.0% Parallel, 16.2% Multiple, and 10.5% Parallel Multiple) were used to fine-tune Qwen3-4B-Instruct-2507 using the swift training framework. - Hardware & Hyperparameters: Multi-node cluster with 4ร— NVIDIA L40s (48GB VRAM) GPUs per node; maximum sequence length of 16,384 tokens; supervised fine-tuning conducted over 2 epochs to avoid overfitting on the compact dataset. - Evaluation Protocol: Evaluated strictly in non-reasoning mode without chain-of-thought (CoT) traces. The model is forced to output structured tool calls directly, rigorously probing the direct mapping capability between user instructions and executable parameters.

Key Experimental Results

Main Results

On the Berkeley Function Calling Leaderboard (BFCL, updated 2025-12-16), ToolLoop-4B is compared against leading proprietary frontier models, open-source weights, and data-centric synthesis baselines across Non-Live and Live subsets (% accuracy):

Model Training Volume Non-Live Simple Non-Live Multiple Non-Live Parallel Non-Live Parallel-Multiple Non-Live Overall Live Overall Overall Score
GPT-5.2 (2025-12-11) Proprietary 72.92 88.00 89.00 77.50 81.85 70.39 76.12
Gemini-3-Pro-Preview Proprietary 75.50 94.00 91.00 82.50 85.75 81.72 83.74
Claude-Opus-4-5 Proprietary 76.83 95.50 93.50 88.50 88.58 79.79 84.19
Qwen3-32B 32B Base 75.58 94.50 93.50 91.50 88.77 82.01 85.39
Qwen3-4B-Instruct-2507 4B Base 75.50 93.50 92.50 90.00 87.88 76.39 82.14
APIGen-4B 60K Synthetic 79.58 94.50 95.00 90.50 89.90 76.31 83.11
ToolMind-4B 55K Synthetic 77.92 96.50 93.50 90.00 89.48 77.57 83.53
ToolLoop-4B-Isolate 10K (Leakage-Filtered) 79.33 97.00 94.50 93.50 91.08 81.05 86.07
ToolLoop-4B 11K Synthetic 79.67 96.50 94.50 94.50 91.29 81.50 86.40

On the comprehensive ACEBench cross-evaluation benchmark across five dimensions:

Model Data Scale Overall Atom Single Turn Similar API Profile
Qwen3-4B-Instruct-2507 (Base) 0 64.9 68.0 59.5 68.0 64.0
APIGen-4B 60K 67.0 76.0 62.0 76.0 54.0
ToolMind-4B 55K 70.2 83.3 69.5 74.0 54.0
ToolLoop-4B 11K 72.1 84.0 66.5 78.0 60.0

Ablation Study

Ablation of feedback mechanisms evaluated on BFCL:

Configuration Non-Live Accuracy Live Accuracy Overall Accuracy Note
Qwen3-4B-Instruct (Base) 87.88 76.39 82.14 Base model without synthetic tool-use fine-tuning
w/o Feedback 85.02 74.97 79.97 Unfiltered synthesis actively hurts base model by 2.17 pts
w/ Final Filtering 88.81 76.31 82.56 Standard generate-then-filter baseline; discards hard instances
ToolLoop (w/ Feedback) 91.29 81.50 86.40 Generate-verify-refine loop gains +4.26 over base and +3.84 over filter

Retry iteration distribution across synthesis stages:

Stage 0 Retries 1 Retry 2 Retries 3 Retries Final Discarded Samples
Stage 1: Ground Truth Sampling 10,720 114 161 29 -
Stage 2: Backward Query Derivation 9,029 1,108 530 357 -
Stage 3: Forward Tool Call Derivation 10,257 299 298 170 -
Total Discarded (Exceeded 3 Retries) - - - - 280 (192 semantic, 83 rule-based, 5 both)

Key Findings

  • Uncontrolled Synthetic Data Degrades Performance: Fine-tuning directly on synthetic data without validation (w/o Feedback) drops overall accuracy by 2.17% below the base model, demonstrating that noisy tool calls inject harmful supervision. Terminal filtering merely claws back 0.42% over the base model, while dynamic iterative refinement unlocks a substantial 4.26% gain.
  • Stage 2 Query Derivation is the Primary Semantic Bottleneck: 18.1% of samples required at least one retry in Stage 2, substantially higher than in Stages 1 and 3. Grounding formal parameters into fluent, non-redundant natural language is the most error-prone translation step, and dynamic self-feedback catches the majority of missing parameter arguments here.
  • Robust Generalization without Schema Memorization: The ToolLoop-4B-Isolate variant, which systematically purges all candidate tools overlapping with the BFCL test set, achieves 86.07% accuracy (only 0.33% below the full model). Furthermore, on the transfer benchmark ACEBench, ToolLoop achieves 72.1% overall accuracy using only 18.3% of APIGen's data volume, demonstrating generalizable tool-selection and parameter-grounding competencies.

Highlights & Insights

  • From Discarding Hard Samples to Guiding Targeted Refinements: By feeding specific AST parser error logs or missing parameter warnings back into the prompt, ToolLoop preserves difficult, multi-function samples that standard filtering pipelines systematically eliminate, preventing dataset mode collapse toward simplistic tool queries.
  • Causal Inversion via Backward Query Generation: Generating the natural query from a pre-sampled ground-truth API chain guarantees that every required parameter has a grounded origin in the user prompt, eliminating phantom parameters and unsolvable tasks at the source.
  • High Data Efficiency in Non-Reasoning Settings: Using only 11K curated samples, a 4B parameter model surpasses 55Kโ€“60K synthetic baselines and outperforms multi-billion parameter proprietary giants (GPT-5.2, Claude-Opus-4.5) on complex non-live function selection benchmarks.

Limitations & Future Work

  • Absence of Real-World Sandbox Execution: Validation relies on static rule-based, AST, and LLM-as-a-judge evaluators without dispatching calls to live sandbox environments, leaving dynamic behaviors such as API execution timeouts, malformed server responses, and cascading runtime failures unexplored.
  • Potential Verifier Bias: All three stages utilize Qwen-Max as the sole semantic verifier. While human-agreement audits show a 94% alignment rate on 100 sampled cases, evaluator-specific inductive biases cannot be completely excluded.
  • Scope Constrained to Single-Turn Invocations: The current study focuses strictly on single-turn multi-tool scenarios (Simple, Multiple, Parallel, Parallel Multiple), leaving multi-turn dialogic interactions and iterative agentic action-observation loops to future research.
  • vs. APIGen / APIGen-MT: APIGen popularized verifiable API generation pipelines but relies on single-pass forward generation with terminal multi-stage verification; ToolLoop introduces decomposed three-stage generation coupled with dynamic self-feedback, achieving superior tool-use precision with under 20% of APIGen's training data.
  • vs. ToolMind: ToolMind bolsters tool invocation by injecting extensive reasoning CoT chains; ToolLoop demonstrates that in non-reasoning direct-dispatch environments, refining data consistency and schema adherence alone provides decisive performance advantages.
  • vs. ToolBench / ToolACE: Earlier web-scale collection methods incur heavy noise and imprecise parameter grounding; ToolLoop shows that a controlled closed-loop generate-verify-refine pipeline produces a far cleaner training signal.

Rating

  • Novelty: โญโญโญโญโ˜† Introduces a clean three-stage decomposed pipeline with causal backward query derivation and dynamic multi-tier self-feedback.
  • Experimental Thoroughness: โญโญโญโญโญ Rigorous evaluations across BFCL and ACEBench, including strict data leakage isolation, ablation studies, retry distributions, and token consumption breakdowns.
  • Writing Quality: โญโญโญโญโญ Clear exposition, well-motivated architectural rationale, and complete prompt templates ensuring reproducibility.
  • Value: โญโญโญโญโญ Provides an efficient, practical blueprint for synthesizing clean function-calling data to empower lightweight edge models.