Skip to content

D3-Gym: Constructing Real-World Verifiable Environments for Data-Driven Discovery

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/OSU-NLP-Group/D3-Gym
Area: LLM Reasoning
Keywords: Data-Driven Scientific Discovery, Real-World Verifiable Environments, Task-Specific Evaluation Scripts, Rejection Sampling Fine-Tuning, Autonomous Research Agents

TL;DR

This paper introduces D3-Gym, the first automatically constructed benchmark of 565 executable and verifiable environments drawn from 239 real scientific repositories across 4 disciplines. By synthesizing task-specific evaluation scripts through a two-stage planning-then-coding pipeline, D3-Gym enables rigorous trajectory verification and rejection sampling fine-tuning that improves Qwen3-32B by +7.8 points SR@3 on ScienceAgentBench, approaching frontier proprietary models.

Background & Motivation

Language agents powered by foundation models are rapidly transforming scientific discovery, expanding their roles from literature synthesis and summarization toward automated hypothesis testing, exploratory data analysis, and end-to-end data-driven experimentation. Groundbreaking paradigms like Autoresearch have underscored the tremendous value of closed-loop verifiable environments: given a natural language instruction, an underlying dataset, pre-configured software dependencies, and an executable verification script, coding agents can iteratively debug and optimize solutions across dozens of experiments within hours. However, in the domain of scientific data-driven discovery, the severe scarcity of real-world verifiable environments has become a critical bottleneck holding back open-weight models from mastering complex scientific reasoning.

Existing environment synthesis pipelines in software engineering (such as SWE-bench, SWE-Gym, and RepoST) heavily rely on existing unit test suites embedded within source repositories to serve as deterministic oracle feedback. In sharp contrast, scientific codebases rarely ship with formal unit tests; worse, programs for scientific data analysis generate outputs spanning continuous numerical distributions, high-dimensional manifolds, topological graphs, and spatial geospatial fields whose scientific correctness cannot be assessed via trivial string matching or universal assert statements. Prior data collection initiatives such as AutoSDT crawled large pools of candidate scripts but suffered from synthetic data substitution, lacked sandboxed dependency isolation, and provided no automated verification, collapsing into static supervised fine-tuning corpora without execution feedback.

Overcoming this hurdle requires an automated workflow capable of extracting authentic data flows from genuine scientific repositories without relying on expensive, unscalable manual annotation, while synthesizing domain-grounded evaluation scripts equipped with rigorous acceptance criteria and physical plausibility thresholds. To address this, the authors develop an end-to-end environment construction pipeline that purges mock logic, enforces multimodal execution checks, and decouples evaluation script creation into scientific methodology planning followed by deterministic implementation. Core idea: by combining strict real-data dependency verification, multimodal output adjudication, and a two-stage planning-then-coding evaluation script synthesis, D3-Gym establishes 565 multi-disciplinary verifiable environments that deliver reliable reward signals for training and iterative agent refinement.

Method

Overall Architecture

The D3-Gym construction pipeline consists of four tightly integrated stages: (1) Candidate Task Collection and Rigorous Real-Data Filtering—curating tasks from scientific repositories, discarding any instances relying on synthetic data or mock objects, and utilizing Claude Code to verify file dependencies and generate data schema previews; (2) Sandboxed Execution and Multimodal Output Verification—running reference programs in isolated environments with automatically resolved dependencies (pipreqs) and filtering out non-executable or degenerate outputs using a multimodal LLM-as-a-judge (GPT-5.2); (3) Two-Stage Evaluation Script Synthesis—employing a planning-then-coding approach with Claude Sonnet 4.5 to produce silver evaluation scripts containing task-specific metrics, numerical tolerances, and sanity checks; and (4) Downstream Training and Autonomous Optimization Loops—leveraging verified execution trajectories for rejection sampling fine-tuning (RFT) and serving as an optimization objective in Autoresearch-style search loops.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["AutoSDT Candidate Pool<br/>5,111 raw scientific tasks"] --> B["Real-Data Filtering & Schema Previews<br/>Dependency auditing and schema previews"]
    B --> C["Execution & Multimodal Output Verification<br/>Dependency resolution and degeneracy auditing"]
    C --> D["Two-Stage Evaluation Script Synthesis<br/>Domain planning and code implementation"]
    D --> E["D3-Gym Verifiable Environments<br/>565 verified scientific discovery tasks"]
    E --> F["Downstream Agent Training & Refinement<br/>RFT policy training and Autoresearch search"]

Key Designs

1. Real-Data Filtering and Dataset Preview Generation: Eliminating synthetic artifacts and exposing structural schemas

Prior automated coding dataset pipelines frequently substituted missing data with synthetic toy arrays or mock instances, compromising ecological validity and disrupting realistic scientific distributions. D3-Gym addresses this by deploying an inspection agent that sweeps the reference program's abstract syntax tree and file access calls. Any candidate relying on unittest.mock, patch, custom synthetic generator functions, hardcoded placeholder arrays, or try-except blocks masking missing packages is immediately purged. For every validated file dependency, the agent extracts a structural excerpt revealing the raw schema—such as CSV column headers paired with sample rows, nested JSON structural snippets, molecular sequence prefixes, or binary image dimensions. These previews serve as crucial context during downstream environment execution and evaluation script generation.

2. Isolated Execution and Multimodal Output Verification: Pruning execution failures and degenerate solutions

Passing static file checks does not guarantee runnable code, as scientific workflows frequently crash due to missing native libraries or emit degenerate trivial artifacts (e.g., all-zero matrices, NaN values, or blank plots). Each surviving solution is executed in an isolated Conda container with environment dependencies resolved automatically via pipreqs across diverse packages (e.g., rdkit, ase, pysam, geopandas). The generated artifacts are evaluated by a multimodal LLM judge (GPT-5.2) operating under strict rubrics: all expected output files must exist, contain substantive non-degenerate content, and conform to the scientific domain's expectations. On a human-annotated validation set, the judge achieves 92.31% raw agreement (Cohen's \(\kappa = 0.85\), precision 1.00, recall 0.87), reliably filtering low-fidelity tasks and leaving 565 high-quality environments.

3. Two-Stage Evaluation Script Synthesis: Decoupling scientific evaluation planning from Python script coding

Formulating programmatic verification for open-ended scientific tasks is exceptionally challenging; direct single-turn prompting causes models to blend abstract domain criteria with low-level syntax, yielding overly punitive or vacuous tests. D3-Gym decouples script generation into an explicit planning phase followed by a coding phase. In the planning phase, Claude Sonnet 4.5 reviews the instruction, dataset previews, and verified reference outputs to draft a structured scientific plan specifying appropriate metrics (e.g., opting for AUPRC or balanced F1 over accuracy on skewed biological datasets; combining MAE, RMSE, and \(R^2\) to catch mean-predicting degenerate baselines), reasonable numerical tolerance thresholds (e.g., relative percentage error \(\text{RPE} < 5\%\)), and physical plausibility boundaries (e.g., strictly positive Madelung constants, bounded geospatial temperatures). In the coding phase, a separate LLM call translates the plan into an executable Python script featuring an isolated eval() entry point and automated visual discrepancy checks.

4. Rejection Sampling Fine-Tuning and Self-Distillation: Translating execution feedback into reasoning capability

Standard supervised fine-tuning is constrained to static human references, leaving models incapable of learning from their own exploration errors. D3-Gym's verifiable scripts provide dense, objective verification signals to supervise exploration. For each task, candidate models sample 16 full reasoning trajectories (thinking traces plus final Python scripts), which are executed in sandboxed containers against the synthesized silver evaluation scripts. Only trajectories that successfully pass the evaluation suite are retained. The framework investigates both RFT-Distill (training student models on trajectories generated by a 32B teacher) and RFT-Self (training on the model's own successful verified solutions). This executable feedback loop equips open-weight reasoning models with robust error-recovery, schema adherence, and scientific computing library mastery.

Loss & Training

For both reasoning-enabled thinking models and non-thinking variants, training optimizes the standard autoregressive sequence-to-sequence cross-entropy loss over silver-verified trajectories:

\[\mathcal{L}_{\text{SFT}}(\theta) = - \sum_{t=1}^{T} \log P_\theta(y_t \mid x, y_{<t})\]

where \(x\) represents the prompt containing the task instruction, file locations, and structural schema previews, and \(y = (y_1, \dots, y_T)\) denotes the verified reasoning trace and code solution. For RFT-Self, the training dataset \(\mathcal{D}_{\text{self}}\) consists exclusively of self-generated trajectories satisfying \(\text{SilverEval}(y) = \text{True}\). Models are fine-tuned using LoRA (rank \(r=16\), alpha \(\alpha=32\), dropout \(0.05\)) via the LlamaFactory framework. Training runs across 4 to 8 NVIDIA H100 GPUs using Distributed Data Parallelism (DDP) for 1 epoch, utilizing a cosine learning rate scheduler with an initial learning rate of \(5 \times 10^{-5}\), warmup ratio of 0.05, per-device batch size of 1, and a sequence cutoff length of 32,768 tokens.

Key Experimental Results

Main Results

Models are evaluated on ScienceAgentBench (SAB) and its cleaned counterpart ScienceAgentBench-Verified (SAB-Verified). We report average Success Rate (SR), Valid Execution Rate (VER), and best-of-3 performance (SR@3, VER@3) across three independent inference runs.

Model & Strategy SAB SR (%) SAB VER (%) SAB SR@3 (%) SAB VER@3 (%) SAB-Ver. SR (%) SAB-Ver. SR@3 (%)
Qwen3-4B-Instruct Base 4.2 25.8 5.9 35.3 4.2 5.9
Qwen3-4B-Instruct RFT-Distill 7.2 (+3.0) 30.4 (+4.6) 8.8 (+2.9) 47.1 (+11.8) 7.2 (+3.0) 8.8 (+2.9)
Qwen3-4B Base 4.9 19.6 6.9 27.5 5.2 7.3
Qwen3-4B RFT-Distill 7.2 23.5 8.8 37.3 7.2 8.8
Qwen3-4B RFT-Self 6.9 (+2.0) 23.2 (+3.6) 11.8 (+4.9) 40.2 (+12.7) 6.9 (+1.7) 11.8 (+4.5)
Qwen3-8B Base 6.5 29.7 11.8 43.1 8.2 13.8
Qwen3-8B RFT-Distill 8.5 31.7 14.7 48.0 8.2 13.7
Qwen3-8B RFT-Self 10.1 (+3.6) 36.9 (+7.2) 12.8 (+1.0) 43.1 (0.0) 10.8 (+2.6) 13.7 (-0.1)
Qwen3-14B Base 9.1 35.3 13.7 49.0 10.3 15.0
Qwen3-14B RFT-Distill 10.5 35.0 13.7 48.1 10.5 13.7
Qwen3-14B RFT-Self 13.1 (+4.0) 37.6 (+2.3) 19.6 (+5.9) 54.9 (+5.9) 14.1 (+3.8) 20.6 (+5.6)
Qwen3-32B Base 14.7 36.3 19.6 50.9 15.0 20.0
Qwen3-32B RFT-Self 18.9 (+4.2) 43.8 (+7.5) 27.4 (+7.8) 58.8 (+7.9) 19.3 (+4.3) 28.4 (+8.4)

Ablation Study

To evaluate the fidelity of synthesized silver evaluation scripts against human gold standards, 50 held-out tasks (175 person-hours of Ph.D. annotation) and 424 executable solutions from Claude Opus 4.6 and GPT-5.4 are scored across pass/fail verdicts and 1–5 Likert evaluation logic dimensions.

Evaluation Script Variant Accuracy (%) Specificity (%) Recall (%) Metric Alignment (1–5) Threshold Alignment (1–5) Artifact Alignment (1–5)
Silver Scripts (Full Pipeline) 87.5 91.0 66.1 4.00 3.82 4.20
Direct Prompting Baseline 86.1 100.0 0.0 2.07 2.16 2.66
w/o Evaluation Planning 74.8 86.3 3.4 2.00 2.04 2.68
w/o Dataset Preview 63.2 70.4 18.6 1.96 1.74 2.64
w/o Code Output Access 85.1 98.9 0.0 2.89 2.64 2.62

Key Findings

  • Evaluation planning is crucial for non-trivial recall: Direct Prompting and ablation without planning exhibit an uncalibrated rejection bias (100.0% and 86.3% specificity with near-zero 0.0% and 3.4% recall), rejecting correct scientific solutions indiscriminately. Decoupling planning from coding elevates recall to 66.1% while maintaining 91.0% specificity.
  • On-policy RFT-Self outpaces cross-model distillation at scale: While RFT-Distill aids smaller 4B models, RFT-Self delivers superior gains on 14B (+5.9 SR@3 vs 0.0) and 32B (+7.8 SR@3). Negative log-likelihood (NLL) and perplexity (PPL) analyses confirm that self-generated trajectories align much more closely with the model's learned distribution (Qwen3-14B PPL: 1.30 on self vs. 1.73 on 32B teacher).
  • Narrowing the open-vs-proprietary capability divide: Fine-tuned Qwen3-32B achieves 27.4% SR@3, outperforming larger open MoE architectures such as Qwen3-80B (21.4%) and Qwen3-235B-A22B (22.6%), while closely approaching frontier models like OpenAI o1-preview (34.3%) and Claude Sonnet 4.5 (35.2%).
  • Pronounced reduction in operational error modes: Error analysis reveals that D3-Gym training shrinks data schema errors by 44.7%, data type/shape misalignments by 60.4%, and visualization flaws by 15.0%, enabling agents to generate robust, runnable, and syntactically sound analytical workflows.

Highlights & Insights

  • Cost-effective benchmark synthesis ($3 per environment): Demonstrates that high-fidelity scientific coding sandboxes can be procedurally generated from raw GitHub repositories at a fraction of the cost required by software engineering environments.
  • Autonomous optimization signal (Autoresearch): In case studies spanning German temperature kriging and molecular property prediction with ChebyNet, raw scalar loss feedback from silver scripts guided a Codex agent across 40 iterations to achieve a 6.4x reduction in RMSE (0.7806 down to 0.1221).
  • Surfacing real-world agent failure modes: In open-ended molecular modeling, the environment successfully diagnosed systematic agent limitations: the model overfitted small data splits by scaling up neural parameters over hundreds of epochs rather than debugging data representations, highlighting critical targets for autonomous research safety.

Limitations & Future Work

  • Environment scale and disciplinary boundaries: While covering 565 tasks across 4 disciplines, the dataset remains smaller than software engineering benchmarks due to stringent real-data filtering; extending to astrophysics, materials informatics, and genomics remains an active goal.
  • Reinforcement learning with sparse reward challenges: Current training relies on dense supervision via rejection sampling fine-tuning (RFT); exploring RL frameworks (e.g., PPO or GRPO) with process-supervised reward models represents a compelling but difficult next frontier.
  • Broadening evaluation benchmarks: The primary downstream evaluation focuses on ScienceAgentBench; assessing cross-benchmark generalization on emerging suites like BixBench and DiscoveryBench will provide deeper insights into transferability.
  • vs AutoSDT (Li et al., 2025): AutoSDT pioneered large-scale scientific task scraping for static SFT pairs but lacked executable containers, permitted synthetic dummy data, and offered no verification scripts; D3-Gym establishes 100% executable environments with automated verification logic.
  • vs SWE-Gym / SWE-smith / RepoST: Software engineering environments repurpose existing test suites; D3-Gym tackles scientific workflows that inherently lack pre-existing tests by synthesizing domain-specific evaluation logic from scratch.
  • vs DS-Gym / MLE-Dojo: Previous data science gyms borrow competition setups with standard tabular metrics; D3-Gym draws directly from authentic research repositories featuring complex multivariant scientific representations (VASP, SMILES, GPS, FASTA).

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Pioneering automated synthesis of executable, verifiable environments and domain evaluation scripts for scientific data-driven discovery.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous human-gold script validation across 50 held-out tasks, extensive RFT scaling across four model sizes, and 40-iteration Autoresearch case studies.
  • Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally structured presentation with rich appendix disclosures of pipeline costs, complete prompts, and verified benchmark corrections.
  • Value: ⭐⭐⭐⭐⭐ Fills a crucial infrastructure gap for open-weight scientific reasoning agents, providing reproducible training sandboxes and autonomous research testbeds.