Skip to content

CoDiQ: Test-Time Scaling for Controllable Difficult Question Generation

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/xxx
Area: LLM Reasoning
Keywords: test-time scaling, question generation, reinforcement learning, curriculum learning, verifier

TL;DR

Pioneers depth-wise test-time scaling for question generation through CoDiQ, integrating six cognitive scaffolding strategies, hybrid difficulty-solvability verification, and breaking-point reinforcement learning to synthesize competition-grade reasoning problems with high fidelity.

Background & Motivation

Large Reasoning Models (LRMs) have achieved unprecedented milestones across complex mathematics competitions and competitive programming benchmarks, with the availability of challenging, verifier-approved reasoning data serving as a foundational catalyst. However, automated question synthesis pipelines encounter three formidable structural bottlenecks: first, the generator capacity ceiling, where models struggle to synthesize problems substantially more difficult than what their internal mechanisms can reliably solve; second, the solvability-complexity trade-off, where aggressively introducing constraints risks breaking logical validity, resulting in vacuous, ambiguous, or unsolvable questions; and finally, the absence of continuous, non-saturating difficulty proxies, which reduces naive prompt-based escalation into arbitrary stylistic rewrites.

The core tension stems from the mismatch between traditional test-time compute—which scales horizontally via Best-of-N sampling or tree search for fixed questions—and the requirement for vertical escalation that sequentially evolves the underlying problem structure toward the model's capability frontier. Concurrently, absolute difficulty metrics quickly saturate near the frontier, while models frequently generate "fake hard" instances, such as stacking redundant modulo constraints that collapse trivially through the Chinese Remainder Theorem without altering reasoning depth.

This paper's angle of attack is: introduce depth-wise test-time scaling into question evolution, steering the generator along its capability boundary under rigorous, round-by-round solvability and relative difficulty checks. Core idea: develop the CoDiQ framework for depth-wise test-time question synthesis, coupling six cognitive scaffolding strategies with dual relative-difficulty estimation and aligning a dedicated generator via reinforcement learning on its own breaking-point failure trajectories.

Method

Overall Architecture

The CoDiQ pipeline systematically scales problem difficulty starting from a seed problem \(Q_0\) across up to \(T_{\max}=8\) sequential rounds. At each round \(i\), the generator produces an upgraded candidate \(Q_i\) under explicit cognitive scaffolding. The candidate is evaluated by two coupled modules: a difficulty estimation module that combines listwise LLM ranking and an early-hidden-state ValueNetwork to enforce non-decreasing relative difficulty (\(\Delta(D) \ge 0\)), and a solvability verification module based on an advanced reasoner (Qwen3-32B) that audits logical completeness and consistency. If a candidate degrades difficulty or fails verification, the pipeline rolls back to \(Q_{i-1}\), terminates, and outputs the approved sequence. To surpass baseline generation limits, the CoDiQ-Generator is trained via reinforcement learning directly on these breaking-point trajectories.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Seed Question Q0<br/>Math / Code Benchmarks"] --> B["Difficulty-Enhancement Strategies<br/>Six Cognitive Scaffolds"]
    B --> C["Dual Difficulty Estimation<br/>LLM Listwise Ranking + ValueNetwork"]
    C --> D{"Difficulty Non-decreasing?<br/>di >= di-1"}
    D -->|No / Decreased| G["Roll-back to Qi-1<br/>Emit Verified Sequence"]
    D -->|Yes| E["Solvability Verification<br/>Qwen3-32B Logical Audit"]
    E -->|Rejected / Unsolvable| G
    E -->|Approved| F{"Max Rounds Reached<br/>or Stop Criteria?"}
    F -->|No| B
    F -->|Yes| H["CoDiQ-Corpus Output<br/>Progressive Difficulty Strata"]
    G --> I["Breaking-Point Trajectory RL<br/>Train CoDiQ-Generator"]

Key Designs

1. Cognitive scaffolding via six Difficulty-Enhancement Strategies: eliminating trivial superficial escalation

To prevent the generator from defaulting to trivial modifications (such as raising array size \(N\) to \(10^5\) while maintaining an \(O(N)\) linear scan), the framework specifies six structured enhancement strategies: Dimensionality & Constraints (introducing dynamic updates or multi-dimensional queries to compel algorithmic complexity shifts), Mathematical Abstraction (recasting procedural problems into closed-form number theoretic, combinatoric, or game-theoretic models), Inverse & Constructive (reversing input-output directions to force structural synthesis), State Explosion (injecting historical dependencies to expand DP states), Theorem Disguise (masking core theorems like Network Flow or Linear Basis behind deceptive narratives), and Edge Case & Rigor Engineering (targeting extreme degenerate boundaries, precision pitfalls, and formal invariants). These strategies act as explicit prompts guiding the generator toward deep logical transformations.

2. Dual relative-difficulty estimation: preventing frontier score saturation

Because evaluating questions near the model's capability frontier causes absolute difficulty metrics to saturate, CoDiQ incorporates relative estimation combining explicit LLM-Ranking (DR-LLM) and implicit ValueNetwork scoring (DR-VN). DR-LLM employs Doubao-Seed-1.8 to perform listwise ranking over query batches with stochastic shuffling, grouping questions with negligible differences into \(G \le n\) ordered bins where group \(j\) receives normalized rank: $\(d_i = \frac{j - 1}{G - 1}, \quad j \in \{1, \dots, G\}\)$ Simultaneously, DR-VN extracts Qwen3-8B hidden states using a quadratic sampling schedule concentrated on early reasoning tokens (within a window \(\le 4,096\) tokens) and trains a lightweight MLP with binary cross-entropy to predict correctness. Lower predicted correctness directly correlates with higher cognitive load. Both scores are scaled to \([0, 100]\) and averaged as DR(AVG), enforcing the monotonic constraint \(\Delta(D) = d_i - d_{i-1} \ge 0\).

3. Solvability verification and rollback termination: screening fake complexity and logical collapse

Increasing structural constraints frequently triggers latent contradictions, missing parameters, or extreme solution space sparsity. CoDiQ employs Qwen3-32B to execute two-stage verification: first producing structured evidence analyzing preconditions and edge cases, followed by a binary decision with confidence score. When candidate \(Q_i\) is flagged as unsolvable or ambiguous, the pipeline executes a strict rollback rule, discarding \(Q_i\) and outputting the prefix \(\{Q_0, \dots, Q_{i-1}\}\), thereby guaranteeing that every retained sequence is verifier-approved end-to-end.

4. Breaking-point trajectory reinforcement learning: tracking the generator's capability frontier

Rather than fine-tuning on average-case difficulties, the training framework harvests trajectories from pipeline runs that were accepted through rounds \(1\) to \(i-1\) but rejected at round \(i\). Pairing the failed boundary candidate with the last valid state converts failure points into precise frontier-aligned learning signals. Utilizing GRPO within the VeRL framework, Qwen3-8B is optimized using a tiered, difficulty-aware reward: $\(r = \begin{cases} 0, & \text{invalid} \\ w_1 \cdot \text{conf}, & \Delta(D) = 0 \\ w_2 \cdot \text{conf} + (1 - w_2) \cdot (w_3 + (1 - w_3)\Delta(D)), & \Delta(D) > 0 \end{cases}\)$ with hyperparameters \(w_1=0.6, w_2=0.2, w_3=0.8\). This partitions the reward into three discrete bands (invalid \(\to 0\), stagnant \(\to [0.30, 0.60]\), increased \(\to [0.74, 1.00]\)), compelling the model to maximize difficulty progression while strictly respecting solvability constraints.

A Worked Example

Appendix C details an end-to-end subsequence counting problem evolution: - Seed Question \(Q_0\): Count subsequences from an array nums with an odd sum modulo \(10^9+7\). Solvable via standard DP with 2 states in \(O(N)\) time. - Round 1 Evolution \(Q_1\): Adds constraints requiring even subsequence length and sum \(\text{mod } 3 = 1\). The state space expands to \(2 \times 2 \times 3 = 12\) states with time complexity \(O(12N)\) and solution density \(\sim 8\%\). Verifier assessment: Solvability score 0.90, Difficulty increased (PASS). - Round 2 Evolution \(Q_2\): Adds constraints: sum \(\text{mod } 5 = 2\) and length \(\text{mod } 4 = 2\). By the Chinese Remainder Theorem (CRT), sum conditions unify to \(\text{mod } 30 = 7\), yielding an effective state space of \(30 \times 4 = 120\) states with time complexity \(O(120N)\) and solution density \(\sim 0.83\%\). Verifier assessment: Solvability score 0.85, Difficulty unchanged (PASS). - Round 3 Evolution Failure \(Q_3\): The generator stacks sum \(\text{mod } 7 = 4\) and sum \(\text{mod } 11 = 6\). By CRT, the state space explodes to \(2,310 \times 8 = 18,480\) states. Although mathematically consistent, the solution space collapses: the probability of a random subsequence satisfying all conditions is \(\approx 1/18480\), rendering the answer 0 for moderate \(N\) and making the problem trivial. Verifier assessment: Solvability FAIL. The pipeline terminates, discards \(Q_3\), and outputs \(\{Q_0, Q_1, Q_2\}\).

Key Experimental Results

Main Results

Evaluated on CoDiQ-Bench (100 mathematics and 100 competitive programming problems), Table 1 reports the maximum verifier-approved difficulty achieved across 8 rounds by long-CoT models:

Model Generation Setting Accepted Rounds Generated Tokens DR-LLM DR-VN DR(AVG)
Qwen3-8B Direct Prompt 3.4 1,130.5 39.2 59.6 49.4
GLM-4.6 Direct Prompt 2.8 3,385.8 71.2 65.8 68.5
GPT-OSS-20B Direct Prompt 2.9 5,528.2 68.5 74.4 71.5
Qwen3-8B CoDiQ Prompt 2.4 4,155.6 49.8 41.9 45.8
Qwen3-32B CoDiQ Prompt 2.2 4,893.6 63.0 46.5 54.8
GLM-4.6 CoDiQ Prompt 2.7 7,143.8 73.2 83.3 78.3
CoDiQ-Gen-8B (Ours) CoDiQ Pipeline 3.4 7,499.6 58.9 58.1 58.5

In dataset-level evaluations across 300 sampled questions, solver token consumption and expert human rankings (DR-Human) confirm the extreme difficulty of the synthesized CoDiQ-Corpus (44,453 sequences):

Dataset Corpus Source Response Tokens ↑ DR-LLM DR-VN DR-Human DR(AVG)
APPS Human-Curated 12,122 39.1 44.8 38.8 40.9
LiveCodeBench Human-Curated 11,914 46.3 45.2 43.8 45.1
AIME (1983–2024) Human-Curated 14,197 59.0 46.6 58.0 54.5
ScaleQuest-Math Synthetic 5,592 15.0 30.5 7.2 17.6
ScaleDiff-Math Synthetic 14,341 50.0 52.5 45.2 49.2
PromptCoT-2.0 Synthetic 16,817 64.8 68.3 68.4 67.2
CoDiQ-Corpus Synthetic (Ours) 17,955 91.5 87.4 95.0 91.3

Ablation Study

Ablation on downstream curriculum reinforcement learning shows that stratifying post-training data across three difficulty strata (L1: seed origin, L2: intermediate stage, L3: hardest frontier) unlocks consistent reasoning gains across major benchmarks:

Model Config Training Strategy MATH-500 AIME 2024 MBPP LiveCodeBench v5
Qwen3-4B Base Model 94.4 63.1 66.4 53.6
Qwen3-RL-4B Standard RL (Unstratified) 95.2 64.3 66.4 52.0
Qwen3-ScaleQuest-4B ScaleQuest Data 95.8 63.1 66.8 51.5
Qwen3-PromptCoT2-4B PromptCoT2 Data 94.8 70.0 67.0 53.6
CoDiQ-L1-4B Curriculum Stage 1 96.0 65.0 65.2 53.2
CoDiQ-L2-4B Curriculum Stage 2 94.8 66.7 66.6 54.8
CoDiQ-L3-4B Curriculum Stage 3 (Full) 96.0 70.6 67.6 54.2
Qwen3-8B Base Model 96.2 70.0 69.2 56.9
Qwen3-PromptCoT2-8B PromptCoT2 Data 96.2 71.5 69.2 57.6
CoDiQ-L3-8B Curriculum Stage 3 (Full) 97.0 73.2 70.2 58.0

In the unfiltered difficulty generation ablation (removing the solvability verifier), CoDiQ-Gen-8B reaches a DR(AVG) of 62.4, outperforming the much larger base model Qwen3-14B (60.9) and confirming that RL alignment effectively unlocks deeper structural synthesis beyond raw parameter scale.

Key Findings

  • Scaling Tendency under Test-Time Compute: Across fixed enhancement prompts and competent verification horizons, reasoning token volume exhibits a strong positive correlation with estimated difficulty (\(r = 0.8962, p < 0.001\)). On natural human-graded problems from MATH-500, Qwen3-32B solver tokens monotonically scale by over \(13\times\) from Level 1 (337 tokens) to Level 5 (4,705 tokens).
  • Depth-vs-Approval Trade-off: Progressively scaling difficulty exposes an empirical trade-off: verifier approval rates collapse sharply in later rounds (often dropping below 20% in smaller models due to logical derailment), whereas CoDiQ-Generator sustains stability and extends average approved rounds to 3.4.
  • Benchmark-Dependent Stratification Gains: Moderately complex benchmarks (MBPP, MATH-500) saturate performance improvements using low-to-mid complexity strata (L1/L2), whereas competition-grade evaluations (AIME 2024, LiveCodeBench v5) strictly require L3 data to drive gains.

Highlights & Insights

  • Breaking-point frontier alignment: Training on near-miss failure cases where a model succeeded for \(i-1\) rounds before violating validity at round \(i\) provides an exceptional boundary learning signal, allowing an 8B model to surpass 32B base models in difficult question synthesis.
  • Early-state value prediction for difficulty: Utilizing a quadratic sampling schedule on early reasoning hidden states to predict solver success offers a non-saturating, continuous difficulty proxy that captures cognitive complexity before answer generation completes.
  • Domain-distribution fidelity: In taxonomic evaluations across 600+ subcategories, CoDiQ-Corpus achieves the lowest Jensen-Shannon divergence (JSD) against human competition corpora (APPS and Code-Contests), verifying that difficulty scaling broadens the long tail without distorting natural competition domain profiles.

Limitations & Future Work

  • The Verifier Paradox and false negatives: Model-based verifiers are bounded by their own reasoning horizon. When generated problems exceed the verifier's deductive capacity, legitimately valid, innovative problems may be discarded as unsolvable.
  • Domain constraint to formal reasoning: CoDiQ is evaluated strictly on closed-world mathematics and algorithmic competitive programming. Transferring depth-wise scaling to reading comprehension, factual question answering, or multimodal reasoning requires grounding strategies and external verifiers that are not yet explored.
  • Inference latency of iterative synthesis: Depth-wise sequential refinement and multi-model verification incur high test-time compute, limiting applicability in interactive or latency-sensitive environments.
  • vs ScaleQuest / ScaleDiff: ScaleQuest prioritizes 0-to-1 synthesis from scratch, while ScaleDiff trains generators on static hard subsets without step-by-step difficulty control. CoDiQ operates in a seed-anchored 1-to-N regime with explicit round-by-round trajectory filtering and fine-grained knobs.
  • vs PromptCoT 2.0: PromptCoT relies on concept sampling and single-pass rejection sampling, which experiences exponential yield decay on competition-grade problems. CoDiQ decomposes structural escalation into an iterative trajectory with rollbacks, vastly improving sample efficiency.
  • vs AutoCode / AutoCodeBench: Focuses heavily on code problem statements paired with test-suite execution, but lacks multi-dimensional cognitive scaffolding (e.g., Theorem Disguise, State Space Explosion) needed to force higher computational complexity classes.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Establishes depth-wise test-time scaling for question generation and pioneers breaking-point trajectory RL.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive multi-model benchmarking, human expert verification (\(\kappa=0.76/0.78\)), taxonomic JSD modeling, and downstream curriculum RL.
  • Writing Quality: ⭐⭐⭐⭐⭐ Rigorous analysis of the solvability-complexity trade-off and precise characterization of boundary conditions.
  • Value: ⭐⭐⭐⭐⭐ The open-sourced 44K corpus and specialized generator provide an impactful asset for post-training reasoning models.