Skip to content

When the Strongest Teacher Is Not the Best Teacher: Student-Centric Answer Selection

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/ppsmk388/Student-Centric-Answer-Selection
Area: LLM Reasoning
Keywords: Knowledge Distillation, Answer Selection, Student-Centric Learning, Gradient Decomposition, Complex Reasoning

TL;DR

Challenging the widespread assumption that the strongest teacher model consistently provides the optimal supervision, this paper introduces Student-Centric Answer Sampling (SCAS), a forward-only, gradient-motivated learning-cost proxy that adaptively samples verified correct teacher solutions best matched to the current student model.

Background & Motivation

In modern large language model (LLM) post-training and knowledge distillation, model-generated synthetic supervision—ranging from chain-of-thought reasoning traces to multi-turn agent interactions—has become an indispensable substitute for expensive, human-annotated data. Prevailing practices routinely adhere to a teacher-centric paradigm: practitioners select the largest, highest-performing flagship model (e.g., proprietary frontier LLMs or 70B+ open models) to generate training trajectories, implicitly treating teacher benchmark competence as an infallible proxy for teaching quality. However, whether superior teacher capability uniformly translates into superior student learning efficiency has remained largely unexamined and assumed without rigorous verification.

A controlled empirical investigation across nine distinct teacher models and multiple Qwen2.5 student scales shatters this long-standing assumption: when restricting the training set strictly to the common subset of mathematical problems for which all candidate teachers produce verified correct answers, teacher test accuracy exhibits a weak or even pronounced negative rank correlation with student downstream accuracy (reaching \(r = -0.70\) on GSM8K and \(r = -0.34\) on DeepScaleR). Even when all candidate teachers arrive at mathematically valid solutions, the latent stylistic jumps, conceptual leaps, and token interaction patterns embedded within their reasoning trajectories impose radically differing optimization burdens on smaller student architectures. A solution from the strongest model often encapsulates compact semantic transitions that a smaller student cannot readily internalize, effectively turning into optimization noise.

The fundamental tension lies in how to move beyond teacher-centric heuristics and dynamically quantify the student-dependent learning burden induced by alternative correct answers without resorting to prohibitively expensive backpropagation. Pairing a prompt with different verified solutions induces distinct hidden states, prediction errors, and parameter gradients within the student. Core idea: decompose the squared Frobenius norm of student weight gradients into token-level interaction blocks to isolate answer-involving dynamics, derive a backward-free forward proxy modulated by final-layer representations and sequence NLL, and perform stratified sampling to iteratively align training data with the evolving student.

Method

Overall Architecture

Student-Centric Answer Sampling (SCAS) formalizes answer selection as minimizing the student-dependent update burden from a candidate pool of verified solutions \(\mathcal{A}(q)\). The end-to-end pipeline operates in four continuous stages: first, candidate answers generated across a diverse 30-teacher pool are filtered using automated execution and reasoning verifiers to guarantee mathematical soundness; second, theoretical analysis decomposes the induced weight gradient norm into answer-answer (AA), answer-conditioning (AQ), and conditioning-conditioning (QQ) blocks, revealing that QQ variance across candidates is negligible; third, an efficient forward-only proxy \(S(q, a; \theta)\) combines mean token NLL with final-layer representation aggregates; finally, candidates are sorted into \(G\) quantile groups where uniform sampling from the lowest-cost group constructs the round-specific training set, with scores adaptively updated across multiple training iterations.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Verified Answer Pool Construction<br/>30 diverse teacher models + automated reasoning verification"] --> B["Token-Wise Gradient Decomposition<br/>Retain AA/AQ blocks while isolating near-constant QQ offset"]
    B --> C["Forward-Only Learning-Cost Proxy<br/>Final-layer hidden states aggregated with sequence mean NLL"]
    C --> D["Stratified Grouped Sampling<br/>Partition into G groups and sample uniformly from lowest-cost group"]
    D --> E["Iterative Student Update<br/>Fine-tune student parameters and refresh candidate scores across R rounds"]

Key Designs

1. Token-Wise Gradient Decomposition: Isolating Student-Centric Learning Burdens Standard data filtering relies on sequence-level heuristics or perplexity, which fail to capture how specific tokens structurally reshape student parameters. To formalize the learning burden, the authors analyze the loss gradient with respect to a generic student weight matrix \(W \in \mathbb{R}^{d \times m}\), given by \(\nabla_W \mathcal{L} = X^\top G\), where \(X\) denotes the pre-weight token representations and \(G\) denotes the loss gradient with respect to pre-activations. Expanding the squared Frobenius norm into pairwise token inner products yields three explicit blocks:

\[\|\nabla_W \mathcal{L}\|_F^2 = \sum_{i,j \in I_A} (z_i^\top z_j)(g_i^\top g_j) + 2 \sum_{i \in I_A, j \in I_Q} (z_i^\top z_j)(g_i^\top g_j) + \sum_{i,j \in I_Q} (z_i^\top z_j)(g_i^\top g_j)\]

where \(I_Q\) and \(I_A\) index the question and answer token positions, respectively. Empirical measurements across student families and benchmarks reveal that the conditioning-conditioning (QQ) block exhibits near-zero variance across different candidate answers for a given prompt (standard deviation \(\le 0.01\)). Consequently, the QQ term functions as an invariant question-level baseline offset. The cross-candidate variance is overwhelmingly dominated by internal answer interactions (AA) and answer-prompt conditioning interactions (AQ/QA), proving that ranking supervision signals requires modeling only the answer-involving blocks.

2. Forward-Only Learning-Cost Proxy: Decoupled Representation and NLL Aggregation Directly evaluating the exact gradient decomposition requires backward passes to retrieve token gradients \(g_i\) for every candidate trajectory, rendering large-scale answer selection computationally infeasible. To eliminate backward passes, SCAS targets the final language modeling head and establishes an inequality linking residual norms \(\|r_t\|_2\) to token negative log-likelihood (NLL). Proving that sequence-level mean NLL serves as the least-squares optimal blockwise constant approximation to token uncertainty, the framework defines a lightweight forward-only proxy:

\[S(q, a; \theta) = (1 - \lambda) C_{AA}(q, a; \theta) + \lambda C_{AQ}(q, a; \theta)\]

Using normalized final-layer hidden representations \(u_t = h_t / \|h_t\|_2\) and their block means \(\mu_A = \frac{1}{|I_A|} \sum_{i \in I_A} u_i\), \(\mu_Q = \frac{1}{|I_Q|} \sum_{j \in I_Q} u_j\), the components are computed as \(C_{AA} = d_A^2 \|\mu_A\|_2^2\) and \(C_{AQ} = d_A d_Q (\mu_A^\top \mu_Q)\), where \(d_A\) and \(d_Q\) represent mean token NLL over answer and question tokens. This proxy reduces per-candidate evaluation time from \(O(F(n) + B(n) + ndm)\) down to \(O(F(n) + nd)\). The resulting score achieves an average Spearman rank correlation of 0.844 with true full-model gradient norms while requiring only a single forward pass.

3. Stratified Grouped Sampling: Robust Selection and Iterative Adaptation Greedily selecting the candidate with the absolute minimum proxy score (Min-score) makes the selection vulnerable to tokenization artifacts, length fluctuations, and minor scoring noise, while also heavily biasing selected data toward a single, homogeneous model family. SCAS implements a robust stratified sampling mechanism: for each question \(q\), verified candidates are sorted by ascending cost \(S(q, a; \theta)\) and partitioned into \(G\) balanced quantile groups \(\Pi_1(q), \dots, \Pi_G(q)\) using floor-based boundaries. The algorithm then draws one solution uniformly at random from the lowest-score group \(\Pi_1(q)\).

This stratified design ensures that while learning cost remains strictly constrained, candidate diversity is preserved—cross-family teacher responses constitute over 64% of sampled answers. Furthermore, SCAS runs across \(R\) rounds of selection and training (default \(R=3\)). As the student model's parameters evolve into \(\theta_{r+1}\), candidate costs are recalculated. Complex solutions that initially presented high learning costs become accessible to the improved student, organically giving rise to an emergent curriculum learning dynamic.

Loss & Training

The student model is fine-tuned on the dynamically assembled subset \(\mathcal{D}_{low} = \{(q, \tilde{a}(q)): q \in \mathcal{D}\}\) using standard autoregressive cross-entropy loss over answer tokens:

\[\mathcal{L}_{SFT}(\theta) = - \sum_{(q, a) \in \mathcal{D}_{low}} \sum_{t \in I_A} \log p_\theta(x_t \mid q, a_{<t})\]

Supervised fine-tuning is implemented via LLaMA-Factory with Transformers and DeepSpeed ZeRO-3 on eight NVIDIA A100 GPUs. Optimization uses AdamW with an initial learning rate of \(1.0 \times 10^{-5}\), scheduled via cosine decay with a 5% warmup ratio across 3 epochs. The maximum sequence length is set to 4,096 tokens, the group count is \(G = 5\), and the trade-off parameter \(\lambda\) is set between 0.4 and 0.6.

Key Experimental Results

Main Results

The evaluation encompasses 30 teacher models, 6 student models across Qwen2.5 and Llama families, 4 mathematical benchmarks (DeepScaleR, GSM8K, MATH, OpenR1-Math), and 2 instruction-following benchmarks (IFEval, LiveBench).

Student Model Method DeepScaleR GSM8K MATH OpenR1-Math IFEval LiveBench Average
Qwen2.5-0.5B RSR 7.13 ± 0.17 26.26 ± 0.35 21.62 ± 0.38 6.40 ± 0.24 22.77 ± 0.20 24.30 ± 0.21 18.08
GRACE 5.67 ± 0.17 37.65 ± 0.53 24.13 ± 0.31 8.30 ± 0.14 23.32 ± 0.19 26.90 ± 0.21 21.00
PPL 4.10 ± 0.14 6.07 ± 0.31 6.44 ± 0.22 6.23 ± 0.38 23.29 ± 0.23 26.30 ± 0.26 12.07
IFD 5.23 ± 0.48 7.00 ± 0.34 6.59 ± 0.21 6.03 ± 0.29 24.03 ± 0.19 27.10 ± 0.14 12.66
Influence 8.77 ± 0.54 7.46 ± 0.16 22.98 ± 0.26 7.63 ± 0.80 23.29 ± 0.33 26.90 ± 0.34 16.17
SCAS (Ours) 10.10 ± 0.85 45.03 ± 0.22 27.62 ± 0.07 10.77 ± 0.19 24.40 ± 0.16 27.20 ± 0.17 24.19
Qwen2.5-3B RSR 21.53 ± 0.29 59.92 ± 0.35 45.02 ± 0.50 17.29 ± 0.45 56.56 ± 0.25 42.20 ± 0.35 40.42
GRACE 28.40 ± 0.65 76.80 ± 0.03 57.55 ± 0.50 30.90 ± 0.78 58.75 ± 0.48 42.30 ± 0.55 49.12
PPL 19.40 ± 0.71 18.88 ± 0.22 16.57 ± 0.36 15.33 ± 1.03 60.40 ± 0.47 39.90 ± 0.50 28.41
IFD 26.80 ± 0.88 18.32 ± 0.04 16.12 ± 0.03 24.57 ± 0.82 60.48 ± 0.42 41.60 ± 0.27 31.32
Influence 27.83 ± 0.73 25.22 ± 0.74 49.98 ± 0.43 28.05 ± 0.65 59.89 ± 0.64 41.60 ± 0.52 38.76
SCAS (Ours) 33.20 ± 0.21 82.82 ± 0.35 61.69 ± 0.09 33.20 ± 0.43 61.37 ± 0.33 42.30 ± 0.20 52.43
Qwen2.5-7B RSR 30.20 ± 0.57 62.55 ± 0.31 53.89 ± 0.31 27.20 ± 0.71 60.07 ± 0.41 47.40 ± 0.37 46.89
GRACE 38.97 ± 1.30 86.85 ± 0.47 65.71 ± 0.21 38.93 ± 0.24 62.70 ± 0.26 47.50 ± 0.18 56.78
PPL 18.03 ± 0.41 20.60 ± 0.47 18.04 ± 0.03 22.17 ± 0.82 62.29 ± 0.40 47.80 ± 0.43 31.49
IFD 17.73 ± 0.78 20.32 ± 0.24 18.25 ± 0.11 21.30 ± 0.62 61.92 ± 0.39 47.90 ± 0.26 31.24
Influence 34.50 ± 0.98 19.61 ± 0.35 18.62 ± 0.31 34.50 ± 0.22 62.29 ± 0.29 47.60 ± 0.25 36.19
SCAS (Ours) 41.97 ± 1.42 89.38 ± 0.16 69.91 ± 0.19 42.13 ± 0.48 68.58 ± 0.32 49.60 ± 0.19 60.26

Ablation Study

The rank correlation between proxy scores and exact ground-truth gradient norms confirms that SCAS accurately mirrors full-parameter optimization signals across diverse tasks:

Model Method DeepScaleR GSM8K MATH OpenR1-Math Magpie Mean Correlation
Qwen2.5-3B GRACE 0.781 0.853 0.829 0.775 0.348 0.717
Influence 0.604 0.676 0.662 0.356 0.261 0.512
IFD 0.025 0.053 -0.074 0.101 0.159 0.053
PPL -0.771 -0.842 -0.809 -0.791 -0.344 -0.712
SCAS (Ours) 0.788 0.883 0.875 0.795 0.572 0.783
Qwen2.5-7B GRACE 0.796 0.891 0.849 0.804 0.628 0.793
Influence 0.183 0.820 0.800 -0.089 0.515 0.446
IFD -0.360 -0.303 -0.483 -0.228 -0.374 -0.349
PPL -0.856 -0.897 -0.871 -0.844 -0.724 -0.838
SCAS (Ours) 0.861 0.904 0.883 0.847 0.726 0.844

Further score component ablations on MATH demonstrate that: 1. Full SCAS consistently achieves the highest accuracy across epochs (reaching 61.69% on Qwen2.5-3B); 2. NLL only and Emb. only both suffer performance drops, verifying that sequence-level likelihood and geometric representation alignment provide strictly complementary selection criteria; 3. Token NLL performs substantially worse, showing that unpooled token uncertainty introduces localized tokenization noise; 4. Min score (removing grouping and greedily taking the minimum) lags behind SCAS, underscoring the stabilizing role of stratified grouping.

Key Findings

  • The Best Teacher Myth Is Broken: Even with verified correct answers, pooling larger unions of answers from top teachers (Top-k Union, up to 9x training data) fails to match SCAS, lagging by 3.14 to 4.70 points on MATH. Data learnability outweighs raw data quantity and teacher prominence.
  • Superior Compute Efficiency: Evaluating 1,000 data items (each with 30 candidate answers) takes 64.9 seconds on Qwen2.5-3B and 79.9 seconds on Qwen2.5-7B, while exact backward-based gradient estimation requires over 600–700 seconds.
  • Deeper Layers Yield Superior Proxies: Ablating intermediate Transformer layers shows monotonic performance gains as extraction moves deeper, with the final hidden layer adjacent to the LM head delivering peak selection accuracy.

Highlights & Insights

  • Theoretically Rigorous Backward-Free Formulation: Rather than relying on heuristic perplexity metrics, the method grounds answer learnability in second-order matrix calculus of weight gradient norms, formally bounding the forward proxy error via mean-field uncertainty reduction.
  • Curriculum-Like Dynamic Adaptation: Multi-round re-scoring allows candidate selection to evolve in step with student proficiency, naturally shifting from simpler, low-cost explanations to sophisticated, concise proofs.
  • Broad Domain Generalization: The framework extends seamlessly beyond arithmetic to formal logical reasoning on ZebraLogicBench (outperforming the strongest baseline by up to 4.9 points) and open-ended instruction following on IFEval.

Limitations & Future Work

  • Exploration Restricted to Mid-Scale Students: Primary empirical validation focuses on 0.5B to 8B parameter student models; the interplay between teacher competence and student cost for larger 32B/70B+ students remains open for future exploration.
  • Offline Multi-Teacher Generation Cost: Constructing the initial 30-teacher candidate pool incurs significant upfront inference costs, suggesting that on-policy candidate generation via self-sampling or Monte Carlo search is a promising next step.
  • Integration with RL Alignment: Future research can explore adapting the learning-cost proxy as an intrinsic regularizer or token-level advantage weighting within online PPO/GRPO pipelines.
  • vs GRACE (Panigrahi et al., 2025): GRACE operates at the coarse model level by pairing a student with an optimal teacher; SCAS operates at the fine-grained answer instance level, recognizing that even the best teacher produces suboptimal trajectories for specific questions.
  • vs RSR / RSR-Item (Yang et al., 2026b): While RSR measures reasoning trajectory alignment, it lacks explicit mathematical derivation tied to weight gradient norms, resulting in inferior downstream accuracy and lower throughput compared to SCAS.
  • vs Classical IFD / PPL Filtering: Traditional perplexity metrics overlook representation-level token interactions and frequently correlate negatively with exact optimization directions, whereas SCAS maintains positive alignment with true gradient norms.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Formulates student-centric answer selection through a rigorous gradient norm decomposition and an elegant forward-only proxy.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluation across 30 teachers, 6 students, and 7 diverse benchmarks, supported by extensive ablation and layer analyses.
  • Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally clear narrative, structured mathematical derivations, and thorough appendix proofs.
  • Value: ⭐⭐⭐⭐⭐ High-impact contribution to post-training data curation, knowledge distillation, and reasoning model alignment.