Skip to content

Training-free LLM Verification via Recycling Few-shot Examples

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/dongseok1220/Referi
Area: Information Retrieval / RAG
Keywords: LLM Verification, Few-shot Learning, Best-of-N, Bayes Decomposition, In-Context Learning

TL;DR

ReFeri introduces a training-free framework that recycles few-shot examples for both generation and post-hoc verification, combining forward likelihood confidence with a leave-one-out backward reconstruction penalty to suppress demonstration-specific artifacts and achieve an 8.2% relative gain across seven diverse reasoning benchmarks.

Background & Motivation

Large language models (LLMs) have demonstrated exceptional capabilities across multi-step mathematical reasoning, programmatic problem-solving, and open-domain text synthesis. However, the stochasticity inherent in autoregressive token decoding remains a fundamental bottleneck: different sampling trajectories frequently derive contradictory conclusions from identical inputs. While majority voting approaches such as Self-Consistency have become the standard heuristic for aggregating multiple candidate solutions, they strictly require outputs to be cleanly parsed and reduced to canonical labels, completely breaking down in open-ended generation tasks like personalized dialogue, headline creation, and code generation. Conversely, supervised Best-of-N reranking using outcome reward models (ORMs) or step-level process reward models (PRMs) demands prohibitive human annotation costs and extensive domain-specific fine-tuning, severely collapsing when transferred to out-of-distribution reasoning domains.

To bypass the substantial overhead of external training, researchers have actively explored training-free selection mechanisms, including LLM-as-a-Judge and confidence-based uncertainty metrics derived from next-token predictive distributions. Unfortunately, these methods remain bounded by the intrinsic parametric knowledge of the evaluator LLM. When deployed on unfamiliar domains, under complex logical constraints, or across mismatched generator-evaluator pairings, internal model confidence becomes severely miscalibrated, frequently collapsing into uniformly high entropy or flat certainty where subtle quality differences cannot be discriminated. Concurrently, with the ascendancy of instruction tuning, few-shot in-context demonstrations are increasingly treated as disposable formatting scaffolds rather than valuable epistemic anchors, completely neglecting their potential for post-hoc verification.

This paper challenges this convention by posing a foundational question: since few-shot demonstrations provide the most direct, high-fidelity manifestation of human intent and logical priors, why should they be discarded once the output is sampled? Demonstrations can serve as robust, external reference points to counteract internal calibration collapse. Core idea: Ground candidate verification in few-shot demonstrations by using Bayes' rule to decompose posterior log-likelihood into a forward confidence score and a leave-one-out backward reconstruction penalty, penalizing candidates that mimic superficial exemplar artifacts while rewarding genuine reasoning logic.

Method

Overall Architecture

Given a test query \(\tilde{q}\), a fixed set of \(K\) few-shot demonstrations \(\mathcal{X} = \{x_i\}_{i=1}^K\) where each exemplar is a query-answer pair \(x_i = (q_i, a_i)\), and a generative language model \(\mathcal{M}\), the system first samples \(N\) distinct candidate responses \(\{r_1, \dots, r_N\}\) via few-shot chain-of-thought prompting. ReFeri's objective is to evaluate each candidate pair \(y_n = (\tilde{q}, r_n)\) using a white-box estimation model \(\mathcal{P}\) (which may be identical to \(\mathcal{M}\) or a significantly lighter, separate model) to reliably select the optimal response \(r_{n^*}\) without updating any parameters.

The operational pipeline operates across four strictly ordered phases: first, an embedding retriever identifies the single demonstration exemplar that is semantically closest to the test query; second, the forward confidence score is computed by evaluating the sequence likelihood of the candidate conditioned on the full exemplar set; third, a leave-one-out prompt substitution strategy constructs a counterfactual context to measure how strongly the candidate reconstructs the retrieved exemplar's ground truth, serving as a backward overfitting penalty; finally, the two terms are subtracted under a principled Bayesian identity to yield the final selection metric.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Query & Candidate Generation<br/>q~ samples N distinct responses r_n"] --> B["Nearest Exemplar Anchoring<br/>Lightweight embedding picks closest x_i†"]
    B --> C["Forward Confidence Scoring<br/>Evaluate candidate likelihood S_Forw | X"]
    C --> D["Leave-One-Out Backward Penalty<br/>Substitute exemplar to compute S_Back"]
    D --> E["Bayesian Decomposition Selection<br/>S_Fin = S_Forw - S_Back picks best r_n*"]

Key Designs

1. Nearest Exemplar Anchoring: Prohibitive linear computational scaling and distant irrelevant exemplar noise -> Cosine semantic retrieval via lightweight dense embeddings -> Constant-time localized manifold grounding Computing backward reconstruction across all \(K\) few-shot examples incurs an \(\mathcal{O}(K)\) inference cost per candidate, which is computationally prohibitive in real-world test-time scaling. More critically, distant, semantically orthogonal demonstrations contribute negligible or noisy signals to the specific logical chain of the test query. To address this, ReFeri deploys a lightweight 110M-parameter embedding model \(\mathcal{E}\) (all-mpnet-base-v2) to identify the single most relevant in-context exemplar: $\(i^{\dagger} = \arg\max_{i=1,\dots,K} \cos\left(\mathcal{E}(\tilde{q}), \mathcal{E}(q_i)\right)\)$ This operation bounds the backward computation to \(\mathcal{O}(1)\), isolating verification to the immediate semantic neighborhood and ensuring maximum cost efficiency during deployment.

2. Forward Confidence Scoring: Direct marginal likelihood suffers from severe calibration collapse in unfamiliar domains -> Autoregressive likelihood conditioned on full demonstration context -> Context-grounded adherence evaluation Conventional likelihood selection directly scores \(\log \mathcal{P}(r_n \mid \tilde{q})\), relying solely on internal parametric knowledge that degrades severely on out-of-domain reasoning benchmarks. Under Bayes' theorem, ReFeri conditions candidate likelihood on the full set of few-shot demonstrations \(\mathcal{X}\): $\(S_{\text{Forw}}(y_n) := \log \mathcal{P}(y_n \mid \mathcal{X}) = \frac{1}{L} \sum_{t=1}^L \log \mathcal{P}(r_{n,t} \mid \tilde{q}, \mathcal{X}, r_{n,<t})\)$ By anchoring token probability estimation within human-curated demonstration reasoning trajectories, the forward score directly quantifies whether a candidate faithfully follows the demonstrated problem-solving schema and stylistic guidelines, filtering out hallucinated and logically unstructured answers.

3. Leave-One-Out Backward Penalty: High forward scores favor superficial mimicry of exemplar artifacts -> Backward reconstruction via leave-one-out prompt substitution -> Quantitative penalty against artifact-driven memorization Relying solely on forward confidence creates a perilous failure mode: candidates that mechanically mimic demonstration-specific lexical artifacts, formatting idiosyncrasies, or shared terminal patterns receive artificially inflated likelihoods. Grounded in the Bayesian expansion \(\log \mathcal{P}(y_n) = \log \mathcal{P}(y_n \mid \mathcal{X}) - [\log \mathcal{P}(\mathcal{X} \mid y_n) - \log \mathcal{P}(\mathcal{X})]\), the backward term measures how much conditioning on candidate pair \(y_n\) increases the likelihood of reconstructing the demonstrations. ReFeri operationalizes this via a leave-one-out substitution strategy, replacing exemplar \(x_{i^{\dagger}} = (q_{i^{\dagger}}, a_{i^{\dagger}})\) with \(y_n\) to form the surrogate demonstration set \(\tilde{\mathcal{X}}_{i^{\dagger}} = (\mathcal{X} \setminus \{x_{i^{\dagger}}\}) \cup \{y_n\}\). The approximated backward reconstruction penalty is formalized as: $\(\tilde{S}_{\text{Back}}(y_n) := \log \mathcal{P}\left(a_{i^{\dagger}} \mid q_{i^{\dagger}}, \tilde{\mathcal{X}}_{i^{\dagger}}\right) - \log \mathcal{P}\left(a_{i^{\dagger}} \mid q_{i^{\dagger}}\right)\)$ Retaining the background exemplars \(\mathcal{X}_{-i^{\dagger}}\) preserves the in-context learning baseline of \(\mathcal{P}\), isolating the exact marginal gain induced by \(y_n\). Candidates that merely echo exemplar artifacts trigger a massive backward term and are penalized accordingly.

4. Bayesian Decomposition Selection: Raw individual signals are vulnerable to length bias and uncalibrated variance -> Difference-based fusion of orthogonal guidance and regularization -> Robust and uncorrupted candidate ranking Empirical inspection proves that the raw backward score does not correlate with ground-truth correctness (\(|r| \le 0.09\)) and has only a 0.22 agreement rate with forward selection (equivalent to random overlap). Thus, the backward term is not a standalone predictor, but rather an essential regularizer. Motivated by Bayes' rule, ReFeri combines the two orthogonal signals via direct difference: $\(S_{\text{Fin}}(y_n) := S_{\text{Forw}}(y_n) - \tilde{S}_{\text{Back}}(y_n), \quad r_{n^*} = \arg\max_{n=1,\dots,N} S_{\text{Fin}}(y_n)\)$ This subtraction counterbalances prompt-dependent artifact exploitation against genuine demonstration compliance, delivering an exceptionally stable selection policy.

Benchmark & Evaluation Protocol

The framework is rigorously validated on seven diverse, high-difficulty reasoning benchmarks: competition mathematics (MATH500), multi-domain expert reasoning (MMLU-Pro), graduate-level Google-proof science (GPQA-Diamond), discrete numerical reading comprehension (DROP), multi-hop passage synthesis (HotpotQA), and long-narrative neuro-symbolic reasoning (MuSR-op and MuSR-ta). Beyond reasoning, it is tested on open-ended generation tasks: LLM personalization (LaMP-4 news headlines and LaMP-5 scholarly titles) and executable code generation (LiveCodeBench). Generative models span closed-source proprietary APIs (GPT-4o, GPT-4o-mini) and open-weights instruction models (LLaMA-3.1-8B-Instruct), evaluated under temperature 1.0 with \(N=5\) candidate paths. Baselines include Zero-shot CoT, Few-shot CoT (equivalent to random candidate selection), LEAP (self-reflection from mistakes), USC (Universal Self-Consistency via prompt-based LLM judgment), CoT-WP (token probability gap), and Self-Certainty (predictive entropy). LLaMA-3.1-8B-Instruct serves as the primary likelihood estimator, with LLaMA-3.2-1B-Instruct demonstrating lightweight scaling.

Key Experimental Results

Main Results

The table below reports performance averaged over three independent sampling runs across all seven reasoning benchmarks (DROP and HotpotQA report EM / F1):

Generator Selection Method MuSR-ta (Acc.) MuSR-op (Acc.) GPQA (Acc.) MATH500 (Acc.) DROP (EM / F1) HotpotQA (EM / F1) MMLU-PRO (Acc.) 7-Task Avg.
GPT-4o Few-shot CoT (Random) 87.2 ±0.6 69.6 ±0.6 47.3 ±1.7 75.5 ±0.1 80.4 / 89.0 44.9 / 58.6 73.7 ±0.2 68.4 ±0.3
GPT-4o LEAP 88.1 ±1.6 68.0 ±1.2 47.8 ±2.8 75.2 ±0.4 81.0 / 89.4 44.5 / 57.8 73.9 ±0.2 68.4 ±0.4
GPT-4o USC (LLM-Judge) 85.9 ±1.9 71.5 ±0.7 48.2 ±1.6 77.3 ±0.3 81.8 / 90.2 45.9 / 60.2 74.9 ±0.5 69.4 ±0.5
GPT-4o CoT-WP 88.3 ±0.2 67.5 ±1.4 49.5 ±1.8 78.1 ±0.6 83.3 / 90.5 46.3 / 59.6 74.4 ±0.3 69.6 ±0.0
GPT-4o Self-Certainty 88.9 ±0.5 71.1 ±2.1 50.2 ±0.8 77.7 ±0.4 81.1 / 89.4 43.9 / 57.7 74.6 ±0.4 69.6 ±0.4
GPT-4o ReFeri (8B Estimator) 90.8 ±0.4 73.7 ±2.2 51.8 ±0.3 78.5 ±0.6 83.6 / 90.9 46.9 / 61.0 75.4 ±0.3 71.5 ±0.4
GPT-4o ReFeri1B (1B Estimator) 90.7 ±0.2 74.1 ±3.8 51.0 ±1.3 78.3 ±0.6 83.3 / 90.7 46.5 / 60.5 75.1 ±0.1 71.3 ±0.4
GPT-4o-mini Few-shot CoT (Random) 77.5 ±0.5 60.3 ±0.8 42.4 ±1.1 74.7 ±0.7 76.5 / 82.9 33.6 / 44.9 63.0 ±0.2 61.1 ±0.2
GPT-4o-mini USC 76.5 ±2.5 60.5 ±1.0 44.6 ±1.2 76.4 ±1.2 78.8 / 85.0 35.1 / 47.0 64.2 ±0.5 62.3 ±0.5
GPT-4o-mini CoT-WP 79.3 ±1.3 58.5 ±2.6 41.9 ±0.9 77.0 ±0.7 76.9 / 82.7 34.7 / 45.9 64.6 ±0.4 61.8 ±0.5
GPT-4o-mini Self-Certainty 81.7 ±1.6 60.1 ±0.6 41.6 ±2.8 76.8 ±1.0 76.7 / 83.1 34.7 / 45.9 63.9 ±0.8 62.2 ±0.1
GPT-4o-mini ReFeri (8B Estimator) 83.1 ±0.2 62.0 ±0.6 44.6 ±2.6 78.2 ±0.4 79.1 / 84.7 35.7 / 47.4 64.9 ±0.4 63.9 ±0.5
GPT-4o-mini ReFeri1B (1B Estimator) 82.8 ±0.4 62.9 ±1.4 45.3 ±1.2 77.7 ±0.2 78.3 / 83.9 35.1 / 46.6 64.6 ±0.5 63.8 ±0.2
LLaMA-3.1-8B Few-shot CoT (Random) 64.5 ±2.2 53.4 ±0.8 23.7 ±0.3 42.4 ±0.5 61.1 / 66.5 19.3 / 25.4 39.2 ±0.4 43.4 ±0.5
LLaMA-3.1-8B USC 67.7 ±4.5 54.8 ±2.2 28.1 ±3.1 48.5 ±1.2 68.6 / 74.3 25.5 / 33.4 45.8 ±0.4 48.4 ±1.1
LLaMA-3.1-8B CoT-WP 71.9 ±0.6 53.1 ±1.4 30.5 ±2.5 48.1 ±1.7 71.0 / 75.1 25.7 / 33.2 45.4 ±0.7 49.4 ±0.4
LLaMA-3.1-8B Self-Certainty 72.4 ±3.6 55.9 ±0.6 30.8 ±2.6 50.7 ±2.2 70.8 / 76.2 25.2 / 32.9 44.5 ±0.8 50.0 ±1.1
LLaMA-3.1-8B ReFeri (8B Estimator) 76.5 ±3.4 57.5 ±0.5 33.5 ±1.7 50.2 ±1.6 70.9 / 76.3 25.7 / 33.6 44.7 ±0.5 51.3 ±0.6
LLaMA-3.1-8B ReFeri1B (1B Estimator) 76.9 ±3.2 55.5 ±0.8 31.8 ±2.0 50.1 ±1.7 70.9 / 75.8 25.3 / 32.9 44.2 ±0.5 50.7 ±0.5

Ablation Study

The contribution of individual scoring terms was rigorously dissected across LLaMA-3.1-8B candidate pools, spanning direct marginal likelihood (Direct), forward likelihood (Forward), and the backward reconstruction penalty (Backward):

Scoring Variant Forward (F) Backward (B) Direct (D) MATH500 (Acc.) GPQA (Acc.) MuSR-ta (Acc.) 7-Task Avg.
Baseline (Random Selection) ✗ ✗ ✗ 42.4 23.7 64.5 43.4
Direct Marginal Likelihood ✗ ✗ ✓ 50.3 32.6 68.4 49.2
Forward Confidence Alone ✓ ✗ ✗ 50.1 32.3 76.9 51.1
Maximize Raw Backward ✗ ✓ (Maximize) ✗ 41.0 22.4 65.1 43.0
Minimize Raw Backward ✗ ✓ (Minimize) ✗ 44.3 25.9 61.5 41.8
ReFeri (Full Exemplar Set) ✓ ✓ (Full Set) ✗ 50.1 32.7 76.8 51.0
ReFeri (Nearest-Neighbor LOO) ✓ ✓ (NN-LOO) ✗ 50.2 33.5 76.5 51.3

Key Findings

  • Complementarity Prevents Task-Specific Degradation: Direct marginal scoring performs adequately on familiar mathematical patterns (MATH500 50.3) but collapses on complex relational reasoning (MuSR-ta 68.4). Conversely, forward confidence excels on MuSR-ta (76.9) but stagnates on GPQA (32.3). ReFeri consistently resolves this dilemma, never dropping below either single-term variant across all 21 generator-task combinations and reaching a peak average of 51.3.
  • Robustness Under Artificial Exemplar Injection: When prepending \(\rho\%\) of the retrieved demonstration answer into low-scoring incorrect candidates, forward-only selection accuracy drops precipitously from 45.29% (\(\rho=0\)) down to 33.31% (\(\rho=100\%\)). In stark contrast, ReFeri registers the sharp spike in backward reconstruction likelihood and increases its penalty, elevating selection accuracy to 47.48% and successfully suppressing superficial prompt imitation.
  • Ultra-Lightweight Verification Frontier: Scaling the estimator down to LLaMA-3.2-1B causes competing methods (CoT-WP, USC) to collapse, whereas ReFeri1B retains virtually identical performance to the 8B estimator (71.3 vs 71.5 on GPT-4o) while running 2.6× faster than 8B CoT-WP on an NVIDIA RTX A6000.
  • Rank-Weighted Synergy with Majority Voting: Integrating ReFeri rank scores as Borda weights (\(p=1\)) into conventional discrete majority voting lifts Self-Consistency performance from 70.7 to 71.3 on GPT-4o, from 64.2 to 64.9 on GPT-4o-mini, and from 50.1 to 52.4 on LLaMA-3.1-8B.

Highlights & Insights

  • Repurposing Demonstrations from Static Prompts to Dynamic Verifiers: The paper overturns the orthodox assumption that in-context exemplars are purely input-side prompts, systematically establishing their role as bidirectional Bayesian verification constraints.
  • Leave-One-Out Substitution as Tractable Backward Scoring: Measuring backward reconstruction in autoregressive LLMs without task-specific retraining is achieved through an elegant prompt-replacement strategy, retaining background context while isolating marginal reconstruction gains.
  • Decoupled Evaluator Invariance: Demonstrates that black-box commercial LLM outputs can be reliably verified using open-source, ultra-small estimators (1B), establishing a highly practical paradigm for low-cost inference-time verification.

Limitations & Future Work

  • Strict Requirement for Seed Demonstrations: ReFeri fundamentally relies on the availability of few-shot exemplars; while it remains resilient against intentionally degraded or synthetic exemplars, it cannot operate in completely cold-start zero-shot domains lacking any exemplar priors.
  • Global Likelihood Uninterpretability: Unlike LLM-as-a-Judge or PRMs, which supply natural-language rationale or step-level diagnostic credit, ReFeri outputs a scalar score difference that cannot localize the specific step where reasoning derailed. Future extensions could decompose token-level backward penalties into fine-grained reasoning error locators.
  • vs Universal Self-Consistency (USC) / LLM-as-Judge: USC concatenates multiple candidate responses into a single judge prompt, suffering severely from positional biases (disproportionately selecting early candidates) and context-length saturation; ReFeri decouples candidates into independent likelihood evaluations, completely eliminating presentation order artifacts.
  • vs CoT-WP & Self-Certainty: Token-probability margin and predictive entropy rely strictly on ungrounded internal model confidence, which collapses when evaluating uniformly confident outputs from advanced base models; ReFeri anchors the evaluation in explicit external exemplars, preserving discrimination under severe generator-evaluator calibration drift.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Elegant Bayesian log-likelihood decomposition recycling few-shot examples into forward guidance and backward anti-overfitting penalties.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous coverage spanning seven multi-step reasoning benchmarks, two personalization tasks, one execution-based code benchmark, and extensive perturbation analyses.
  • Writing Quality: ⭐⭐⭐⭐⭐ Clear mathematical formulation, logically seamless narrative flow, and high-density empirical figures.
  • Value: ⭐⭐⭐⭐⭐ Delivers an immediately deployable, training-free test-time compute scaling mechanism with a superior cost-accuracy Pareto frontier.