Skip to content

iPOE: Interpretable Prompt Optimization via Explanations

Conference: EMNLP 2026
Paper: https://aclanthology.org/2026.emnlp-main.1631/
Code: https://www.uni-bamberg.de/en/nlproc/projects/inprompt/
Area: LLM Reasoning
Keywords: Prompt Optimization, Explainability, Annotation Guidelines, Explanation-Guided, Discrete Search

TL;DR

This paper proposes iPOE, an interpretable prompt optimization approach that derives modular, rule-like annotation guidelines from decision explanations and optimizes them via discrete editing operators, delivering up to a 39 percentage point gain over baseline prompts while maintaining high human-LLM alignment in decision rationale.

Background & Motivation

The performance of large language models (LLMs) on complex classification and reasoning tasks is notoriously sensitive to prompt engineering. Because of prompt brittleness, superficial wording variations or subtle token shifts can trigger significant output degradation. To alleviate the heavy burden of manual prompt engineering, existing automated prompt optimization frameworks—ranging from token-level search algorithms (e.g., GrIPS) to meta-prompting optimizers (e.g., OPRO)—typically frame prompt discovery as a discrete black-box search problem. However, these methods produce opaque prompt variants that obscure why or where specific phrasing adjustments lead to performance gains, making it nearly impossible for domain experts and laypeople to inspect, verify, or inject domain constraints into the optimized instructions.

In contrast, human annotation workflows achieve consistency and rigor through meticulously formulated annotation guidelines. Annotation researchers continuously refine these guidelines across iterative pilot rounds by examining inter-annotator disagreements and establishing explicit decision boundaries, edge-case criteria, and disambiguation rules. Nonetheless, directly prepending existing human-written annotation guidelines to LLM prompts often yields disappointing results: LLMs are not human annotators, and their internal representational spaces do not spontaneously adhere to abstract sociological or conceptual guidelines written for human consumption, frequently leading to instruction-following inconsistencies and distraction from tangential narrative details.

To bridge this fundamental gap, this paper unifies the iterative refinement principle of human annotation guidelines with automated discrete prompt optimization, proposing an explanation-guided guideline generation and evolution paradigm. The key insight is that post-hoc decision explanations (whether provided by human annotators or generated post-hoc by LLMs, spanning natural language justifications to feature attributions) encapsulate concrete discriminative rules that can be systematized into an actionable knowledge pool. Core idea: Automatically convert decision explanations into a modular pool of rule-based guidelines, and iteratively search for an optimal, human-interpretable guideline prompt using discrete editing operators—including adding, removing, replacing, merging, and shuffling.

Method

Overall Architecture

The iPOE framework operates through three sequential phases: (1) Rule-based Guidelines Pool Generation; (2) Iterative Optimization with Guideline Operators; and (3) Guideline-Augmented Zero-Shot Inference.

In the initial phase, iPOE constructs a task-specific guideline pool \(\mathcal{R}\) from training instances and gold labels accompanied by explanations (either human-authored or LLM-generated). A meta-prompt instructs an LLM to convert instance-level explanations into concise, generalizable rule-like guidelines. During the iterative optimization phase, starting from an empty guideline set \(\mathcal{G}=\emptyset\) and a fixed task instruction prefix \(P\), the framework iteratively samples a subset of candidate guidelines \(g \subset \mathcal{R}\) and evaluates candidate guideline sets produced by a bank of discrete operators \(\mathcal{O}\). The candidate set \(\mathcal{G}'\) achieving the highest Macro-F1 score on a training validation split is greedily accepted if \(F(P \oplus \mathcal{G}') > F(P \oplus \mathcal{G})\). The loop continues until the iteration budget \(I\) is exhausted, outputting the final interpretable prompt \(P \oplus \mathcal{G}\).

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Training Data & Labels<br/>(with Explanations / Attributions)"] --> B["Rule-based Guidelines Pool Generation<br/>Convert instance explanations to rule pool R"]
    B --> C["Candidate Guideline Sampling<br/>Random or Label-controlled sampling"]
    subgraph OPT["Discrete Guideline Optimization"]
        direction TB
        C --> D["Guideline Editing Operators<br/>Add / Remove / Replace / Merge / Shuffle"]
        D --> E["Evaluation & Greedy Acceptance<br/>Update G if F(P ⊕ G') > F(P ⊕ G)"]
    end
    E -->|Iterations < Max Budget I| C
    E -->|Budget Reached| F["Final Interpretable Prompt<br/>Prefix P + Optimized Guideline Set G"]
    F --> G["Zero-Shot Task Inference<br/>Structured JSON Prediction"]

Key Designs

1. Rule-based Guidelines Pool Generation: Transforming instance explanations into generalizable decision criteria

Rather than asking an LLM to hallucinate instructions from scratch or relying on raw few-shot examples, iPOE leverages granular decision explanations to form the foundation of its search space. For each training instance, the framework takes the text, gold label, and corresponding explanation (either human-written rationales or feature attribution highlights) and prompts the LLM via a structured meta-prompt to abstract a general rule for the task. For feature attributions (e.g., lexical tokens flagging hate speech), the model abstracts the pragmatic nuance, derogatory connotations, or contextual exceptions associated with those terms; for natural language explanations, it isolates the underlying causal link. This generates an extensive, diverse database of discrete guideline primitives \(\mathcal{R}\).

2. Guideline Editing Operators: Navigating the structured prompt space with semantic-preserving transformations

Instead of applying token-level lexical perturbations (which often produce ungrammatical or bizarre text), iPOE defines five semantic-preserving guideline-level operations \(\mathcal{O}\): - Add: Appends the sampled candidate guidelines to the end of current guideline set \(\mathcal{G}\), expanding decision coverage. - Remove: Deletes \(n\) guidelines at random (or one per sampled label under label control) to prevent context bloat and discard ineffective rules. - Replace: Substitutes removed guidelines with freshly sampled ones, exploring alternative formulations while preserving prompt compactness. - Merge: Invokes an LLM to rewrite and synthesize multiple guidelines that address the same target label into a unified, coherent, and non-redundant guideline entry. This operator proves to be the most frequently accepted operation across all tasks. - Shuffle: Permutes the ordering of guidelines within \(\mathcal{G}\) to mitigate position bias in LLM attention.

In each optimization step, the algorithm evaluates all proposed candidate sets \(o(\mathcal{G}, g)\) for \(o \in \mathcal{O}\) on a validation slice, updating the master guideline set \(\mathcal{G} \leftarrow \mathcal{G}'\) only when a strict metric improvement is verified.

3. Label-Controlled Sampling Strategy: Enforcing balanced class coverage

To evaluate the impact of label distribution on prompt evolution, iPOE introduces two distinct candidate sampling regimes: unconstrained sampling (iPOE) and label-controlled sampling (iPOE-lb). Unconstrained sampling draws guidelines uniformly at random from the pool \(\mathcal{R}\) without regard to label distribution. In contrast, label-controlled sampling enforces stratified uniform sampling across all gold labels, ensuring that every class maintains explicit representational guidance within the prompt. Under the Remove operator, label control enforces that one guideline is pruned per active class, preventing the search trajectory from degenerating into a majority-class bias.

Loss & Training

iPOE operates as a gradient-free black-box discrete optimization pipeline, leaving underlying model weights untouched. The objective function maximizes the Macro-F1 score on a representative subsample of the training split:

\[\mathcal{G}^* = \arg\max_{o \in \mathcal{O}, g \subset \mathcal{R}} F_{\text{eval}}(P \oplus o(\mathcal{G}, g))\]

To balance convergence speed with computational constraints, each iteration evaluates candidates on fixed training subsets (subsample ratio: 0.2 for crowd-enVent, 1.0 for HealthFC, 0.1 for HealthVer, and 0.1 for HateXplain). The search runs for \(I=300\) iterations per round across 5 independent evaluation rounds. Candidate sampling draws 3 guidelines per step. Target task predictions are executed using deterministic greedy decoding to avoid sampling variance, with outputs parsed as structured JSON objects. Conversely, guideline generation and merging utilize stochastic sampling (\(T=0.6, \text{top\_p}=0.9, \text{top\_k}=50\)) to encourage creative semantic synthesis.

Key Experimental Results

Main Results

The framework was evaluated across four NLP benchmarks spanning three distinct domains: emotion classification (crowd-enVent), health fact-checking (HealthFC, HealthVer), and hate speech detection (HateXplain). Three locally deployed LLMs were examined: Qwen3-4B-Instruct, Llama-3.1-8B-Instruct, and Qwen3-30B-A3B-Instruct. Baselines included Vanilla (prefix only), Random (randomly sampled guidelines matching class count), Human-written guidelines, Chain-of-Thought (CoT), Instruction Rewriting (IR), and OPRO. Performance is reported in Macro-F1.

Dataset Model Vanilla Human CoT IR OPRO iPOE-h iPOE-lb-h iPOE-llm iPOE-lb-llm
crowd-enVent Qwen3-4B 0.35 0.35 0.34 0.32 0.35 0.50 0.49 0.50 0.50
Llama3-8B 0.34 0.30 0.35 0.37 0.38 0.48 0.51 0.49 0.48
Qwen3-30B 0.40 0.36 0.40 0.39 0.41 0.51 0.51 0.54 0.50
HealthFC Qwen3-4B 0.60 0.62 0.62 0.67 0.69 0.84 0.83 0.80 0.84
Llama3-8B 0.48 0.65 0.52 0.75 0.60 0.84 0.76 0.79 0.78
Qwen3-30B 0.71 0.73 0.69 0.63 0.67 0.80 0.77 0.79 0.79
HealthVer Qwen3-4B 0.40 0.60 0.43 0.58 0.46 0.62 0.63 0.64 0.62
Llama3-8B 0.41 0.61 0.44 0.52 0.40 0.60 0.60 0.63 0.63
Qwen3-30B 0.63 0.63 0.64 0.65 0.58 0.64 0.64 0.68 0.67
HateXplain Qwen3-4B 0.44 0.45 0.44 0.47 0.50 0.53 0.53 0.54 (Na) 0.52
Llama3-8B 0.36 0.28 0.36 0.32 0.44 0.55 0.53 0.54 (Fe) 0.54
Qwen3-30B 0.45 0.40 0.42 0.38 0.44 0.57 0.56 0.57 (Fe) 0.56

(Note: HealthVer lacks human explanations; iPOE-h transfers guidelines derived from HealthFC human annotations. For HateXplain, Fe denotes feature attributions and Na denotes LLM-generated natural language explanations)

Ablations & Interpretability Analysis

To understand why iPOE works, the authors analyzed operator dynamics and conducted an interpretability validation study comparing human judgment with SHAP attribution scores across the optimized guidelines:

Evaluation Dimension / Task Qwen3-4B Llama3-8B Qwen3-30B Key Analysis & Takeaways
Human-LLM Agreement (Cohen's κ) High inter-annotator agreement on guideline utility:
- crowd-enVent (Emotion) 0.63 0.68 0.56 Substantial agreement across models (\(\kappa = 0.56 - 0.68\)).
- HealthFC (Fact-Checking) 0.65 0.80 0.74 High agreement in scientific reasoning (\(\kappa\) up to 0.80).
- HateXplain (Hate Speech) 0.83 0.81 0.95 Near-perfect alignment (\(\kappa = 0.95\)), confirming transparency.
Operator Distribution & Dynamics Frequency Trajectory Role Behavioral Characteristics Merge operator accounts for the majority of successful edits.
- Merge Operator 45% - 55% Primary synthesizer Condenses overlapping rules into structured, nuanced paragraphs.
- Add / Remove / Replace ~15% each Dynamic balancing Regulates guideline quantity; stabilizes near the number of classes.
- Shuffle Operator < 10% Fine-tuning order Alleviates position bias at critical optimization plateaus.

Key Findings

  • LLM-generated explanations match or surpass human explanations: iPOE-llm achieved performance completely parity with iPOE-h (e.g., 0.54 vs. 0.51 on crowd-enVent with Qwen3-30B, and 0.68 vs. 0.64 on HealthVer), proving that costly human rationales can be entirely substituted with synthetic explanations generated by LLMs.
  • Robust out-of-distribution transferability: Applying guidelines derived from HealthFC (German clinical trial claims) to HealthVer (COVID-19 claims) matched the performance of guidelines generated directly from HealthVer's own training data (0.64 F1), validating the domain generalizability of abstracted rule-like guidelines.
  • Cognitive parallels between humans and LLMs: The human study revealed that while human annotators focus heavily on positive guidelines supporting the chosen label, LLMs leverage SHAP-attributed guidelines both to support the target class and to refute alternative classes. Despite this difference, their Cohen's kappa scores reach up to 0.95, confirming that iPOE yields human-aligned reasoning pathways.

Highlights & Insights

  • Elevating prompt optimization from token search to rule evolution: While prior techniques search over obscure tokens or broad instruction paraphrases, iPOE optimizes human-readable guidelines. This provides a transparent interface that allows laypeople and domain practitioners to directly inspect and calibrate the decision boundaries of LLMs.
  • The emergent power of the Merge operator: The merge operator emerged as the primary driver of optimization success, acting as an automated synthesis engine that combines scattered, sample-specific heuristics into cohesive, rule-book-style instructions complete with exceptions and illustrative examples.
  • Modality agnosticism across explanation types: The method seamlessly processes both structured natural language rationales and raw lexical feature attributions, making it readily applicable to a wide spectrum of explainable NLP benchmarks.

Limitations & Future Work

  • High computational expenditure: Evaluating candidates across 300 iterations per round requires substantial compute (estimated at ~9,500 GPU hours for the full experimental suite), highlighting the need for early-stopping heuristics or surrogate reward models.
  • Guideline redundancy in terminal prompts: While the merge operator compresses rules, some converged guideline sets still contain redundant or slightly overlapping criteria for the same label.
  • Incomplete ablation on sampling budgets: Due to compute bounds, the sensitivity of the optimization process to the number of sampled guidelines per iteration (fixed at 3) remains unexplored.
  • vs GrIPS & Instruction Rewriting: GrIPS relies on syntactic edit operations (delete, swap, paraphrase) that lack task-level semantic grounding; iPOE operates on modular guidelines that explicitly formalize task criteria, yielding superior performance (+24 pp average gain) and full interpretability.
  • vs OPRO: OPRO uses an LLM to propose new instructions based on history trajectories, treating the prompt as an opaque string; iPOE structures the prompt into modular rules that can be independently inspected, swapped, and merged, beating OPRO by 23 pp on average.
  • vs Human-Written Guidelines: Standard human guidelines are verbose and geared toward human cognitive models; iPOE generates guidelines that directly cater to LLM representational styles, outperforming human guidelines by 21-25 pp.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ [Pioneering integration of explanation mining, human annotation guideline concepts, and discrete prompt search]
  • Experimental Rigor: ⭐⭐⭐⭐⭐ [Extensive evaluation across 4 datasets, 3 modern LLM architectures, OOD transfer, and rigorous human interpretability studies]
  • Writing Quality: ⭐⭐⭐⭐⭐ [Extremely articulate, transparent analysis of operator dynamics, and comprehensive appendices]
  • Practical Impact: ⭐⭐⭐⭐⭐ [Bridges automated optimization with human-in-the-loop auditability, highly relevant for high-stakes deployment]