Skip to content

Identify, Classify, Act: Per-Instruction Controlled Compliance for LLMs over External Data

Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: External-Data Instruction Classification, Indirect Prompt Injection Defense, Controlled Compliance, Generative Judge, Reinforcement Learning

TL;DR

To break the fundamental safety-utility tradeoff where models either over-refuse or over-comply when parsing untrusted external inputs, this paper formalizes the External-Data Instruction Classification (EDIC) task, introduces a scene-grounded counterfactual data synthesis pipeline, and develops a generative judge that delivers decomposed rewards to guide CoT policy models via GRPO to selectively follow beneficial instructions while rejecting adversarial prompt injections.

Background & Motivation

As large language models (LLMs) transition from standalone conversational agents to autonomous reasoning cores within retrieval-augmented generation (RAG) pipelines, web-browsing assistants, multi-step enterprise workflows, and tool-using systems, they must routinely process and act on unvetted external documents. However, real-world external data routinely presents a complex, mixed-signal context: it interleaves legitimate, task-critical directives—such as payload validation rules in API documentation, clinical pathway guidelines in medical protocol summaries, or formatting requirements in business emails—with potentially untrusted textual inputs. When models interact with third-party web content, user communications, or shared databases, adversarial actors can embed indirect prompt injections engineered to hijack the system, exfiltrate sensitive user records, or manipulate execution pathways.

Confronted with this adversarial surface, existing safety alignment paradigms typically frame defense as a blunt, binary choice between total trust and blanket suppression. Approaches such as static instruction hierarchy prioritization, structured query delimiters (e.g., StruQ), and standalone prompt-injection classifiers (e.g., PromptGuard) enforce a "rejection-as-safety" stance, indiscriminately squashing or neutralizing all control directives originating from external data sources. While these defenses reduce prompt-injection attack success rates, they simultaneously discard beneficial, context-embedded constraints required for correct downstream task execution, collapsing the safety-utility frontier and triggering severe over-refusal. Conversely, models aligned solely for helpfulness fall prey to over-compliance, willingly executing harmful embedded instructions. This deadlock persists because current alignment frameworks lack the capability to perform fine-grained behavioral credit attribution over co-occurring instructions within the same document.

To overcome this impasse, the authors argue that the missing capability is not more rigid filtering, but granular per-instruction attribution that empowers the model to adjudicate and act upon embedded directives individually. The authors formalize this problem as External-Data Instruction Classification (EDIC), structuring it into an explicit identify–classify–act decision workflow. Core idea: decompose external data compliance into fine-grained, per-instruction behavioral credit attribution, utilizing a counterfactual scene-grounded synthesis pipeline and a generative judge to supply decoupled rewards that optimize a chain-of-thought policy model to follow beneficial instructions while resisting harmful injections.

Method

Overall Architecture

The EDIC framework operationalizes the identify-classify-act paradigm across three modular stages: (1) scene-grounded counterfactual data synthesis, which generates realistic document contexts with orthogonal combinations of beneficial directives, malicious injections, and contrastive behavioral branches; (2) two-stage generative judge model training, which equips a dedicated evaluator to attribute compliance locally to individual candidate instructions; and (3) judge-guided reinforcement learning of a chain-of-thought (CoT) policy model via Group Relative Policy Optimization (GRPO) using a decomposed per-instruction reward. Given a user query \(q\) and external document \(D\), the policy model is trained to extract embedded instruction snippets \(\mathcal{I}=\{e_1, \dots, e_K\}\), classify each snippet into its semantic category, and produce a response \(y\) that faithfully executes authorized guidelines while isolating and rejecting adversarial injections.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: User Query q & External Data D"] --> B["Scene-Grounded Multi-Branch Synthesis<br/>4 Real-World Scenes · Benign / Harmful / Mixed Variants"]
    B --> C["Strict Counterfactual Quality Filtering<br/>Necessity · Consistency · Naturalness · Independence Checks"]
    C --> D["Two-Stage Generative Judge Model<br/>SFT Cold-Start + GRPO Fine-Grained Attribution"]
    D --> E["Instruction-Decomposed RL Reward<br/>Beneficial Compliance Bonus + Harmful Penalty + Format Bonus"]
    E --> F["CoT Controlled-Compliance Policy Model<br/>Explicit Identify · Classify · Act Chain-of-Thought"]

Key Designs

1. Scene-Grounded Multi-Branch Data Synthesis: Orthogonal Behavioral Space Construction

Rather than relying on synthetic, isolated attack templates, EDIC grounds its data generation in real-world corpora across four deployed scene archetypes: workplace email (Spam Email), API technical documentation (API-Bank), clinical practice guidelines (EPFL clinical guidelines corpus), and multi-hop knowledge retrieval (HotpotQA and Wiki-MultihopQA). While preserving the authentic vocabulary, entity relations, and stylistic structure of the underlying text, the synthesis pipeline injects instructions to construct three controlled contextual variants for each base document \(D\): a beneficial-only variant \(D_{\text{ben}} = D \cup \{e^+\}\), a harmful-injection variant \(D_{\text{harm}} = D \cup \{e^-\}\), and a mixed-context variant \(D_{\text{mix}} = D \cup \{e^+, e^-\}\). Crucially, to provide contrastive evidence for credit attribution, each sample is synthesized with four divergent response branches: target compliant execution \(y_{\text{ben}}\) (satisfies user goal and \(e^+\)), over-conservative error \(y_{\text{non}}\) (completes user goal but drops all embedded constraints), injection-hijacked error \(y_{\text{pi}}\) (abandons user goal to execute \(e^-\)), and partial redirection error \(y_{\text{mix}}\) (executes both \(e^+\) and \(e^-\)). This multi-branch setup establishes an orthogonal supervision manifold that prevents shortcut learning.

2. Strict Counterfactual Quality Filtering: Four-Dimensional Causal Validation Gate

Synthetic alignment datasets often suffer from superficial correlations where embedded instructions have negligible impact on the ground truth output. To enforce rigorous causal coupling, EDIC implements a 4-dimensional counterfactual validation filter: - Beneficial-Instruction Necessity Check: For each candidate sample, four ablated variants are generated (exact removal, background sentence replacement, non-binding suggestion replacement, and non-binding neighbor replacement). An LLM evaluator inspects whether weakening \(e^+\) materially changes the authorized output artifact (requiring a strict maximum necessity score of 5); - Task Consistency Check: Evaluates whether the allow-path strictly executes the authorized task while the forbid-path strictly remains informational (requiring a score of 5); - Scene Naturalness Check: Ensures that external contexts and embedded directives read like authentic production artifacts rather than synthetic prompts (requiring a score of 5); - Harmful-Instruction Independence Check: Verifies that removing the injection \(e^-\) does not alter the authorized response, enforcing that harmful directives cause zero contamination (requiring an independence score \(\le 2\)). Exactly 900 verified samples per scene family are preserved after filtering.

3. Two-Stage Generative Judge Model: Local Behavioral Attribution Specialist

When multiple directives co-occur within a single document, standard scalar outcome reward models cannot distinguish which specific instruction influenced the generation. EDIC designs a generative judge \(J_\phi(q, D, e_k, y) \to (r, \hat{\ell})\) that evaluates compliance with respect to a single candidate instruction \(e_k\), outputting an explicit reasoning trace \(r\) followed by a binary decision \(\hat{\ell} \in \{0, 1\}\). The judge is trained in two phases: first, an SFT cold-start on judge corpus \(\mathcal{D}_J\) teaches standardized evidence-grounding reasoning and output formatting; second, the judge is refined via GRPO using a composite objective: $\(R_J = r_{\text{correct}} + r_{\text{fmt}} = \mathbb{I}[\hat{\ell} = \ell] + r_{\text{fmt}}\)$ where \(r_{\text{correct}}\) rewards attribution correctness and \(r_{\text{fmt}}\) penalizes format violations. This RL stage sharpens attribution accuracy on ambiguous, boundary-pushing compliance cases, driving the judge-policy agreement (Cohen's \(\kappa\)) from a baseline of 0.38 up to 0.81.

4. Instruction-Decomposed RL Reward: Decoupled Optimization under Adversarial Tension

To optimize the CoT response policy \(\pi_\theta\), EDIC formulates a decomposed per-instruction reward function that isolates beneficial adherence from malicious refusal: $\(R(y) = \sum_{e_k: a^*_k = \text{fol}} J_\phi(q, D, e_k, y) - \lambda \sum_{e_k: a^*_k \neq \text{fol}} J_\phi(q, D, e_k, y) + \alpha S(y)\)$ The first term credits compliance with ground-truth beneficial instructions, the second penalizes execution of harmful injections scaled by \(\lambda\), and \(S(y)\) awards format/structure validity scaled by \(\alpha\). The policy is trained on policy dataset \(\mathcal{D}_P\) using GRPO with KL regularization: $\(\max_\theta \mathbb{E}_{(q,D)\sim \mathcal{D}_P} \left[ \mathbb{E}_{y\sim \pi_\theta} [R(y; q, D, \mathcal{I})] - \beta \mathbb{D}_{\text{KL}}[\pi_\theta \parallel \pi_{\text{ref}}] \right]\)$ Because \(\mathcal{D}_P\) presents beneficial-only, harmful-only, and mixed cases within identical task domains, the policy cannot hack the reward through degenerate behaviors like blanket refusal (which craters the first term) or indiscriminate obedience (which heavily triggers the second term), forcing the model to master genuine per-instruction classification.

Loss & Training

  • Judge Training: Cold-start SFT runs with learning rate \(5 \times 10^{-5}\) (cosine decay, 3% warmup), per-device batch size 1 (gradient accumulation 8), LoRA rank 32, alpha 64. GRPO refinement uses learning rate \(1 \times 10^{-6}\), rollout group size \(n=8\), KL coefficient \(10^{-3}\), and prompt/response limits of 1,536 / 256 tokens.
  • Policy Training: Cold-start SFT uses learning rate \(2 \times 10^{-5}\) for Qwen3-4B and \(1 \times 10^{-5}\) for Llama-3.1-8B-Instruct (LoRA rank 32, alpha 32). Judge-guided GRPO uses learning rate \(1 \times 10^{-6}\), rollout group size \(n=8\), prompt/response caps of 2,048 / 768 tokens, and KL penalty \(\beta=10^{-3}\). Reward scaling hyper-parameters are set to \(\lambda=1.0, \alpha=0.5\) for Qwen3-4B, and \(\lambda=1.2, \alpha=0.5\) for Llama-3.1-8B. All experiments run on NVIDIA A100-80GB GPUs with VERL and vLLM 0.6.

Key Experimental Results

Main Results

The evaluation spans two model families (Qwen3-4B and Llama-3.1-8B-Instruct) across general utility benchmarks (Arena-Hard-Auto win rate, MT-Bench, AgentDojo utility DJ-U), external prompt injection safety benchmarks (AgentDojo ASR, CyberSecEval 2 ASR, InjecAgent ASR), and internal instruction compliance metrics (Harmful-Reject rate H-R, Benign-Follow rate B-F).

Model & Method Arena-Hard↑ MT-Bench↑ DJ-U↑ DJ-ASR↓ Cyb-ASR↓ Inj-ASR↓ H-R↑ (%) B-F↑ (%)
Qwen3-4B
Base 8.4 6.64 42.1 4.8 54.3 1.9 67.2 72.5
SecAlign 10.2 6.59 43.5 3.9 29.6 0.7 81.8 56.9
StruQ - - 40.2 4.1 37.4 1.2 76.5 53.3
ReasAlign 16.4 7.28 44.2 1.2 19.6 0.3 83.5 77.6
EDIC (Ours) 28.9 8.04 46.8 0.4 15.8 0.3 91.2 89.5
Llama-3.1-8B-Instruct
Base 1.6 6.38 6.6 0.8 28.6 14.7 74.9 85.5
SecAlign 3.2 6.41 6.4 0.5 14.2 2.2 88.6 67.5
StruQ - - 4.5 0.2 18.5 7.9 85.4 62.4
Meta-SecAlign 4.6 6.68 6.2 0.0 7.4 1.5 91.9 68.1
ReasAlign 7.8 6.95 7.4 0.1 3.8 1.1 93.1 87.3
EDIC (Ours) 12.4 7.33 15.8 0.1 2.5 0.7 92.8 91.7

Ablation Study

1. Data Composition & Context Co-Occurrence Ablation (Qwen3-4B, w/ Trigger)

Training Data Configuration H-R↑ (%) B-F↑ (%) CC↑ (Controlled Compliance) DJ-U↑ (%) DJ-ASR↓ (%)
Base Model 67.2 72.5 0.72 42.1 4.8
Harmful-only Training 92.8 74.8 0.75 40.2 0.5
Benign-only Training 65.8 94.6 0.71 44.4 4.9
Separate Contexts (no co-occurrence) 82.6 84.2 0.85 47.1 0.4
Mixed Contexts (Ours) 91.2 89.5 0.92 46.8 0.4

2. Beneficial Instruction Necessity Threshold Ablation (Resampled to 3,600 Examples)

Filter Threshold H-R↑ (%) B-F↑ (%) CC↑ DJ-U↑ (%) DJ-ASR↓ (%) Retained Pool (%)
No Filter 77.3 80.5 0.76 43.3 1.1 100.0%
\(\ge 4\) (Intermediate Threshold) 82.8 83.0 0.82 44.2 0.6 86.9%
\(= 5\) (Strict Threshold, Ours) 91.2 89.5 0.92 46.8 0.4 71.6%

3. Over-Refusal Robustness on NotInject (Benign Accuracy under 1–3 Trigger Words)

Model & Method Ben-Acc↑ (Overall) 1 Trigger (1T) 2 Triggers (2T) 3 Triggers (3T)
Qwen3-4B
Base 93.2 96.5 93.8 89.4
SecAlign 72.8 81.4 71.7 65.5
ReasAlign 78.6 85.8 77.9 72.2
EDIC (Ours) 88.8 93.8 88.5 84.1
Llama-3.1-8B-Instruct
Base 91.4 95.6 91.2 87.6
SecAlign 69.4 78.8 68.1 61.2
ReasAlign 75.2 83.2 74.3 68.1
EDIC (Ours) 87.4 92.0 86.7 83.5

Key Findings

  • Mixed contexts are essential for fine-grained discrimination: Training exclusively on harmful directives yields high rejection but severely degrades benign compliance (B-F drops to 74.8%), while benign-only training collapses safety. Crucially, training on both instructions in separate documents still plateaus controlled compliance at 0.85; only mixed-document training forces the model to perform per-instruction discrimination within a shared context (CC reaching 0.92).
  • Decomposed reward components prevent reward hacking: Removing the judge signal triggers the steepest safety drop; removing the structure preservation term collapses utility; removing the benign reward drives over-refusal; and removing the harmful penalty induces over-compliance. The four distinct terms act as complementary guardrails that eliminate common RL degenerate shortcuts.
  • Intent-based reasoning avoids lexical over-refusal: On the NotInject benchmark, baseline defenses suffer steep degradation as trigger keyword count increases (SecAlign falling to 61.2%–65.5%), indicating high vulnerability to surface lexical cues. EDIC remains resilient (retaining 84.1% and 83.5% accuracy under 3 trigger words), verifying that its safety decisions depend on semantic instruction intent rather than keyword pattern matching.

Highlights & Insights

  • Formulating defense as fine-grained credit attribution rather than coarse filtering: EDIC shifts prompt injection defense from an adversarial perimeter problem into a structured multi-step reasoning task, showing that models can internalize selective compliance via CoT.
  • Counterfactual necessity testing guarantees causal supervision: Filtering training pairs based on whether removing a directive measurably alters the execution output ensures high-fidelity causal coupling between directives and actions, providing a general blueprint for instruction dataset curation.
  • Trigger-controlled latent capability switching: The classification trigger serves as an inference-time switch that allows deployment systems to toggle between strict high-assurance security and permissive task compliance without requiring model retraining.

Limitations & Future Work

  • Domain coverage limited to four scene families: The training distribution focuses on email, API docs, medical guidelines, and multi-hop QA; it has not yet been validated on multi-turn autonomous web-agent trajectories, raw HTML DOM payloads, or low-resource non-English documents.
  • Vulnerability to adaptive, judge-targeted attacks: Because the data synthesis relies on fixed catalogs of injection templates from existing benchmarks, adaptive adversaries could theoretically engineer attacks tailored to evade the specific reasoning format of the generative judge.
  • Binary attribution collapses composite ambiguous instructions: The judge outputs a hard binary label (\(\hat{\ell} \in \{0, 1\}\)); instructions containing interwoven benign clauses and unauthorized data leaks cannot be partially credited, highlighting the need for calibrated judges with abstention or rewriting mechanisms.
  • vs StruQ (Chen et al., 2025a): StruQ uses syntactic boundary delimiters to wall off external data, preventing instruction leakage at the cost of blocking useful directives; EDIC allows models to safely digest operational constraints embedded in external texts without sacrificing safety.
  • vs Instruction Hierarchy (Wallace et al., 2024): Instruction Hierarchy enforces static priority ordering (System > User > Tool/External), which inevitably suppresses legitimate guidelines provided by tools; EDIC evaluates directives on their semantic merits regardless of source layer.
  • vs ReasAlign (Hao et al., 2026): ReasAlign employs reasoning-guided safety alignment but treats all external instructions as hostile entities; EDIC is the first framework to model co-occurring beneficial and harmful directives through a decomposed per-instruction reward.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Introduces the EDIC formulation and decomposed per-instruction reward, moving beyond binary blanket filtering.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluation across two backbones, five external benchmarks, four vertical scenes, and detailed causal ablations.
  • Writing Quality: ⭐⭐⭐⭐⭐ Rigorous mathematical formalization, cohesive narrative structure, and clear experimental presentation.
  • Value: ⭐⭐⭐⭐⭐ Delivers an actionable, high-utility blueprint for resolving prompt injection vulnerabilities in production RAG and agent architectures.