COPILOT: Controlling Fine-grained Provider Preference in LLM Recommendations¶
Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/shiningrain/ProviderPreferenceControl
Area: LLM Reasoning
Keywords: Recommender Systems, Provider Preference Control, Diffusion Language Models, Constrained Decoding, Training-Free Framework
TL;DR¶
Addressing the severe cross-intent contamination and distractor clutter of LLM recommenders under complex requests, COPILOT establishes a training-free "Plan-Draft-Complete" pipeline that decomposes sub-tasks, injects preferred services as hard anchors into a diffusion language model's initial mask to generate structural drafts in parallel, and refines them via the target LLM, boosting preference adherence by up to 58.9% while cutting irrelevant mentions by up to 87.8%.
Background & Motivation¶
Large language models (LLMs) are progressively evolving from passive information assistants into autonomous recommendation engines (LLM as RS) capable of synthesizing technical workflows, online service selections, and executable programming solutions. However, prevailing alignment paradigms—predominantly grounded in Reinforcement Learning from Human Feedback (RLHF), representation steering, or prompt engineering—have overwhelmingly targeted broad, coarse-grained behavioral virtues such as safety, harmlessness, or stylistic tone. They conspicuously overlook fine-grained provider preferences, which characterize real-world user or organizational demands for specific commercial brands, technology platforms, or cloud ecosystems (such as AWS vs. Azure vs. Google Cloud, or specialized analytics APIs). When models fail to respect these vendor choices, users are forced into tedious prompt-repair cycles, while enterprises suffer from code incompatibility with existing technology stacks, deployment policies, and licensing constraints.
Controlling fine-grained provider preferences is hindered by two fundamental structural tensions. First, real-world user configurations typically aggregate a comprehensive registry of dozens of preferred services across potential scenarios, yet any single user query typically articulates multi-intent requests that activate only a sparse, localized subset of those registered preferences. Standard prompt-based baselines (e.g., zero-shot, grouped, or step-wise prompting) lack on-demand activation precision; they frequently suffer from cross-intent confounding, either hallucinating preferred services into completely unrelated sub-tasks or mechanically dumping the entire registry into the output, producing catastrophic distractor contamination. Second, user preferences are inherently dynamic, shifting rapidly with budgetary allocations, vendor contracts, or architectural upgrades. Adapting models via parameter fine-tuning or activation steering vectors incurs prohibitive data curation and retraining costs that cannot support runtime updates, whereas multi-candidate sampling and re-ranking incur unacceptable latency and compute overheads in interactive production systems.
To resolve these challenges, this paper draws upon the classic cognitive process model of writing—which formalizes high-quality composition into iterative stages of planning, translating, and reviewing—and adapts it to LLM recommendations. The pivotal insight is that while autoregressive decoders inherently suffer from token-level left-to-right drift where soft prompt constraints are gradually diluted by language fluency pressures, non-autoregressive diffusion language models (DLMs) operate over bidirectional contexts and can treat arbitrary token positions as immutable boundary conditions. Core idea: COPILOT establishes a training-free framework that decomposes multi-intent prompts into scenario-matched sub-tasks, utilizes a diffusion language model to synthesize draft skeletons under hard-anchored service constraints in parallel, and leverages the target LLM to weave these drafts into a fluent, cohesive response, realizing strict preference adherence with minimal distractor leakage.
Method¶
Overall Architecture¶
COPILOT instantiates a training-free "Plan-Draft-Complete" architecture that coordinates an arbitrary target autoregressive LLM \(\theta_l\) (e.g., Qwen3-32B, Gemma-3-12B, or Gemini 3.1 Flash-Lite) with a lightweight, non-autoregressive diffusion language model \(\theta_d\) (specifically LLaDA-1.5). The system accepts a compound user prompt \(P\) and a pre-registered provider preference configuration \(C = \{(s_i, v_i)\}_{i=1}^{|C|}\), where each key \(s_i\) denotes a functional scenario (e.g., Cloud Hosting, Video Processing) and \(v_i\) specifies the designated preferred service entity (e.g., Microsoft Azure, Microsoft Azure Video Indexer). The objective is to produce an end-to-end response \(R\) where every sub-task matching an active scenario strictly executes via \(v_i\), unmatched sub-tasks remain unconstrained under \(\theta_l\)'s default implementation, and unrequested distractor providers in \(C\) are rigorously filtered out.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["User Prompt P + Preference Configuration C"] --> B["Scenario Planning: Sub-task Decomposition & Intent Matching<br/>Parses prompt into sub-tasks and tags active preferred services"]
B --> C["Draft Generation: Diffusion Masked Parallel Denoising<br/>Embeds fixed provider anchors in initial masks to draft skeletons"]
C --> D["Response Completion: Multi-Draft Stitching & Detail In-filling<br/>Target LLM fills mask slots and generates fluent code/text"]
D --> E["Final High-Adherence Recommendation Response R"]
Key Designs¶
1. Scenario Planning: Sub-task Decomposition & Intent Matching
Standard single-turn prompting fails when confronted with multi-intent inputs because it forces the model to perform joint semantic segmentation and constraint satisfaction within a single monolithic pass, leading to cross-scenario leakage. The scenario planning module decouples intent isolation from generation. Guided by a structured JSON schema, target model \(\theta_l\) decomposes the compound prompt \(P\) into an ordered sequence of granular sub-tasks \(T = \{t_1, t_2, \dots, t_k\}\). To prevent domain-specific ambiguities (e.g., distinguishing whether "hotel" refers to a booking action or a geographical landmark), COPILOT augments the planner with domain-specific system extensions: the code extension forces categorization along cloud and engineering abstractions (hosting, storage, speech, media), while the text extension evaluates contextual semantic roles. Sub-tasks matching an active entry in \(C\) are tagged with their required service \(v_i\) and expanded into concrete lexical anchors (service names and representative API identifiers, such as azure.mgmt.cdn). Crucially, sub-tasks lacking a corresponding preference in \(C\) are tagged as null (general tasks), explicitly forbidding the planner from hallucinating synthetic tasks to satisfy distractor entries.
2. Draft Generation: Diffusion Masked Parallel Denoising Autoregressive decoding generates tokens monotonically from left to right; under such dynamics, lexical constraints introduced early in a prompt can easily be overwritten or substituted by the model's dominant pre-training priors when generating deep into a lengthy code block. The draft generation module bypasses this limitation by exploiting the bidirectional masked-denoising capability of diffusion language models. For each preference-bearing sub-task, COPILOT constructs a masked token sequence \(X_0\) (128 tokens for code, 192 tokens for hybrid text/API configurations) and pre-populates discrete token positions with the extracted preference anchors separated by fixed token intervals (8-token gaps for code skeletons, 6-token gaps for text). These anchor tokens are frozen as immutable constants throughout the reverse diffusion process, compelling \(\theta_d\) to iteratively denoise surrounding mask tokens conditioned bidirectionally on the anchored services. Because these constraints are structurally embedded across multiple coordinates before generation begins, they provide robust anchors that cannot be bypassed. General sub-tasks lacking preference constraints bypass this module entirely, preserving generation efficiency and preventing spurious provider noise.
3. Response Completion: Multi-Draft Stitching & Detail In-filling
While the diffusion model provides hard structural guarantees for vendor service inclusion, lightweight DLMs lack the broad parametric reasoning required to handle intricate programming nuances, robust exception handling, and seamless multi-module dataflow integration. The response completion module acts as the "reviewing and refining" phase, tasking the target LLM \(\theta_l\) with unifying the synthesized components. COPILOT concatenates the sub-task drafts in sequential order, interleaving explicit \(\langle \text{mask} \rangle\) placeholder tokens between them to mark transition boundaries. A composite completion prompt is assembled containing the user's original query, the verified list of required services, the planned sub-task schema, and the concatenated draft scaffold. The target LLM is instructed to strictly preserve the established service APIs, flesh out missing logic and helper functions, eliminate draft artifacts, and complete unconstrained sub-tasks using neutral, open-source utilities (e.g., utilizing pyttsx3 and PyPDF2 for document voice synthesis). This hierarchical division of labor guarantees both exact lexical vendor adherence and high contextual fluency.
Benchmark & Evaluation Protocol¶
To comprehensively evaluate preference adherence against distractor interference, the authors formulate four complementary quantitative metrics: - Prompt-level Accuracy (PA): The proportion of queries where every applicable preference in the prompt is successfully satisfied (\(\text{PA} = \frac{1}{m} \sum_{i=1}^{m} \prod_{j=1}^{|P_i|} a_{ij}\)). - Instruction-level Accuracy (IA): The average fraction of individual applicable preferences satisfied across all sub-tasks (\(\text{IA} = \frac{1}{m} \sum_{i=1}^{m} \frac{1}{|P_i|} \sum_{j=1}^{|P_i|} a_{ij}\)). - Distractor Rate (DR): The frequency with which a response erroneously introduces services from distractor scenarios registered in \(C_i\) but unrequested by prompt \(P_i\) (\(\text{DR} = \frac{1}{m} \sum_{i=1}^{m} d_i\), where lower is strictly better). - Quality Score (QS): Method-blind holistic assessment (1–10 scale) rated by GPT-5.1 evaluating fluency, relevance, and utility, validated against human annotators (Spearman \(\rho = 0.831\)) and cross-checked via DeepSeek-v4-flash and Kimi-k2.6. Furthermore, in the code domain, the authors deploy an isolated sandbox with mocked cloud SDK interfaces (AWS, Azure, GCP) to track top-level imports, client instantiations, and method invocations, producing rigorous execution-level metrics (R-PA, R-IA, R-DR).
Key Experimental Results¶
Main Results¶
Experiments were conducted on 1,797 prompts across 15 coding scenarios and 5 everyday text recommendation scenarios, yielding 5,391 prompt-configuration test cases evaluated over three repeated generations (16,173 total responses). The table below details the performance of COPILOT against Zero-shot, Grouped, Step-wise, and Constrained decoding baselines across Qwen3-32B, Gemma-3-12B, and Gemini 3.1 Flash-Lite:
| Model | Method | Single-task Code PA↑ / DR↓ | Multi-task Code PA↑ / DR↓ | Single-task Text PA↑ / DR↓ | Multi-task Text PA↑ / DR↓ |
|---|---|---|---|---|---|
| Qwen3-32B | Zero-shot | 41.7% / 45.8% | 56.9% / 65.6% | 99.9% / 99.4% | 100.0% / 98.0% |
| Qwen3-32B | Grouped | 70.5% / 70.3% | 60.9% / 57.7% | 100.0% / 59.1% | 99.9% / 75.7% |
| Qwen3-32B | Step-wise | 18.9% / 17.6% | 15.5% / 16.9% | 100.0% / 92.9% | 100.0% / 90.0% |
| Qwen3-32B | Constrained | 69.9% / 74.3% | 56.7% / 65.5% | 100.0% / 97.1% | 100.0% / 96.7% |
| Qwen3-32B | COPILOT (Ours) | 78.5% / 15.5% | 80.3% / 19.0% | 98.8% / 7.2% | 98.6% / 39.8% |
| Gemma-3-12B | Grouped | 70.1% / 75.4% | 53.5% / 70.5% | 100.0% / 29.0% | 99.9% / 67.2% |
| Gemma-3-12B | COPILOT (Ours) | 80.0% / 21.7% | 85.0% / 18.3% | 95.4% / 25.9% | 98.1% / 42.6% |
| Gemini 3.1 | Grouped | 64.5% / 74.5% | 63.4% / 73.2% | 100.0% / 76.4% | 100.0% / 96.5% |
| Gemini 3.1 | Step-wise | 62.9% / 67.5% | 69.3% / 75.2% | 100.0% / 100.0% | 100.0% / 100.0% |
| Gemini 3.1 | COPILOT (Ours) | 95.8% / 12.5% | 95.8% / 25.0% | 100.0% / 18.1% | 100.0% / 26.4% |
In the mocked execution sandbox evaluating 300 multi-task code generations, COPILOT achieves an execution-level adherence R-PA of 46.3% and R-IA of 62.0% on Qwen3-32B, outperforming the best baseline (Grouped: 18.7% R-PA, 40.3% R-IA) by a factor of 2.4, while suppressing execution-level distractor calls R-DR to 18.3% compared to 41.0% for Grouped and 53.3% for Zero-shot.
Ablation Study¶
To dissect the contribution of each modular stage, systematic ablations were conducted on 100 randomly sampled code cases using Qwen3-32B:
| Configuration | PA (%)↑ | IA (%)↑ | DR (%)↓ | QS↑ | Note |
|---|---|---|---|---|---|
| COPILOT (Full Model) | 82.6 | 83.5 | 19.3 | 6.0 | Full three-stage pipeline |
| - w/o Scenario Planning | 50.7 | 63.2 | 71.0 | 5.3 | Replaces intent planning with punctuation-based heuristic segmentation |
| - w/o Draft Generation | 69.3 | 78.3 | 20.3 | 5.4 | Bypasses DLM drafts; sends planned sub-tasks directly to target LLM |
| - w/o Response Completion | 72.0 | 84.8 | 23.7 | 5.7 | Returns raw unrefined DLM draft directly to the user |
| - DLM drafts for all subtasks | 75.3 | 81.3 | 23.3 | - | Forces unconstrained sub-tasks through the DLM drafting module |
| - Lexically guided AR draft | 75.3 | 79.2 | 21.7 | - | Replaces diffusion drafts with autoregressive drafts under token biasing |
| - w/o scenario descriptions | 62.7 | 73.2 | 21.0 | - | Removes descriptive scenario context from the planning prompt |
Key Findings¶
- Scenario planning is the paramount barrier against distractor contamination: Ablating scenario planning (
- w/o Scenario Planning) results in severe degradation, causing PA to plummet from 82.6% to 50.7% while DR skyrockets from 19.3% to 71.0% on code (and jumps to 90.0% on text). Without explicit semantic grounding, models are incapable of bounding constraints in multi-intent contexts. - Diffusion models offer structural advantages over autoregressive drafting: Replacing the DLM with a lexically guided autoregressive draft model (
- Lexically guided AR draft) drops PA by 7.3% and IA by 4.3%. Even though AR drafts initially contain the provider names, their single left-to-right mention is frequently rewritten or smoothed away during completion; in contrast, DLM drafts bind multiple discrete anchor points across the skeleton, yielding an overall 85.6% anchor retention rate in the final executable code. - Negligible computational overhead in runtime practice: Across end-to-end code generation on Qwen3-32B, the median per-output latency of COPILOT is 224.4s, which sits squarely within the baseline span (168.4s to 237.1s). Scenario planning requires only 15.0s and parallel DLM drafting takes only 15.7s, while final completion dominates with 190.6s. Furthermore, on the Gemini API panel, COPILOT incurs an API charge of \(0.431 per 216 generations—lower than the prompting baselines (\)0.480 to $0.571)—because the compact draft scaffold prevents the target model from verbosely enumerating unnecessary distractor providers.
Highlights & Insights¶
- Inverting the utility of diffusion language models: Rather than treating non-autoregressive DLMs as stand-alone text generators, COPILOT ingeniously exploits their bidirectional masked-denoising property to construct hard-constrained skeleton scaffolds for autoregressive LLMs, overcoming the inherent architectural weakness of autoregressive left-to-right decoding in multi-point lexical steering.
- Unveiling the "superficial adherence" fallacy in text benchmarks: The study uncovers a critical evaluation blind spot in standard prompting methods: on text benchmarks, baselines superficially achieve ~100% PA solely by indiscriminate, verbatim enumeration of every registered provider in the global configuration (exhibiting DR > 95%), which severely degrades practical conversational utility.
- A plug-and-play governance paradigm for enterprise AI: The "Plan-Draft-Complete" decoupling enables organizations to declaratively enforce software procurement rules, approved API catalogs, and licensing boundaries through static configurations without requiring expensive fine-tuning or model weight modifications.
Limitations & Future Work¶
- Author-admitted limitations: The synthesized multi-intent benchmark concatenates at most three sub-tasks, which may not capture the full diversity, noise, and conversational length of real-world user logs; the preference configuration is assumed to be fully consistent and clean without conflicting or ambiguous overlap; and all experiments are conducted exclusively in English.
- Independent observations: The framework relies heavily on the open-source DLM (LLaDA-1.5) having basic prior knowledge of the target code domains; for highly specialized, proprietary enterprise APIs, the diffusion model may struggle to generate meaningful surrounding syntax around the anchors. Furthermore, 14.4% of injected anchors are still lost or rewritten during the completion phase, indicating the need for an explicit post-generation verification and repair loop.
- Promising future directions: Developing automated preference validation and conflict-resolution modules upstream of generation; exploring scaling properties with larger discrete diffusion backbones; and extending the evaluation to multilingual and multi-turn interactive conversational environments.
Related Work & Insights¶
- vs Prompt Engineering Baselines (Zero-shot / Grouped / Step-wise): Conventional prompting relies entirely on soft contextual attention, which suffers from severe attention distraction and cross-intent contamination in multi-task scenarios; COPILOT grounds constraints structurally into a code draft before fluency refinement, preventing constraint drift.
- vs Decoding-Time Alignment (NeuroLogic / Reward-Guided Search): Methods like NeuroLogic depend on beam search or dynamic logits manipulation at every autoregressive decoding step, which is computationally expensive, incompatible with greedy/sample-based APIs, and lacks global layout awareness; COPILOT achieves non-autoregressive parallel constraint satisfaction without touching the target model's logits.
- vs Training-based Alignment (RLHF / Activation Steering): Fine-tuning and steering vectors require extensive paired preference datasets and heavy gradient updates, rendering them incapable of adapting on the fly to rapid shifts in user vendor choices or project budgets; COPILOT is entirely training-free and adapts dynamically through declarative configuration.
Rating¶
- Novelty: ⭐⭐⭐⭐⭐ Pioneering use of diffusion language models as hard-constraint structural draft scaffolding for autoregressive LLM recommendation engines.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous evaluation across 3 model families, 2 domains, comprehensive metrics (PA/IA/DR/QS), human correlation validation, and sandbox mocked execution tracking.
- Writing Quality: ⭐⭐⭐⭐⭐ Cohesively grounded in cognitive writing theory, with crystal-clear motivation, thorough ablations, and insightful case analyses.
- Value: ⭐⭐⭐⭐⭐ Addresses a critical engineering and commercial bottleneck in productionizing LLM coding assistants and recommendation platforms without parameter retraining.