In-Place Feedback: Reliable Refinement for Multi-Turn Expert-LLM Collaboration¶
Conference: EMNLP 2026
Paper: ACL Anthology
Area: LLM Reasoning
Keywords: Interactive reasoning, Expert-in-the-loop refinement, Context pollution, Localized editing, Multi-turn reasoning
TL;DR¶
Addressing the severe failure of LLMs to reliably absorb multi-turn conversational corrections due to error accumulation in dialogue history, this work proposes In-Place Feedback—an interaction paradigm where experts directly edit erroneous spans in the model's prior response and prune dependent reasoning, driving continuation from a clean corrected state while dramatically reducing token consumption and expert fatigue.
Background & Motivation¶
Large language models are increasingly integrated into mission-critical expert workflows such as complex software engineering, legal drafting, and specialized clinical documentation. Because expert-level tasks entail intricate, multi-step logical reasoning, subtle factual hallucinations or flawed deduction steps generated by LLMs can cascade and disrupt entire operational pipelines. To mitigate this risk, real-world systems adopt an expert-in-the-loop paradigm where human professionals systematically review intermediate outputs and deliver fine-grained corrective feedback across multiple conversational rounds. However, the throughput and viability of this collaborative workflow are severely bottlenecked by the model's chronic inability to reliably absorb iterative corrections: as dialogue turns accumulate, models frequently ignore explicit critiques, corrupt previously correct passages, or introduce fresh downstream errors, trapping domain experts in a frustrating cycle of repeated re-reading and redundant correction.
The root cause of this multi-turn failure stems from context pollution inherent to the standard conversational format. In conventional multi-turn refinement, every flawed model response and subsequent user critique is appended chronologically to the ongoing prompt history. This mechanism introduces two major pathologies: first, erroneous reasoning patterns and conflicting feedback co-exist in the context window, where attention heads remain heavily anchored to stale, incorrect reasoning traces and fail to discern which hypotheses should be discarded; second, natural language feedback arrives as an external meta-instruction that the model must self-interpret, locate, and map onto specific output spans—a brittle requirement that easily breaks in complex multi-step settings. Furthermore, standard conversational feedback relies on full regeneration at every single turn, which not only causes token usage to explode linearly with conversation length, but also imposes a punishing cognitive re-verification tax on experts who must re-audit the entire response from scratch.
To resolve these structural bottlenecks, this work reframes feedback not as an additional conversational turn appended to history, but as an in-situ repair to the model's active cognitive state. The core idea is to abandon appending conversational instructions to dialogue history in favor of In-Place Feedback, empowering the expert to directly edit the erroneous span within the model's prior output, prune all downstream dependent reasoning, and prompt the LLM to resume autoregressive generation strictly from this purified prefix.
Method¶
Overall Architecture¶
In-Place Feedback reorganizes the collaborative refinement loop into a streamlined two-stage pipeline: in-place editing and continuation generation. Given an initial reasoning problem \(x\) and a model-generated trajectory \(y_t\) containing logical or factual defects, the system circumvents conversational tree expansion. Instead, the expert pinpoints the critical erroneous span and replaces it directly within the text with the valid deduction. Downstream clauses and subsequent steps that logically depend on the flawed premise are pruned away, while upstream valid derivations are held strictly immutable. The model then executes continuation generation, using the repaired and truncated prefix as conditioning context to autoregressively synthesize the remainder of the solution. This process maintains a clean, compact context, eliminates contradictory history, and strictly bounds generation costs.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Reasoning Query & Flawed Generation y_t"] --> B["Targeted Span In-Place Editing<br/>Localize error & repair state directly"]
B --> C["Dependent Reasoning Pruning<br/>Truncate contaminated downstream logic"]
C --> D["Conditioned Autoregressive Continuation<br/>Generate remainder from clean prefix"]
D --> E["Three-Dimensional Metric Diagnosis<br/>Decouple CPR / FAR / CTRR dynamics"]
E --> F["Hybrid Collaborative Workflow<br/>Balance structural chat & precision edits"]
Key Designs¶
1. Targeted Span In-Place Editing: Direct State Repair Eliminates Instruction Parsing Ambiguity To overcome the failure mode where models misinterpret natural language instructions or fail to ground feedback onto specific entities, this design materializes feedback directly as an in-situ string replacement within the model's own output. When a calculation slip, erroneous assumption, or incorrect variable assignment is detected at step \(i\), the expert directly updates the text span in place. Because the correction is seamlessly embedded within the model's native syntactic style, the LLM no longer needs to spend attention capacity parsing meta-instructions such as "change the number in line 3." This converts an ambiguous instruction-following challenge into a standard prefix-conditioned autoregressive continuation task.
2. Dependent Reasoning Pruning: Severing Stale Error Propagation and Attention Interference Applying an edit in isolation while leaving subsequent sentences intact is counterproductive, as the model's self-attention remains heavily biased by downstream text that was derived from the rejected premise. To eliminate this interference, the framework pairs localized editing with strict dependency pruning. In strictly sequential tasks like mathematical algebra, all tokens following the edited span are immediately pruned. In structured tasks with parallel constraint trees like ZebraLogic grid puzzles, dependency analysis isolates and deletes only the subset of reasoning steps that directly cite the modified attribute, preserving independent and valid constraint deductions. This surgical pruning removes poisoned context and ensures that future autoregressive steps build only on verified truths.
3. Conditioned Autoregressive Continuation: Prefix Reuse Slashes Token Costs and Human Verification Tax Addressing the severe inefficiency of full-sequence regeneration—which inflates token consumption and frequently overwrites segments that were already correct—this design freezes the validated, edited prefix and invokes standard text continuation to synthesize only the missing tail. Because the prefix remains identical to what the expert previously verified, the human collaborator is relieved from re-reading the entire draft and needs only to inspect the incremental continuation snippet. At the same time, generated token length drops by more than 50% across turns, drastically lowering inference latency and computational overhead.
4. Three-Dimensional Metric Diagnosis: Decoupling Multi-Turn Dynamics at the Cell Level Standard benchmark evaluations report only aggregate end-to-end task accuracy, concealing which failure modes actually undermine multi-turn interaction. To establish granular visibility into interactive behavior, this work introduces three orthogonal turn-level diagnostic metrics evaluated on the discrete constraint grids of ZebraLogic: - Correctness-Preserving Ratio (CPR): Measures the proportion of grid cells that were correct in response \(y_t\) and remain correct in \(y_{t+1}\), directly quantifying the model's resistance to corrupting valid prior work (Failure Mode i); - Feedback Acceptance Ratio (FAR): Measures the proportion of erroneous cells explicitly flagged by feedback \(f_t\) that are successfully resolved in \(y_{t+1}\), directly isolating instruction uptake (Failure Mode ii); - Correction Through Reasoning Ratio (CTRR): Measures the proportion of incorrect cells that were not mentioned in feedback \(f_t\) but are nevertheless independently corrected in \(y_{t+1}\) via multi-hop logical deduction, quantifying reasoning extension and suppression of new errors (Failure Mode iii).
5. Hybrid Collaborative Workflow: Synergizing Global Macro-Restructuring and Micro-Precision In-Place Edits Recognizing that real-world expert collaboration demands both flexible semantic direction and surgical local precision, this design formalizes a hybrid interaction paradigm. When an output exhibits holistic structural issues—such as an inappropriate rhetorical tone, poor high-level organization, or flawed narrative framing—experts efficiently leverage standard conversational prompting to trigger sweeping global revisions. Once the macro-structure stabilizes, experts switch to In-Place Feedback to execute rapid, high-confidence corrections on factual nuances, specific formulas, and numerical values. This workflow delivers the lowest cognitive friction and the highest overall user satisfaction.
Benchmark & Evaluation Protocol¶
The empirical evaluation spans five reasoning-intensive benchmarks: 1. MATH-hard: 500 Level-5 competition mathematics problems evaluated via exact final-answer match; 2. MMLU-pro: 493 free-form generation questions across diverse academic subjects; 3. GPQA: 126 graduate-level scientific reasoning questions from the rigorous GPQA-Diamond subset; 4. LiveCodeBench: Coding problems evaluated strictly against executable test suites in isolated sandboxes; 5. ZebraLogic: 573 complex grid puzzles requiring \(N \times M\) constraint satisfaction, with ground-truth verification and automated critique generated by an exact Z3 SMT solver. Underlying LLMs include Qwen2.5-7B-Instruct, Gemma-3-4b-it, Llama-3.1-8B-Instruct, Gemma-3-27b-it, and Llama-3.1-70B-Instruct, running with vLLM under greedy decoding (temperature 0.0, max length 2048). Baselines include standard Multi-turn feedback and Memory-based feedback retaining the \(k=3\) most recent feedback items.
Key Experimental Results¶
Main Results¶
Across all five datasets and all model scales, In-Place Feedback consistently and substantially outperforms standard Multi-turn and Memory-based feedback. Below are representative cumulative accuracy results across 10 interaction turns (final answer match on MATH-hard, MMLU-pro, and GPQA; full grid solution accuracy on ZebraLogic under top-4 feedback):
| Dataset | Model | Initial Acc (Turn 0) | Multi-turn Final (Turn 9) | Memory Final (Turn 9) | In-place Final (Turn 9) | Gain vs. Multi-turn |
|---|---|---|---|---|---|---|
| MATH-hard | Qwen-2.5-7B | 55.0% | 86.2% | 77.4% | 92.8% | +6.6% (surpasses MT final by Turn 5 at 88.8%) |
| MATH-hard | Gemma-3-4b | 49.2% | 82.8% | 78.0% | 87.6% | +4.8% |
| MATH-hard | Llama-3.1-8B | 14.8% | 75.6% | 69.8% | 84.0% | +8.4% |
| MMLU-pro | Qwen-2.5-7B | 56.2% | 80.5% | 80.9% | 85.4% | +4.9% (surpasses MT final by Turn 4 at 81.9%) |
| MMLU-pro | Gemma-3-4b | 41.4% | 73.4% | 72.2% | 81.3% | +7.9% (surpasses MT final by Turn 4 at 73.6%) |
| MMLU-pro | Llama-3.1-8B | 42.2% | 75.1% | 78.7% | 84.8% | +9.7% |
| GPQA | Qwen-2.5-7B | 10.3% | 46.0% | 38.9% | 69.0% | +23.0% (1.5x relative accuracy) |
| GPQA | Gemma-3-4b | 5.6% | 27.0% | 23.8% | 53.2% | +26.2% (nearly 2.0x relative accuracy) |
| GPQA | Llama-3.1-8B | 4.0% | 43.7% | 40.5% | 61.9% | +18.2% |
| ZebraLogic (Grid) | Qwen-2.5-7B | 0.4% | 69.0% | 66.5% | 92.2% | +23.2% |
| ZebraLogic (Grid) | Gemma-3-4b | 0.0% | 77.1% | 23.4% | 90.6% | +13.5% |
| ZebraLogic (Grid) | Llama-3.1-8B | 0.6% | 65.2% | 68.2% | 79.0% | +13.8% |
Ablation Study¶
1. Disentangling Localization and Continuation Generation (Table 1 Raw Data)
To isolate whether performance gains stem primarily from localized text editing or from conditioning strictly on a pruned, clean prefix, the authors evaluate Localized Multi-turn (Localized MT), which uses the same in-place editing agent but presents the edited response inside a new user turn while preserving the full dialogue history:
| Dataset | Model | Method | Turn 1 | Turn 3 | Turn 5 | Turn 7 | Turn 9 (Final) |
|---|---|---|---|---|---|---|---|
| MATH-hard | Qwen-2.5-7B | Multi-turn | 0.684 | 0.766 | 0.816 | 0.850 | 0.862 |
| MATH-hard | Qwen-2.5-7B | Localized MT | 0.682 | 0.790 | 0.830 | 0.848 | 0.858 |
| MATH-hard | Qwen-2.5-7B | In-place | 0.714 | 0.812 | 0.888 | 0.918 | 0.928 |
| MATH-hard | Gemma-3-4b | Multi-turn | 0.606 | 0.730 | 0.776 | 0.804 | 0.828 |
| MATH-hard | Gemma-3-4b | Localized MT | 0.612 | 0.750 | 0.794 | 0.824 | 0.846 |
| MATH-hard | Gemma-3-4b | In-place | 0.626 | 0.748 | 0.814 | 0.850 | 0.876 |
| GPQA | Qwen-2.5-7B | Multi-turn | 0.175 | 0.381 | 0.429 | 0.437 | 0.460 |
| GPQA | Qwen-2.5-7B | Localized MT | 0.238 | 0.381 | 0.484 | 0.564 | 0.611 |
| GPQA | Qwen-2.5-7B | In-place | 0.254 | 0.484 | 0.579 | 0.643 | 0.691 |
| GPQA | Gemma-3-4b | Multi-turn | 0.095 | 0.143 | 0.191 | 0.230 | 0.270 |
| GPQA | Gemma-3-4b | Localized MT | 0.119 | 0.270 | 0.421 | 0.468 | 0.516 |
| GPQA | Gemma-3-4b | In-place | 0.119 | 0.333 | 0.429 | 0.460 | 0.532 |
2. Domain Expert-in-the-Loop User Study (15 Published Authors, 1–10 Scale)
In a formal user study where 15 domain experts iteratively refined scientific paper summaries generated by Gemma-3-27b-it, In-Place and Mixed modes dominated Multi-turn interaction:
- Final-output satisfaction: Mixed (8.47) > In-place (7.60) > Multi-turn (5.93)
- Feedback acceptance: Mixed (8.67) > In-place (8.40) > Multi-turn (6.67)
- Correctness preservation: Mixed (8.53) > In-place (8.13) > Multi-turn (5.73)
- Correction through reasoning: Mixed (7.53) > In-place (7.07) > Multi-turn (5.47)
- Fatigue reduction (10 = no fatigue): Mixed (8.40) > In-place (7.73) > Multi-turn (4.07) (Multi-turn induced severe cognitive exhaustion)
Key Findings¶
- The Dialogue History Trade-off: Controlled history-pruning experiments in Section 5.3 uncover a fundamental tension: accumulating complete conversation history assists models in reproducing previously generated valid facts (CPR increases), but simultaneously suppresses forward logical reasoning (CTRR drops severely, turning negative for Gemma and Llama). This indicates that history accumulation induces excessive induction-head copying, blinding the model to necessary deductive branches.
- Asymmetric Token Conservation: Standard multi-turn feedback incurs linear context growth, causing input tokens on GPQA with Gemma-3 to surge from 205 to 8,824 tokens by Turn 9. In-Place Feedback keeps conditioning context bounded at ~980 tokens while halving generated token lengths per turn (from ~800 down to ~300), slashing overall computation costs by over 60%.
- High Robustness Under Incomplete Guidance: When feedback indicates only that an intermediate variable is incorrect without providing the ground truth (e.g., Table E.1 and Table E.3), In-Place Feedback maintains a consistent lead over Multi-turn (e.g., on ZebraLogic with Qwen, Grid Acc reaches 0.086 vs. 0.024), demonstrating that eliminating erroneous context is indispensable for autonomous hypothesis re-exploration.
Highlights & Insights¶
- Refining Prompts vs. Purifying Cognitive State: Prior research attempted to overcome multi-turn degradation through intricate prompting techniques or recursive reflection agents. This paper demonstrates that simply intervening on the input-output pipeline to physically purge incorrect tokens is far more reliable than expecting models to follow complex instructions about what to ignore.
- Microscopic Diagnostic Value of CPR/FAR/CTRR: Moving beyond black-box aggregate accuracy, this diagnostic metric trio provides the reasoning community with an actionable analytical tool to diagnose whether interactive failures stem from forgetting good content, ignoring feedback, or failing to propagate truths.
- The Pragmatic Superiority of Hybrid Human-AI Teaming: Qualitative interview insights illuminate the real-world sweet spot: users prefer conversational prompts for macro-level structural overhauls, but strongly demand in-place pointers for precision tuning. Exposing both modes yields optimal collaborative throughput.
Limitations & Future Work¶
- Reliance on Raw Continuation API Endpoints: In-Place Feedback fundamentally depends on a text-continuation API that accepts an arbitrary prefix and continues generation. Most commercial closed-source model providers (e.g., OpenAI, Anthropic) currently expose conversational Chat Completions APIs that append messages turn-by-turn, complicating the plug-and-play adoption of this paradigm for proprietary endpoints.
- Imperfect Automated Editing in Simulation: In the automated benchmark experiments, the surrogate in-place agent (GPT-5-mini) achieved appropriate edits in only 40.6% of MATH-hard and 53.9% of ZebraLogic instances, with common failure modes being partial fixes or under-pruning. Developing programmatic or formal verifier-backed dependency parsers would further unleash performance.
- Safety and Alignment Surface: Allowing direct user modification of an LLM's previous response before triggering continuation opens potential attack vectors, such as prefix injection or jailbreaks. Robust input-filtering guardrails must be placed on in-place edits in untrusted public environments.
Related Work & Insights¶
- vs. Self-Refinement Paradigms (Self-Refine, Reflexion, Madaan et al.): Self-refinement architectures prompt the model to generate a critique and then regenerate the full answer from scratch. Because the flawed response remains in context, the model repeatedly stumbles over its previous errors; In-Place Feedback cleanly eliminates the error from the conditioning window.
- vs. Memory-Augmented Prompting (MemPrompt, MemGPT, Packer et al.): Memory-augmented approaches store previous corrections in an external buffer to guide future prompts. However, they continue to regenerate full responses and often reintroduce past mistakes; In-Place Feedback directly repairs the local token sequence with zero memory-retrieval overhead.
- vs. UI-Level Interactive Interfaces (DirectGPT, WaitGPT, Pang et al.): Prior HCI systems provide localized editing in the user interface, but their backend still appends the edits as conversational turns and submits the entire prior response to the LLM. This work extends localized editing down to the model's actual conditioning tensor context.
Rating¶
- Novelty: ⭐⭐⭐⭐☆ Repositions feedback from a conversational addendum to a direct cognitive state repair, elegantly breaking long-standing context pollution habits.
- Experimental Thoroughness: ⭐⭐⭐⭐⭐ Extremely rigorous—spans 5 diverse benchmarks, deep token-efficiency audits, granular turn-level CPR/FAR/CTRR diagnostics, and a 15-expert IRB-approved user study.
- Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally clear, mathematically precise, logically tight, and supported by exhaustive appendix documentation.
- Value: ⭐⭐⭐⭐⭐ Highly actionable for the design of next-generation IDEs, clinical copilot interfaces, and interactive theorem-proving platforms.