ACL2026 LLM Agent AI paper notes paper summaries Agents LLM Reasoning Adversarial Robustness Dialogue Multimodal/VLM

🦾 LLM Agent¶

💬 ACL2026 · 121 paper notes

📌 Same area in other venues: 🧠 NeurIPS2026 (8) · 🎞️ ECCV2026 (36) · 📷 CVPR2026 (44) · 🔬 ICLR2026 (209) · 🧪 ICML2026 (83) · 🤖 AAAI2026 (59)

🔥 Top topics: Agents ×66 · LLM ×41 · Reasoning ×11 · Adversarial Robustness ×7 · Dialogue ×4

OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning: OctoTools is a training-free, user-friendly, and extensible multi-agent framework. By utilizing standardized Tool Cards to encapsulate heterogeneous tools, a Planner-Executor separation paradigm, and a task-specific toolset optimization algorithm, it achieves an average accuracy improvement of +9.3% over GPT-4o and up to +10.6% over frameworks like AutoGen and LangChain across 16 diverse benchmarks.
Mem^p: Exploring Agent Procedural Memory: This paper proposes the Mem^p framework to systematically study how to build learnable, updatable, and lifelong evolving procedural memory for LLM Agents. By distilling past task trajectories into fine-grained step-by-step instructions and high-level script abstractions, combined with a dynamic update mechanism (addition/validation/reflection/elimination), the authors achieve continuous success rate improvements and significant reductions in execution steps on TravelPlanner and ALFWorld.
MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agents: MAGMA decouples the memory of LLM agents into four orthogonal relation graphs: semantic, temporal, causal, and entity. It employs intent routing and adaptive beam search for policy-guided traversal across the appropriate graphs, complemented by a dual-stream writing mechanism ("Fast Path" for synchronous ingestion and "Slow Path" for asynchronous LLM consolidation). On LoCoMo, it achieves a Judge score of 0.700, comprehensively outperforming A-MEM, Nemori, and MemoryOS, while maintaining a query latency of only 1.47s (40% faster than the runner-up).
OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulation: OPeRA is a user behavior dataset collected from real Amazon shopping processes. It aligns personas, web observations, fine-grained actions, and real-time rationales on the same timeline to evaluate whether LLMs can truly simulate a specific user's next shopping behavior.
ExpSeek: Self-Triggered Experience Seeking for Web Agents: ExpSeek proposes a proactive experience-seeking framework based on step-level entropy self-triggering, allowing Web Agents to determine when and what guidance is needed based on internal signals during interaction. It achieves absolute improvements of 9.3% and 7.5% on Qwen3-8B/32B respectively.
From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms: This paper provides a systematic survey of LLM Agent memory mechanisms using an evolutionary framework of "Storage → Reflection → Experience." It utilizes formal definitions to map these three stages to three functional signatures: "Trajectory Retention → Trajectory Refinement → Cross-Trajectory Abstraction." The storyline is structured around three RQs (Why-How-What), with a deep dive into two transformative mechanisms of the Experience stage: Active Exploration and Cross-Trajectory Abstraction.
MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End RL: MemSearcher replaces the "history concatenation" of search agents with "LLM-managed compact memory"—where only the (question, memory) is processed each round instead of (question, t₁, a₁, o₁, …). Utilizing multi-context GRPO, it propagates the advantage of the entire trajectory to each round for independent optimization. MemSearcher outperforms same-sized ReAct baselines across 3B/7B/14B scales on 7 QA benchmarks (the 7B model even surpasses the 32B ReSearch) while maintaining a constant context length of <4K tokens.
AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts: AgencyBench is proposed as a comprehensive benchmark comprising 138 real-world tasks to evaluate 6 core agent capabilities. Each scenario averages 90 tool calls and 1 million tokens, achieving fully automated evaluation via user simulation agents and Docker sandboxes.
TiMem: Temporal-Hierarchical Memory Consolidation for Long-Horizon Conversational Agents: TiMem organizes long-range conversational memory into a five-layer Temporal Memory Tree with explicit temporal containment. By employing complexity-aware retrieval to dynamically balance fine-grained facts and high-level personas, it improves accuracy on LoCoMo and LongMemEval-S while significantly reducing recalled context length.
Verified Critical Step Optimization for LLM Agents: CSO identifies "verified critical steps" from an agent's own failed trajectories where "changing a single action leads to task success." It constructs DPO preference pairs only at these critical decision points, enhancing the post-training performance of long-horizon LLM agents with fewer and more reliable supervisory signals.
LPO: Towards Accurate GUI Agent Interaction via Location Preference Optimization: This paper proposes Location Preference Optimization (LPO), which optimizes the spatial localization accuracy of GUI agents through entropy-based window rewards and physical distance-based dynamic location rewards integrated with the GRPO framework, achieving SOTA performance in both offline and online evaluations.
ZARA: Training-Free Motion Time-Series Reasoning via Evidence-Grounded LLM Agents: Ours proposes ZARA, a knowledge and retrieval-augmented multi-agent framework. By distilling sensor signals into a structured text knowledge base, performing class-wise retrieval, and employing hierarchical LLM reasoning, it achieves interpretable human activity recognition in a completely training-free setting, significantly outperforming existing methods on 8 datasets.
Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments: This paper proposes the FTRL framework, which constructs a stable and controllable tool-use training environment through a five-stage automated pipeline. It designs a verifiable reward mechanism combining tool call precision and task completion. When paired with preference optimization RL algorithms, it achieves an average tool-use performance improvement of over 10% on 7B-14B models, even surpassing the strongest closed-source models.
StructMem: Structured Memory for Long-Horizon Behavior in LLMs: StructMem proposes a structure-enhanced hierarchical memory framework. Through event-level dual-view extraction and cross-event semantic integration, it achieves SOTA performance (76.82%) on the LoCoMo long-dialogue benchmark while significantly reducing token consumption (1.94M vs. 35.8M for graph memory) and API call counts.
Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunities: This paper proposes the first formal framework for Agent Uncertainty Quantification (Agent UQ): modeling agent problem-solving trajectories as stochastic processes on Dynamic Bayesian Networks $P(\mathcal{F}_{\leq T}) = P(E_0, O_0) \prod_{i=1}^{T} P_{\pi,\mathcal{T}}(A_i|E_{i-1}, O_{i-1}) P(O_i|A_i, E_i)$. It unifies existing UQ paradigms (single-step QA, multi-step reasoning) as special cases and identifies four unique technical challenges of agent UQ through empirical analysis on $\tau^2$-bench.
Mem²Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation: This paper proposes Mem²Evolve, a self-evolving agent framework that achieves co-evolution of capability expansion and experience distillation through a dual memory mechanism (Asset Memory + Experience Memory). It achieves an average Pass@1 of 70.24% across 8 benchmarks in 6 task categories, outperforming the strongest experience-centric and capability-centric baselines by 11.80% and 6.46%, respectively.
CoEvolve: Training LLM Agents via Agent-Data Mutual Evolution: CoEvolve proposes an agent-data mutual evolution framework that extracts three types of weakness signals—forgetting, boundary, and rare patterns—from training trajectories to guide targeted environment re-exploration and task synthesis. This enables the training data distribution to dynamically adapt to agent capabilities, yielding absolute gains of 19-23% on AppWorld and BFCL.
Don't Act Blindly: Robust GUI Automation via Action-Effect Verification and Self-Correction: This paper proposes the VeriGUI framework, which utilizes a Thinking-Verification-Action-Expectation (TVAE) closed-loop reasoning mechanism and a two-stage training pipeline (Robust SFT + GRPO). It enables GUI Agents to verify the success of each operation and perform self-correction upon failure, significantly outperforming baselines at both 3B and 7B scales.
Exploring Reasoning Reward Model for Agents: The authors identify that current agentic RL typically employs sparse outcome rewards (evaluating only final correctness), which discards high-quality signals from intermediate reasoning steps. They propose Agent-RRM, a reasoning reward model generating structured feedback in three segments: <think>/<critique>/<score>. By systematically comparing three integration methods (C: pure critique refinement, R: scalar reward enhancement, U: combined critique + score GRPO), Reagent-U achieves 43.7% on GAIA and 46.2% on WebWalkerQA using Qwen3-8B. The results demonstrate that joint supervision using "language-level critique + numerical reward" is significantly more effective than single-signal approaches.
MCP-Flow: Facilitating LLM Agents to Master Real-World, Diverse and Scaling MCP Tools: MCP-Flow proposes a Web Agent-based automated pipeline to collect tool information from 1166 real-world MCP servers and synthesize 68,733 high-quality training data points. This allows small-scale fine-tuned models (0.6B-8B) to outperform SOTA large models like GPT-4o in MCP tool utilization.
Dynamic Generation of Multi-LLM Agents Communication Topologies with Graph Diffusion Models: This paper proposes Guided Topology Diffusion (GTD), which models the communication topology generation of multi-LLM agents as a conditional graph diffusion process. It utilizes a proxy reward model to perform zero-order guidance at each denoising step, thereby generating task-adaptive collaboration networks that are sparser, more token-efficient, and more robust.
SynthAgent: Adapting Web Agents with Synthetic Supervision: This paper presents SynthAgent, a framework for adapting Web Agents based entirely on synthetic supervision. It systematically covers functional areas of web pages to synthesize diverse tasks through categorical exploration. Then, a dual refinement strategy is employed: task refinement (triggered by conflict detection to correct hallucinations) and trajectory refinement (denoising from a global perspective). SynthAgent significantly outperforms existing synthesis methods on WebArena and Online-Mind2Web.
AdaRubric: Task-Adaptive Rubrics for Reliable LLM Agent Evaluation and Reward Learning: This paper identifies that "LLM-as-Judge + Fixed Rubrics" (Helpfulness/Safety/Fluency) are poorly matched for evaluating goal-oriented agent trajectories. It proposes AdaRubric—where an LLM automatically generates task-specific N-dimensional evaluation rubrics based on task descriptions, followed by confidence-weighted step-by-step evaluations to produce dense reward signals. A DimensionAwareFilter is designed for DPO data construction to prevent "dimension masking." Evaluated on WebArena/ToolBench/AgentBench, it achieves a Pearson $r=0.79$ and brings a $+6.8$ to $+8.5\%$ task success rate improvement through DPO training.
Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agents: The authors propose TraitBasis—a fine-tuning-free, model-agnostic, and lightweight method that extracts user trait directions like "impatient/confused/skeptical/incoherent" within the hidden space using contrastive activation differences. These directions can be scaled, combined, and injected during inference to simulate challenging users with high fidelity. Integrating this into $\tau$-Bench to create the $\tau$-trait benchmark, they found that frontier agent performance drops by 4%–20% on average (up to 46%) under varying user behaviors, exposing the illusion that high benchmark scores equate to real-world robustness.
Lightweight LLM Agent Memory with Small Language Models: This paper proposes LightMem, a lightweight LLM agent memory system driven by multiple specialized Small Language Models (SLMs). By modularizing memory operations into a Controller (SLM-1), Selector (SLM-2), and Writer (SLM-3), and decoupling online processing from offline consolidation, it achieves an average F1 improvement of approximately 2.5 on the LoCoMo benchmark (compared to A-MEM), while maintaining an 83ms retrieval latency and 581ms end-to-end latency.
Robust Tool Use via Fission-GRPO: Learning to Recover from Execution Errors: Ours proposes Fission-GRPO, which dynamically transforms tool execution errors into online-policy correction training instances within the RL loop. By utilizing a learned error simulator to generate diagnostic feedback and resampling recovery trajectories, it improves the error recovery rate of Qwen3-8B by 5.7% and the overall accuracy from 42.75% to 46.75%.
Topology Matters: Measuring Memory Leakage in Multi-Agent LLMs: This paper systematically measures how communication topology affects the leakage of personally identifiable information (PII) in multi-agent LLM systems through the MAMA framework. It identifies dense connectivity and the distance between attackers and targets as critical factors determining leakage risk.
Why LLM Web Agents Fail: A Hierarchical Planning Perspective: This paper systematically analyzes the failure causes of LLM web agents through a hierarchical planning framework (high-level planning, low-level execution, and replanning). It discovers that PDDL representations outperform natural language planning, but low-level execution and perceptual grounding are the primary bottlenecks.
Temp-R1: A Unified Autonomous Agent for Complex Temporal KGQA via Reverse Curriculum Reinforcement Learning: Temp-R1 transforms Temporal Knowledge Graph Question Answering (TKGQA) from manually designed fixed prompt workflows into an autonomous agent trainable via reinforcement learning. By employing explicit internal actions, SFT cold start, GRPO, and a "hard-first" reverse curriculum, it outperforms strong baselines driven by GPT-4o/DeepSeek-V3 using an 8B open-source model.
ChartAgent: A Multimodal Agent for Visually Grounded Reasoning in Complex Chart Question Answering: ChartAgent transforms chart question answering from "textual chain-of-thought" to "acting on the image itself." By using a suite of chart-specific visual tools (segmenting pie slices, isolating bars, locating axes) within a ReAct loop and performing self-verification on intermediate visualizations, it achieves gains of up to 16.07% on ChartBench / ChartX for unannotated and numerical-heavy challenges, with a 17.31% improvement on the unannotated subset.
ToolOmni: Enabling Open-World Tool Use via Agentic Learning with Proactive Retrieval and Grounded Execution: This paper proposes ToolOmni, a unified agent framework that integrates proactive tool retrieval and retrieval-based tool execution into a single reasoning loop. Through a two-stage approach of cold-start SFT and decoupled multi-objective GRPO, it jointly optimizes retrieval and execution capabilities, achieving an end-to-end success rate on ToolBench that surpasses strong baselines by +10.8%.
Your LLM Agents are Temporally Blind: The Misalignment Between Tool Use Decisions and Human Time Perception: This paper reveals the "Temporal Blindness" of LLM Agents in multi-turn interactions—their inability to adjust tool-calling decisions based on the actual time elapsed between messages—and constructs the TicToc benchmark to evaluate this issue.
CodeStruct: Code Agents over Structured Action Spaces: This paper proposes the CodeStruct framework, which redefines code repositories as AST-based structured action spaces. It enables LLM code agents to perform read and edit operations through named program entities (rather than text snippets), achieving a $1.2-5.0\%$ accuracy improvement on SWE-Bench Verified while reducing token consumption by $12-38\%$.
Don't Adapt Small Language Models for Tools; Adapt Tool Schemas to the Models: This paper proposes PA-Tool, a training-free tool schema optimization method. By utilizing the "peakedness" signal borrowed from data contamination detection, it identifies naming patterns familiar to the model from pre-training. By renaming tool components to align with the internalized knowledge of Small Language Models (SLMs), PA-Tool achieves up to a 17% improvement on MetaTool and RoTBench, and reduces schema misalignment errors by 80%.
Grounding Agent Memory in Contextual Intent: STITCH introduces "contextual intent" (thematic scope + event type + key entity types) triples as structured retrieval cues for LLM agent long-term memory. These triples are induced online at each trajectory step. During inference, retrieval follows "label density ranking," performing structural matching before semantic scoring. On the newly constructed CAME-Bench, STITCH maintains performance as trajectories grow, outperforming the strongest baseline by 35.6% absolute (100% relative) on the Large subset.
How Adversarial Environments Mislead Agentic AI: This paper formalizes the "Adversarial Environment Injection" (AEI) threat model, decomposing it into Breadth Attacks (poisoning retrieval results to induce cognitive drift) and Depth Attacks (injecting phantom nodes to construct navigation traps leading to policy collapse). Through 11,000+ experiments, the study reveals that robustness against these two attacks is completely independent—a "robustness split" suggesting that current point-solution defense strategies are insufficient.
OCR-Memory: Optical Context Retrieval for Long-Horizon Agent Memory: OCR-Memory renders long-horizon agent interaction trajectories into images with numbered anchors, allowing a fine-tuned OCR retriever to first localize relevant segments in visual space and then retrieve original text by index. This approach maintains complete history under strict context budgets and improves long-horizon task performance on Mind2Web and AppWorld.
SEARL: Joint Optimization of Policy and Tool Graph Memory for Self-Evolving Agents: SEARL jointly optimizes agent policy parameters and external Tool Graph Memory. It addresses credit assignment in long trajectories by utilizing tool-anchored step-level advantages and process rewards, enabling small models to continuously create, reuse, and integrate tools in multi-hop QA and complex mathematical tasks.
BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Search: Addressing the reliability issue where RL-trained agentic search models rarely say "I DON'T KNOW," leading to hallucinations, BAPO introduces "group-based boundary-aware rewards + adaptive reward modulators" on top of GRPO. This allows the model to reject answering only when truly exceeding its boundaries. Compared to GRPO, BAPO improves reliability across four multi-hop QA datasets by approximately 9.7% on average and outperforms Search-R1 (trained on 90k samples) using only 5k training samples.
HeLa-Mem: Hebbian Learning and Associative Memory for LLM Agents: HeLa-Mem proposes a neuroscience-inspired memory architecture for LLM agents that models conversation history as a dynamic graph with Hebbian learning dynamics. It strengthens inter-memory connections through co-activation, condenses hub memories into semantic knowledge via reflective distillation, and combines semantic similarity with Hebbian spreading activation in a dual-path retrieval process, achieving state-of-the-art performance on LoCoMo with significantly fewer tokens.
PersonaAgent: Bridging Memory and Action for Personalized LLM Agents: PersonaAgent connects user history with tool-based actions through "personalized memory + personalized actions + test-time optimizable persona prompts," significantly outperforming baselines such as RAG, PAG, ReAct, and MemBank on multiple LaMP personalized decision-making tasks.
ProPer Agents: Proactivity Driven Personalized Agents for Advancing Knowledge Gap Navigation: ProPer models proactive agents as the problem of "discovering and calibrating unspoken task dimensions." Through a Dimension Generating Agent, a post-hoc reranker, and a Response Generating Agent, it selectively fills knowledge gaps, significantly improving response quality and win rates across medical, coding, and shopping recommendation tasks.
RecMem: Recurrence-based Memory Consolidation for Efficient and Effective Long-Running LLM Agents: RecMem draws from the "consolidation through repetition" principle in human memory, placing raw interactions into a lightweight subconscious memory first. It only invokes the LLM to generate episodic and semantic memory upon detecting semantic recurrence, thereby reaching or exceeding the QA accuracy of mainstream memory systems on LoCoMo and LongMemEval-S at significantly lower construction token costs.
Shopping Companion: A Memory-Augmented LLM Agent for Real-World E-Commerce Tasks: Shopping Companion constructs an e-commerce task benchmark with long-term user preference memory and a real product library. It employs a two-stage agent with dual rewards and tool-level rewards to jointly optimize preference identification and product recommendation, enabling a 4B model to approach the performance of strong closed-source models.
ToolGrad: Efficient Tool-use Dataset Generation with Textual "Gradients": ToolGrad reverses tool-use data generation from "writing queries first and searching tool chains via DFS" to "generating successfully executable tool chains first and then back-inferring user queries." By using an API selection loop similar to textual gradients to construct ToolGrad-500, the pass rate for data generation reaches 99.8%. Small models like Gemma-3 trained on this data outperform several powerful closed-source models in single-turn tool calling.
AVA: Attentive VLM Agent for Mastering StarCraft II: This paper proposes AVACraft—the first StarCraft II multimodal benchmark supporting both MARL and VLM decision-making paradigms (21 scenarios / RGB + Text + Structured State). It introduces the VLM baseline AVA (Multimodal Priority Reasoning + RAG + Dynamic Role Assignment). Experiments demonstrate that while MARL achieves only a 19–27% win rate after 5M training steps in base 3m scenarios, zero-shot VLM reaches 75–90%.
HiGMem: A Hierarchical and LLM-Guided Memory System for Long-Term Conversational Agents: This paper proposes HiGMem, a two-layer event-turn memory system. By enabling the LLM to browse event summaries first before predicting which fine-grained dialogue turns are worth reading, it achieves state-of-the-art F1 scores in four out of five categories on the LoCoMo10 benchmark with an order of magnitude lower retrieval volume.
ImplicitMemBench: Measuring Unconscious Behavioral Adaptation in Large Language Models: Ours proposes ImplicitMemBench, the first benchmark for systematically evaluating implicit memory in LLMs. It includes 300 test items across three cognitive paradigms: procedural memory, priming, and classical conditioning. Evaluations across 17 models reveal severe limitations: the best model achieves only 66% overall accuracy, far below the human baseline.
PRInTS: Process Reward Modeling for Long-range Information Retrieval: PRInTS migrates "Process Reward Models (PRM)" from short-form mathematical reasoning to long-range information retrieval (IR) Agents. By utilizing a 4B model that simultaneously learns to "assign dense scores to each step based on information gain" and "recursively compress expanding trajectory contexts," the method achieves a 9.3% average improvement for 32B-scale Agents via test-time best-of-$n$ selection. Notably, the 30B+4B combination outperforms the 671B DeepSeek-V3.1 on the GAIA benchmark.
SafeMCP: Proactive Power Regulation for LLM Agent Defense via Environment-Grounded Look-Ahead Reasoning: SafeMCP is an agent defense plugin deployed on the MCP server side. It utilizes an environmental dynamics world model for look-ahead reasoning to first filter tools that might expand dangerous power boundaries, and kemudian performs real-time interception of initiated hazardous calls. It simultaneously enhances safety and preserves task utility across PowerSeeking Bench, ToolEmu, and AgentHarm.
The Bitter Lesson of Diffusion Language Models for Agentic Workflows: A Comprehensive Reality Check: This paper systematically evaluates the performance of diffusion language models (dLLMs) in embodied and tool-use agents. It finds that despite the speed potential offered by parallel decoding, dLLMs significantly lag behind autoregressive (AR) LLMs in long-horizon causal planning and strict format generation. Furthermore, the authors utilize DiffuAgent to demonstrate that dLLMs are better suited as non-causal auxiliary modules, such as for memory compression and tool filtering.
What Makes an LLM a Good Optimizer? A Trajectory Analysis of LLM-Guided Evolutionary Search: Through large-scale experiments (15 LLMs × 8 tasks, 72K candidate solutions), this paper finds that excellent LLM optimizers function as "local refiners"—continuously producing frequent incremental improvements and gradually concentrating search within semantic space, rather than generating high-novelty jumpy breakthroughs. A key finding is that novelty itself does not predict optimization performance; novelty is beneficial only when the search remains sufficiently localized.
CLAG: Adaptive Memory Organization via Agent-Driven Clustering for Small Language Model Agents: This paper proposes CLAG, a cluster-based Agent memory framework. It organizes memories into semantically consistent clusters via SLM-driven routing, performs local evolutionary updates within clusters, and filters noise through two-stage retrieval. It significantly outperforms global memory pool baselines across multiple QA datasets.
FAMA: Failure-Aware Meta-Agentic Framework for Open-Source LLMs in Interactive Tool Use Environments: FAMA employs an independent "failure analysis agent + orchestration agent" set to automatically diagnose dominant failure modes of a baseline tool-use agent on multi-turn benchmarks like τ-bench. It then directs a mitigation agent to select a minimal subset of helper agents for context injection, achieving up to a 27% increase in task success rate on Qwen series open-source models.
FregeLogic at SemEval 2026 Task 11: A Hybrid Neuro-Symbolic Architecture for Content-Robust Syllogistic Validity Prediction: The authors propose FregeLogic, a hybrid neuro-symbolic system that combines a five-member LLM ensemble with a Z3 SMT solver as a tie-breaking judge, reducing the content effect by 16% while improving accuracy by 0.9% in syllogistic validity judgment.
Hierarchical Reinforcement Learning with Augmented Step-Level Transitions for LLM Agents: Ours proposes STEP-HRL, which iteratively condenses interaction histories into compact text summaries through a local progress module. This allows high-level and low-level policies to make decisions based only on step-level transitions rather than full histories, significantly improving performance and generalization on ScienceWorld and ALFWorld while reducing token consumption.
IntrAgent: An LLM Agent for Content-Grounded Information Retrieval through Literature Review: IntrAgent decomposes the process of "researchers reading papers to find information" into a two-stage pipeline: "ranking sections by structure, then iteratively reading sections and stopping when sufficient." This allows the LLM to extract fine-grained answers faithfully aligned with queries from a full scientific paper without relying on vector retrieval, outperforming RAG and research agent baselines by an average of 13.2% across five STEM fields on the new IntraBench benchmark.
LiTS: A Modular Framework for LLM Tree Search: LiTS decomposes LLM tree search into Policy, Transition, RewardModel, and unified data structures. Utilizing a decorator registry, it enables the modular reuse of search algorithms, components, and task logic across mathematical reasoning, environmental planning, and tool-use tasks. Furthermore, the study identifies that policy diversity in open-text action spaces serves as a primary bottleneck for tree search.
Mina: A Multilingual LLM-Powered Legal Assistant Agent for Bangladesh: Mina is developed as a multilingual LLM legal assistant specifically for Bangladesh's legal landscape. By utilizing a two-stage RAG pipeline to accurately retrieve acts and sections, combined with a toolchain and multilingual embeddings, it achieved a $75\text~~}80\%$ passing rate in the Bangladesh Bar Council MCQ exams. The operational cost of legal consultation is reduced to only $0.12\text{~~0.61\%$ of traditional methods.
Polaris: A Gödel Agent Framework for Small Language Models through Experience-Abstracted Policy Repair: Polaris transforms the recursive self-improvement of Gödel Agents into a "failure analysis → experience abstraction → minimal code patch → execution validation" policy repair loop tailored for 7B/8B small models. This achieves interpretable, persistently reusable policy-level improvements on MGSM, DROP, GPQA, and LitBench.
Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment: It is discovered that LLM Agents exhibit human-like "Actor-Observer Asymmetry" (AOA) cognitive bias during role-playing—tending to attribute their own failures to external factors as actors, while attributing others' failures to internal errors as observers. ReTAS is proposed to eliminate this bias through dialectical reasoning (Thesis-Antithesis-Synthesis) and GRPO alignment.
GOAT: A Training Framework for Goal-Oriented Agent with Tools: GOAT enables small open-source models to decompose high-level goals into sequences of interdependent API calls without human annotation. By automatically constructing a "dependency graph + call-first synthetic data" pipeline from API documentation, it drives open-source models to SOTA performance on RestBench, API-Bank, and the self-constructed GOATBench, even surpassing closed-source models in specific scenarios.
HAG: Hierarchical Demographic Tree-based Agent Generation for Topic-Adaptive Simulation: The HAG framework is proposed, formalizing population Agent generation as a two-stage hierarchical decision process. It uses a world knowledge model to construct a topic-adaptive demographic distribution tree for macro-distribution alignment, followed by real data retrieval and Agent augmentation to ensure micro-level individual consistency. On multi-domain benchmarks, it reduces aggregate alignment error by an average of 37.7% and improves sociological consistency by 18.8%.
Meta-Tool: Efficient Few-Shot Tool Adaptation for Small Language Models: Through a systematic comparison of hypernetwork LoRA adaptation vs. carefully designed few-shot prompting across four benchmarks, it was found that a 228-million-parameter hypernetwork provides zero gain—few-shot examples contribute +21.5%, document encoding contributes +5.0%, and the hypernetwork contributes 0%. A 3B model with effective prompting achieves 79.7% of average GPT-5 performance with 10x lower latency.
Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems: The paper proposes BRIGHT-PRO, which re-evaluates reasoning-intensive retrievers using multi-aspect evidence annotation and agentic search protocols. It also introduces RTriever-Synth to train RTriever-4B, demonstrating that retrievers should optimize for "evidence portfolio coverage" rather than single-passage relevance.
Spec-o3: A Tool-Augmented Vision-Language Agent for Rare Celestial Object Candidate Identification: This paper proposes Spec-o3, a tool-augmented vision-language agent that simulates the spectral inspection workflow of astronomers through Interleaved Multimodal Chain-of-Thought (iMCoT). Using a two-stage training approach with cold-start SFT and outcome-based RL, it improves the macro-F1 of rare celestial object identification from 28.3% to 76.5%, achieving an inference speed ~50x faster than manual inspection.
When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors: This paper proposes two complementary metrics, RPS and AGS, to quantify distillation-induced homogenization in LLM Agent tool-use behaviors. By distinguishing between mandatory and optional behaviors, cross-family behavioral inheritance patterns are revealed across 18 models. Notably, the behavioral similarity between Kimi-K2 and Claude Sonnet 4.5 is found to exceed even that of Anthropic's own models.
Agent-GWO: Collaborative Agents for Dynamic Prompt Optimization in Large Language Models: This paper proposes Agent-GWO, which introduces the leader-follower mechanism of the Grey Wolf Optimizer into a multi-agent framework to jointly optimize prompt templates and decoding hyperparameters (temperature, top-p, etc.). It consistently out-performs existing prompt optimization methods across 11 mathematical and hybrid reasoning benchmarks.
AnchorMem: Anchored Facts with Associative Contexts for Building Memory in Large Language Models: The AnchorMem memory framework is proposed, inspired by the Proust phenomenon. It decouples the retrieval unit (atomic facts) from the generation context (original interactions) and connects fragmented memories via an associative event graph. It significantly outperforms existing systems like A-Mem and Mem0 on the LoCoMo benchmark.
Benchmarking Web Agent Safety under E-commerce Deceptive Interfaces: The authors developed WebDecept—a lightweight, pluggable "deceptive interface injection layer" that can insert seven types of common real-world deceptive patterns (pop-ups, banners, domain redirection, hidden cart additions, price changes, etc.) into the VisualWebArena e-commerce environment at specific trigger times to test the safety of multimodal web agents. The results show that advanced agents like GPT-5.1, Claude 4.5, and Gemini 2.5 are generally vulnerable, particularly to "hidden cart/total price manipulation," where they almost entirely failed, and safety prompts were unable to mitigate these risks.
Context-Value-Action Architecture for Value-Driven Large Language Model Agents: The CVA (Context-Value-Action) architecture is proposed based on the S-O-R psychological model and Schwartz Value Theory. By utilizing a Value Verifier trained on real human data to decouple behavior generation from cognitive reasoning, it effectively alleviates the behavior polarization issue in LLM agents, significantly outperforming baselines on CVABench which contains over 1.1 million real interaction trajectories.
Do LLM Agents Mirror Socio-Cognitive Effects in Power-Asymmetric Conversations?: This paper simulates power-asymmetric conversations using professional roles and personas, finding that LLM agents replicate socio-cognitive effects such as pronoun usage patterns, language coordination, authoritative persuasion, and harmful compliance. While some effects enhance conversational realism, others introduce significant safety risks.
Don't Click That: Teaching Web Agents to Resist Deceptive Interfaces: This work formalizes "adversarial deceptive UI" as an independent defense problem for web agents. It proposes the two-stage framework DUDE (hybrid-reward RL with asymmetric penalties to train an evaluator + iterative experience summarization to distill failure modes into transferable context) and releases the RUC benchmark containing 1407 real/synthetic scenarios. Across three VLM agent bases, it reduces deception-induced failure rates from 23.5% to 1.5%, pushes task success rates from 9.5% to 60.5%, and demonstrates zero-shot transferability of Stage-2 prompts to closed-source models.
FedGUI: Benchmarking Federated GUI Agents across Heterogeneous Platforms, Devices, and Operating Systems: FedGUI is the first comprehensive federated learning benchmark for cross-platform GUI agents, containing six datasets covering mobile, web, and desktop platforms. It systematically investigates the impact of four dimensions of heterogeneity—platform, device, operating system, and data source—on the training of federated GUI agents.
MOOSE-Copilot: A Web-Based Interactive Assistant for Unified Exploratory and Fine-Grained Scientific Hypothesis Discovery: MOOSE-Copilot unifies divergent exploration of scientific ideas and convergent refinement of fine-grained hypotheses into a visual human-AI collaborative system, significantly enhancing hypothesis discovery through three explicit human signals: initial blueprints, stage routing, and feedback.
SOLAR-RL: Semi-Online Long-horizon Assignment Reinforcement Learning: SOLAR-RL utilizes offline trajectory reconstruction, failure point detection, and target-aligned reward shaping to process static GUI data into long-horizon training signals with pseudo-online feedback. This allows the Qwen2.5-VL-7B scale GUI agent to achieve stable performance on Android Control, GUI-Odyssey, and Android World, matching or exceeding strong offline baselines.
Supplement Generation Training for Enhancing Agentic Task Performance: SGT (Supplement Generation Training) trains a small LLM (1.7B) to generate instance-specific supplemental text (reasoning clues, summaries, error reminders, etc.). When appended to the input, these supplements allow a frozen large Actor model to solve tasks more effectively, achieving an average improvement of 21% across 5 benchmarks without modifying the Actor's parameters.
TheraAgent: Self-Improving Therapeutic Agent for Precise and Comprehensive Treatment Planning: TheraAgent transforms treatment plan generation from a one-shot response into a generate-reflect-refine self-improving agent workflow. By utilizing a clinical multidimensional evaluator, TheraJudge, and score-aware memory to continuously refine plans, it significantly outperforms strong baselines in the HealthBench treatment planning subset and blind physician evaluations.
Towards Scalable Lightweight GUI Agents via Multi-role Orchestration: This paper proposes the LAMO framework, which trains a lightweight 3B MLLM into a GUI Agent capable of flexible multi-role orchestration through role-oriented data synthesis and two-stage training (SFT with Perplexity-Weighted Cross-Entropy + Multi-task RL). Operating in three modes—monolithic inference, multi-agent collaboration, and plug-and-play policy executor——it achieves a 77.6% success rate on AndroidWorld when paired with a GPT-5 planner, surpassing specialized GUI Agents with 72B parameters.
Waking Up Blind: Cold-Start Optimization of Supervision-Free Agentic Trajectories: This paper proposes SPECTRA, a framework that requires no supervised trajectories. By utilizing cold-start Reinforcement Learning (GRPO) and soft-structured multi-round rollout topological constraints, it enables Small Vision-Language Models (SVLMs) to autonomously discover effective tool-calling and visual reasoning behaviors through pure environment interaction. It achieves up to a 5% increase in task accuracy and a 9% improvement in tool efficiency across 4 multimodal benchmarks, while introducing the Tool Instrumental Utility (TIU) metric to quantify tool efficacy in unsupervised settings.
WebClipper: Efficient Evolution of Web Agents with Graph-based Trajectory Pruning: WebClipper models long tool-call trajectories of Web Agents as "Action Node-Information Node" state graphs and mines the minimum necessary DAG to prune cyclic searches and invalid branches. This reduces the average tool rounds by approximately 21% and tokens by 19.4% for Deep Research agents while maintaining or even improving accuracy.
YIELD: A Large-Scale Dataset and Evaluation Framework for Information Elicitation Agents: This paper proposes Information Elicitation Agents (IEA) as a new dialogue paradigm and releases YIELD, the first large-scale human-to-human information elicitation dialogue dataset (2,281 dialogues, 26M tokens). The study formalizes information elicitation as a finite-horizon POMDP and designs specialized evaluation metrics (Conformity, Progression, TLR). Experiments demonstrate that fine-tuning on YIELD significantly improves the alignment of Large Language Models (LLMs) with authentic elicitation behaviors.
A Multi-Agent Framework for Feature-Constrained Difficulty Control in Reading Comprehension Item Generation: This paper proposes MAFIG, a framework that leverages multi-agent collaboration, feature-level evaluators, and iterative revision to generate multiple-choice reading comprehension questions. Compared to single-turn prompting, it significantly improves the satisfaction rate of constraints such as vocabulary, passage length, sentence length, reasoning complexity, factuality, and option neutrality, while providing a more stable monotonic increase in difficulty.
AgenticEval: Toward Agentic and Self-Evolving Safety Evaluation of Large Language Models: AgenticEval redefines LLM safety evaluation as a "continuous, self-evolving red-teaming process": the Specialist decomposes unstructured regulatory text into an atomic rule knowledge base; the Generator creates multimodal and multi-format Question Groups centered around each rule; the Evaluator + Analyst continuously transform failures from the current round into more aggressive attack strategies for the next. After three iterations, the compliance rate of GPT-5 under the EU AI Act plummeted from 72.50% to 36.36%, revealing that static benchmarks significantly overestimate the safety levels of large models.
ATLAS: Adaptive Trading with LLM AgentS Through Dynamic Prompt Optimization and Multi-Agent Coordination: This paper proposes the ATLAS multi-agent financial trading framework and the Adaptive-OPRO prompt optimization method. By utilizing specialized analyst agents to prepare heterogeneous market information and dynamically optimizing the instruction prompts of the central trading agent based on delayed noisy feedback, the system significantly outperforms baselines across diverse volatile market environments.
AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage: AutoReproduce proposes a multi-agent framework that utilizes a "Paper Lineage" algorithm to mine implicit domain knowledge from referenced literature. This enables end-to-end automatic reproduction of paper experiments, achieving a code execution rate of 94.87% and a performance gap of only 19.72% on the self-constructed ReproduceBench.
BookAgent: Orchestrating Safety-Aware Visual Narratives via Multi-Agent Cognitive Calibration: BookAgent is a safety-aware multi-agent framework that utilizes a three-stage closed-loop architecture consisting of a Value-Aligned Storyboard (VAS) + Iterative Cross-modal Refinement (ICR) + Temporal Cognitive Calibration (TCC) to generate high-quality, character-consistent, and safety-compliant picture book stories end-to-end from user drafts.
CIA: Inferring the Communication Topology from LLM-based Multi-Agent Systems: This paper proposes CIA (Communication Inference Attack), which, under a strict black-box setting where only the final output is observable, induces multi-agent systems to expose intermediate agent reasoning through adversarial queries. By combining global bias disentanglement with LLM weak supervision to model semantic correlations, it successfully reconstructs the MAS communication topology, achieving an average AUC of 0.87 and a peak of 0.99.
Collaborative Multi-Agent Scripts Generation for Enhancing Imperfect-Information Reasoning in Murder Mystery Games: Ours proposes a collaborative multi-agent framework for the automated generation of high-quality Murder Mystery game scripts and training data. Through a two-stage training strategy (CoT fine-tuning + GRPO reinforcement learning with ScoreAgent reward shaping), the multi-hop reasoning capability of VLMs under imperfect information is enhanced. This significantly improves VLM narrative reasoning, fact extraction, and deception resistance on WhodunitBench.
Conjunctive Prompt Attacks in Multi-Agent LLM Systems: This paper investigates conjunctive prompt attacks in multi-agent LLM systems: trigger keys embedded in user queries and hidden templates in compromised remote agents appear harmless individually, but activate harmful behavior when routing brings them to the same agent. Existing defenses (PromptGuard, Llama-Guard, etc.) cannot reliably prevent these attacks.
ConSensus: Multi-Agent Collaboration for Multimodal Sensing: ConSensus is a training-free multi-agent sensor fusion framework that assigns specialized agents to independently interpret different sensing modalities. By utilizing semantic fusion, statistical consensus, and hybrid arbitration, it achieves an average 7.1% accuracy improvement over single-agent methods across five multimodal sensing benchmarks, while reducing fusion token costs to approximately 1/12.7 of multi-round debate methods.
Debating the Unspoken: Role-Anchored Multi-Agent Reasoning for Half-Truth Detection: This paper proposes the RADAR framework, which detects half-truths based on omitted context through role-anchored (Politician vs. Scientist) multi-agent debate. Combined with a dual-threshold adaptive early stopping mechanism, it consistently outperforms single-agent and traditional multi-agent baselines under noisy retrieval conditions.
Diversity Collapse in Multi-Agent LLM Systems: Structural Coupling and Collective Failure in Open-Ended Idea Generation: This paper systematically reveals the "diversity collapse" phenomenon in multi-agent LLM systems by evaluating over 10,000 research proposals across three levels: model intelligence, agent cognition, and system dynamics. It demonstrates that stronger models, authority-driven role assignments, and dense communication topologies suppress semantic diversity, identifying the root cause as the interaction structure rather than a lack of model capability.
Efficient Multi-Agent System Training with Data Influence-Oriented Tree Search: This paper proposes DITS, using "training data influence scores" instead of traditional Q-values as the guiding signal for MCTS tree search and preference data selection. It derives an influence score estimation formula for non-differentiable metrics that can be calculated via forward inference, enabling MAS to achieve a 2.5–2.7% average improvement over Optima-iSFT-DPO across 7 datasets and 3 multi-agent tasks.
EvoSci: A Bio-Inspired Multi-Agent Framework for the Evolution of Scientific Discovery: This paper proposes EvoSci, which models the generation of scientific ideas as a multi-agent collaboration and bio-inspired evolutionary cycle. By constructing a problem space, executing research in teams, providing reviewer feedback, and performing entity-level crossover/variation/selection, it generates research ideas with higher novelty and overall quality across 10 scientific topics.
EvoSpark: Endogenous Interactive Agent Societies for Unified Long-Horizon Narrative Evolution: EvoSpark proposes a multi-agent framework for long-horizon narrative evolution. It addresses social memory stacking and narrative-spatial misalignment through three designs: hierarchical recursive memory (RSB for social cognitive metabolism), Generative Mise-en-scène (GMS for character-location-plot alignment), and the Emergent Character Grounding Protocol (ECGP to transform LLM hallucinations into persistent characters).
Explicit Trait Inference for Multi-Agent Coordination: This paper proposes the Explicit Trait Inference (ETI) method, which enables LLM agents to reason about and track the behavioral characteristics of partners based on the psychological dimensions of warmth and competence. This approach reduces payoff losses by 45-77% in economic games and improves task performance by 3-29% on MultiAgentBench.
From Query to Counsel: Structured Reasoning with a Multi-Agent Framework and Dataset for Legal Consultation: This paper constructs JurisCQAD—a large-scale dataset containing 43,000+ real Chinese legal consultations—and proposes the JurisMA multi-agent framework. By utilizing legal element graphs for structured task decomposition and dynamic multi-agent collaboration (Manager Agent + Format Check + Law Search), it significantly outperforms general and legal-specific LLMs on LawBench.
HACHIMI: Scalable and Controllable Student Persona Generation via Orchestrated Agents: HACHIMI formalizes "student persona generation" as the TAD-PG (Theory-Aligned and Distribution-Controllable) task. By employing a "propose-validate-revise" multi-agent framework integrated with neuro-symbolic validators and stratified sampling, it produces 1 million synthetic student personas for grades 1–12. Group-level evaluations on CEPS / PISA 2022 reveal a distinct "fidelity gradient"—high alignment for constructs like mathematics and curiosity, but weak alignment for well-being and family dynamics.
Latent Agents: A Post-Training Procedure for Internalized Multi-Agent Debate: The Internalized Multi-Agent Debate (IMAD) framework is proposed, utilizing a two-stage post-training pipeline (SFT + GRPO) to "internalize" multi-agent debate into a single LLM. This approach reduces token consumption by up to 93% and demonstrates through activation steering that the internalized model retains separable and controllable "agent subspaces" within its latent space.
LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey: This paper provides the first systematic review of "LLM-based Human-Agent Collaboration and Interaction Systems (LLM-HAS)"—reintegrating humans into the agent loop. It establishes a unified taxonomy across five dimensions (Environment/Profiling, Human Feedback, Interaction Type, Orchestration, and Communication) and introduces a Human Agency Scale (A1–A5) to quantify the necessary depth of human involvement in tasks.
MAGEO: From Experience to Skill — Multi-Agent Generative Engine Optimization via Reusable Strategy Learning: This paper reformulates Generative Engine Optimization (GEO) from instance-wise heuristic optimization into a strategy learning problem. It proposes the MAGEO multi-agent framework—where the execution layer consists of collaboration between preference/planning/editing/evaluator agents, and the learning layer distills validated editing patterns into reusable engine-specific strategy skills. By introducing the Twin Branch causal evaluation protocol and DSV-CF dual-axis metrics, MAGEO significantly outperforms heuristic baselines across three mainstream engines.
MASFactory: A Graph-centric Framework for Orchestrating LLM-Based Multi-Agent Systems with Vibe Graphing: MASFactory models LLM Multi-Agent Systems (MAS) as Node/Edge computational graphs and introduces the "Vibe Graphing" three-stage pipeline (Role Assignment → Structure Design → Semantic Completion) to compile natural language intent into executable MAS workflows. It provides Context/Message Adapters, ComposedGraph templates for reuse, and VS Code visualization. On 7 benchmarks, it replicates 5 representative MAS with comparable or superior performance; end-to-end Vibe Graphing reduces ChatDev's 1,511 lines of code to 45 lines, with API costs an order of magnitude lower than Vibe Coding.
MATA: Multi-Agent Framework for Reliable and Flexible Table Question Answering: The authors propose MATA, a multi-agent framework for TableQA that uses a scheduler to prioritize reasoning paths (CoT/PoT/text2SQL), a confidence checker to filter answers, and a judge agent for arbitration. This model-agnostic, efficient framework achieves an average EM improvement of 40.1% across 10 LLMs.
Memory-Augmented LLM-based Multi-Agent System for Automated Feature Generation on Tabular Data: MALMAS is proposed as a memory-augmented LLM multi-agent system for automated feature generation on tabular data. Through a workforce of six specialized Agents exploring different feature space dimensions and a three-level memory mechanism (Procedural/Feedback/Conceptual) for cross-iteration optimization, it outperforms existing baselines across 16 classification and 7 regression datasets.
Multi-Agent Reasoning Improves Compute Efficiency: Pareto-Optimal Test-Time Scaling: This paper compares self-consistency, self-refinement, multi-agent debate, and Mixture-of-Agents under a unified computational budget. It finds that multi-agent reasoning, particularly MoA, is more efficient on the Pareto front, improving MMLU-Pro accuracy from 64.3% to 71.4% at approximately 20x CoT budget.
ODUTQA-MDC: A Task for Open-Domain Underspecified Tabular QA with Multi-turn Dialogue-based Clarification: This paper proposes the ODUTQA-MDC task and benchmark, systematically investigating the detection and multi-turn dialogue clarification of user query ambiguity in open-domain scenarios for the first time. It constructs a large-scale dataset containing 25,105 QA pairs and designs the MAIC-TQA multi-agent framework to perform end-to-end "detection-clarification-reasoning" tabular QA.
OxyGent: Making Multi-Agent Systems Modular, Observable, and Evolvable via Oxy Abstraction: OxyGent encapsulates agents, tools, LLMs, and reasoning processes into pluggable Oxy atomic components. By utilizing permission-driven dynamic planning and the OxyBank data feedback mechanism, it simplifies the construction, monitoring, and continuous evolution of industrial-grade multi-agent systems.
PaperMentor: A Human-Centered Multi-Agent Writing Tutor for AI Research Papers on Overleaf: PaperMentor codifies the writing experience of senior researchers into an "Expert Skill Library" and employs 12 agents with distinct divisions of labor to review LaTeX papers in parallel. It provides actionable revision suggestions via Overleaf's native inline annotations without ghostwriting for the user. In user studies, 90.6% of comments were judged as "actionable," with both validity and actionability significantly exceeding the GPT-5.2 baseline without the skill library.
PosterForest: Hierarchical Multi-Agent Collaboration for Scientific Poster Generation: PosterForest utilizes a Poster Tree, which simultaneously encodes the hierarchical semantics of a paper and the spatial layout of a poster, as an intermediate representation. It employs recursive collaborative optimization between Content, Layout, and Feedback agents to generate scientific posters in a training-free manner. In human evaluations, it achieved a 59.2% overall preference, significantly outperforming P2P and Paper2Poster.
Preference Estimation via Opponent Modeling in Multi-Agent Negotiation: Proposes a preference estimation method that integrates LLM-extracted natural language preference signals into a Bayesian opponent modeling framework. By combining qualitative cues with quantitative bidding information through a linguistic likelihood function in multi-party multi-issue negotiations, the Full Agreement Rate (FAR) is improved from 37% to 62%.
PROTEA: Offline Evaluation and Iterative Refinement for Multi-Agent LLM Workflows: PROTEA is an offline debugging platform for multi-agent LLM workflows. It localizes the cause of degraded final answers to specific nodes through node-level evaluation, backward-generated intermediate expectations, and editable prompt revisions, enabling a closed-loop verification of modification effects.
RoadMapper: A Multi-Agent System for Roadmap Generation of Solving Complex Research Problems: This paper proposes the RoadMap research roadmap generation benchmark and the RoadMapper multi-agent system, which forms a closed loop consisting of knowledge retrieval, logic/granularity critiques, revision, and a DPO evaluator. On complex Chinese and English research problems, it improves performance by an average of 7-9 points compared to direct prompting and significantly reduces the time cost for experts to design roadmaps.
Scaling External Knowledge Input Beyond Context Windows of LLMs via Multi-Agent Collaboration: The authors propose ExtAgents, a multi-agent framework that addresses the performance bottleneck where existing multi-agent methods fail to scale when external knowledge exceeds the context window. By implementing global knowledge synchronization (information exchange among all Seeking Agents) and cumulative reasoning (gradual injection of filtered knowledge into the Reasoning Agent), the framework significantly improves performance in multi-hop QA and long summary generation tasks.
Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systems: TraceElephant advocates that failure attribution in multi-agent systems should be evaluated under full execution traces visible to developers. It provides 220 failed traces with annotations for responsible agents and critical failure steps, demonstrating that full observability improves step-level attribution from 16% (output-only) to over 28%-30%.
SILO-BENCH: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systems: This paper proposes SILO-BENCH, a role-agnostic benchmark for evaluating distributed coordination in multi-agent LLM systems. Comprising 30 algorithmic tasks across three communication complexity levels and 1620 experiments over 54 configurations, it reveals a critical "communication-reasoning gap": while agents can spontaneously form rational communication topologies and actively exchange information, they systematically fail to integrate distributed states into correct answers.
Social Dynamics as Critical Vulnerabilities that Undermine Objective Decision-Making in LLM Collectives: This paper proves that representative agents in LLM multi-agent systems are not only limited by their own reasoning capabilities but are also significantly influenced by "social dynamics"—such as the number of peers, peer capabilities, argument length, and rhetorical style—leading to incorrect decisions on tasks with objective answers.
To Trust or Not to Trust: Attention-Based Trust Management for LLM Multi-Agent Systems: This paper proposes the first comprehensive definition of "trustworthiness" for LLM Multi-Agent Systems (LLM-MAS) based on six orthogonal dimensions of Grice's Cooperative Principle. It discovers that LLM attention patterns can distinguish different types of trustworthiness violations, leading to the design of A-Trust, a lightweight evaluation method and an end-to-end Trust Management System (TMS) that improves malicious message detection rates to 77-90% under various attacks.
Towards Robust Real-World Spreadsheet Understanding with Multi-Agent Multi-Format Collaboration: Proposes SpreadsheetAgent, a two-stage multi-agent framework that achieves robust real-world spreadsheet understanding by performing progressive region reading and cross-verification using code execution, vision, and LaTeX formats, without exceeding LLM context limits.
Towards Self-Improving Error Diagnosis in Multi-Agent Systems: Ours proposes the ErrorProbe framework, which achieves self-improving semantic fault attribution in multi-agent systems through MAST taxonomy-driven structured decomposition, symptom-driven backward tracing, and a verified memory mechanism, significantly outperforming baselines in step-level error localization.
When Identity Skews Debate: Anonymization for Bias-Reduced Multi-Agent Reasoning: This paper points out that LLMs in multi-agent debates change their stances based on "who said it" rather than "what was said," and quantifies and mitigates this identity-driven bias through response anonymization and the Identity Bias Coefficient (IBC).