Skip to content

LegalWorld: A Life-Cycle Interactive Environment for Legal Agents

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://chidaic.github.io/Legal-world/
Area: LLM Agent
Keywords: Legal Agents, Life-Cycle Simulation, Interactive Environment, Cross-Stage State Transmission, Civil Litigation

TL;DR

Grounded in 75,309 paired first- and second-instance Chinese civil judgments, this paper introduces LEGALWORLD—the first five-stage life-cycle interactive environment spanning consultation to final appellate judgment—and LongJud-Bench, uncovering cross-stage error cascading and exposing dynamic courtroom advocacy as the primary frontier for legal agents.

Background & Motivation

Substantial advances have been achieved in legal artificial intelligence, encompassing legal domain-specific pre-training, judicial benchmarks, and multi-agent debate simulators. However, the vast majority of existing legal AI benchmarks and environments remain strictly confined to isolated single-scenario tasks. Standard benchmarks measure isolated abilities such as statute retrieval, legal element extraction, or single document drafting over static, fixed inputs. Meanwhile, emerging multi-agent legal simulators either model only a narrow slice of the process—such as adversarial courtroom debates—or reinitialize each successive scenario from shared ground-truth case materials rather than consuming the actual procedural artifacts produced by preceding steps, fundamentally ignoring cross-stage dependencies.

Real civil litigation is inherently a long-horizon, life-cycle causal chain: what a lawyer gathers from a client during initial consultation directly shapes the claims, legal grounds, and evidence structured in the complaint; the complaint and written defense establish the factual and legal boundaries for first-instance court investigation and debate; and the factual findings and disposition articulated in the first-instance judgment determine the scope of appeal and counter-arguments in the second instance. Drafting flaws, factual omissions, or misguided litigation strategies committed on day one inevitably propagate downstream into courtroom proceedings and alter the final adjudicative outcome. Evaluating legal agents on decoupled tasks fails to assess whether an agent possesses genuine procedural competence and cross-stage tactical coherence.

Furthermore, existing interactive environments suffer from inadequate behavioral and structural modeling: clients, lawyers, and judges possess distinct, heterogeneous epistemic boundaries and adversarial stances that co-evolve throughout the litigation, yet existing simulators rely on coarse, generic personalities or unconstrained visibility that fail to capture real-world information asymmetry. Long-horizon multi-turn interaction also causes catastrophic fact drifting and procedural hallucinations without disciplined memory and tool mechanisms. Core idea: model civil litigation as a causally connected five-stage, seven-sub-scenario state chain grounded in 75k paired court judgments, coupled with a domain-specific client persona framework (LCPF), role-asymmetric visibility matrices, a dual-layer memory system, and standardized Skill/Tool libraries to establish a high-fidelity interactive environment and life-cycle capability benchmark.

Method

Overall Architecture

LEGALWORLD converts real paired civil judgments into runnable, causally linked litigation trajectories spanning five connected stages across seven concrete sub-scenarios: \(S^{(0)}_c \xrightarrow{\text{LC}} S^{(1)}_c \xrightarrow{\text{CD/DD}} S^{(2)}_c \xrightarrow{\text{FIT}} S^{(3)}_c \xrightarrow{\text{AD/AR}} S^{(4)}_c \xrightarrow{\text{SIT}} S^{(5)}_c\). Three agent types participate in the simulation: clients (plaintiff \(a_p\) and defendant \(a_d\)), lawyers (plaintiff counsel \(a_{lp}\) and defendant counsel \(a_{ld}\), where one acts as the target lawyer \(a^*\) under evaluation and the other provides adversarial counter-moves), and stage-bound presiding judges (\(a_{j1}\) for first-instance and \(a_{j2}\) for second-instance).

Each stage state \(S^{(t)}_c = \langle D_c, R^{(t)}_c, O^{(\le t)}_c, H^{(\le t)}_c, M^{(t)}_c \rangle\) encapsulates the case seed \(D_c\), role mapping \(R^{(t)}_c\), accumulated legal artifacts \(O^{(\le t)}_c\), full dialogue history \(H^{(\le t)}_c\), and structured memory handle \(M^{(t)}_c\). Agents interact exclusively via role-specific, stage-bound interfaces \(I^{(t)}_{a,c} = \langle V^{(t)}_{a,c}, \Phi^{(t)}_a, \Sigma^{(t)}_a, U^{(t)}_a \rangle\), which enforce strict visibility restrictions, procedural instructions, Skill/Tool access, and permitted action sets, completely preventing omniscient ground-truth leakage.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["75,309 Paired Judgment Pairs<br/>Structured Case Seeds and Role Mappings"] --> B["Life-Cycle Causal State Chain<br/>Consultation→1st Drafting/Trial→Appeal Transition→2nd Drafting/Trial"]
    B --> C["Domain Client Persona Framework & Asymmetric Interfaces<br/>4D LCPF Persona Scheme and Stage-Bound Visibility Matrices"]
    C --> D["Dual-Layer Memory & Structured Tool/Skill Support<br/>In-Scenario Dialogue Cache + Global Structured Ledger + Skill Manuals"]
    D --> E["LongJud-Bench & Capability Evaluation<br/>Eight Fine-Grained Legal Capabilities & Trajectory-Level Reflection"]

Key Designs

1. Life-Cycle Causal State Chain: Isolated Task Evaluation -> 5-Stage 7-Subscenario Sequential Handoff -> Reflecting Cross-Stage Error Cascading and Causal Dependence
Prior benchmarks isolate subtasks, unable to observe how early procedural decisions constrain downstream outcomes. LEGALWORLD formulates civil litigation as an unbroken five-stage sequence: Legal Consultation (LC), Pre-Trial Complaint/Defense Drafting (CD/DD), First-Instance Trial (FIT), Pre-Appellate Appeal/Response Drafting (AD/AR), and Second-Instance Trial (SIT). Crucially, downstream stages do not re-initialize from ground truth; they consume the actual artifacts and statements generated upstream. The claims and evidence marshalled in the complaint dictate the scope of first-instance trial investigation, the first-instance judgment produced at trial becomes the primary target for appeal drafting, and early omissions directly weaken appellate standing. An environment-level transition (AD-Det) inspects genuine appellate conditions to dynamically remap original parties into appellant and appellee roles, establishing an authentic end-to-end procedural trajectory where early errors irreversibly amplify downstream.

2. Domain Client Persona Framework and Asymmetric Role Interfaces: Coarse Personalities and State Leakage -> 4-Dimensional Legal Personas and Stage Visibility Matrix -> Grounding Epistemic Gaps and Procedural Frictions
Generic personality frameworks (e.g., Big Five) fail to reflect how litigants behave in legal proceedings, where friction arises from varying legal comprehension and disclosure willingness rather than general extroversion. LEGALWORLD introduces the Legal Client Persona Framework (LCPF), specifying four orthogonal, legal-specific dimensions: Legal Literacy (confusing claims with personal grievances), Information Disclosure Willingness (withholding unfavorable facts until probed), Emotional Stability (displaying procedural anxiety or hostility), and Narrative Proficiency (providing fragmented or chronological accounts). Each dimension is assigned High, Medium, or Low levels. In parallel, the environment enforces a strict visibility matrix \(V^{(t)}_{a,c}\): clients cannot inspect opposing private claims or unpresented evidence, judges remain blind to settlement bottom lines, and lawyers must actively elicit facts through multi-turn dialogue, capturing genuine information asymmetry.

3. Dual-Layer Memory and Structured Tool/Skill Infrastructure: Memory Hallucination and Format Drift -> Decoupled Local Cache and Durable Case Ledger -> Maintaining Cross-Stage Consistency and Action Compliance
To prevent fact drifting, catastrophic forgetting, and format degeneration over lengthy multi-stage interactions, LEGALWORLD establishes a dual-layer memory mechanism. In-scenario local memory \(L^{(t)}_{a,c}\) maintains turn-level dialogue coherence within a single scenario without durable commitment. At stage completion, participating lawyers and clients consolidate new information into global case memory \(G^{(t)}_{a,c}\), a structured persistent ledger covering factual timelines, evidence status (sources, proof targets, admission), dispute focuses, legal analyses, client profiles, and confirmed settlement thresholds. Memory updates occur exclusively through bounded revise and expand tool operations. Furthermore, the architecture equips agents with a declarative Skill library (procedural step-by-step guidance manuals) and an executable Tool library (statute search, case retrieval, citation verification, and formal document exporting), ensuring actions remain legally compliant.

Interactive Mechanism and Evaluation Criteria

Trial stages (FIT and SIT) execute a structured trial procedure governed by the presiding judge agent (Algorithm 1). Across court investigation, evidence cross-examination, judicial questioning, legal debate, and closing statements, the judge autonomously directs turns based on extracted dispute focuses \(Q\) and accumulated dialogue traces \(H^{(t)}_c\). The judge also probes settlement mediation; if rejected, the judge synthesizes the final structured judgment artifact.

Built upon LEGALWORLD, LongJud-Bench measures eight core professional legal capabilities for the target lawyer \(a^*\), organized across the three procedural phases: 1. Legal Consultation: Issue Spotting (identifying legal relationships, applicable statutes, procedural risks, and actionable legal advice from unstructured client inquiries); 2. Document Drafting: Party Identification (exact-match accuracy on structured party slots), Claim Construction, Fact Marshalling, and Evidence Marshalling (semantic scoring against ground-truth case records); 3. Courtroom Advocacy: Position Consistency (consistency between oral arguments and pleaded filings), Evidentiary Advocacy (handling cross-examination and fact integration), and Legal Reasoning (depth and validity of judicial arguments).

Scoring combines exact matching for structured entities and Claude-Sonnet-4.6 LLM-as-Judge scoring (normalized to \([0, 1]\)) guided by detailed 0–10 rubrics. Drafting and advocacy are evaluated independently in both first- and second-instance contexts to expose procedural adaptation.

Key Experimental Results

Main experiments are conducted on the cause-balanced Light split (100 cases across the top 20 causes of action), requiring ~500,000 tokens per case trajectory. Non-target environment agents are instantiated with Qwen3.5-Plus. The table below presents the eight capability scores across six leading LLM backbones (reporting first-instance / second-instance scores for drafting and advocacy):

Model (Lawyer Backbone) Consult: Issue Spotting Drafting: Party ID Drafting: Claim Constr. Drafting: Fact Marshalling Drafting: Evidence Marshalling Trial: Position Consist. Trial: Evidentiary Advocacy Trial: Legal Reasoning
Kimi-K2.5 0.67 0.80 / 0.85 0.69 / 0.71 0.68 / 0.74 0.74 / 0.52 0.63 / 0.65 0.55 / 0.59 0.56 / 0.58
Qwen3.5-Plus 0.62 0.71 / 0.85 0.72 / 0.72 0.66 / 0.70 0.69 / 0.70 0.62 / 0.64 0.53 / 0.54 0.53 / 0.58
GPT-5.2 0.63 0.53 / 0.83 0.60 / 0.70 0.60 / 0.68 0.71 / 0.63 0.62 / 0.64 0.61 / 0.55 0.57 / 0.57
DeepSeek-V4-Flash 0.62 0.64 / 0.83 0.65 / 0.65 0.67 / 0.69 0.69 / 0.57 0.60 / 0.63 0.52 / 0.54 0.52 / 0.57
GLM-4.7 0.54 0.56 / 0.82 0.66 / 0.66 0.67 / 0.70 0.72 / 0.71 0.61 / 0.61 0.53 / 0.52 0.48 / 0.51
Qwen3.5-Flash 0.56 0.46 / 0.82 0.56 / 0.69 0.62 / 0.72 0.50 / 0.53 0.53 / 0.55 0.46 / 0.41 0.45 / 0.45

Cross-Stage Causal Dependence & Error Propagation

To verify that downstream stages are causally dependent on upstream artifacts, the authors conducted intervention experiments by swapping the drafted pleadings with high-quality (aligned with ground-truth reference) or low-quality (deleting or reversing key claims and evidence) variants, then re-executing downstream trials.

Intervention Condition Downstream Target Stage Baseline Score (Base) Post-Intervention Score Induced Shift (\(\Delta\))
High-Quality Pleading Intervention FIT (First-Instance Trial) 54.70 62.73 +8.03
High-Quality Pleading Intervention SIT (Second-Instance Trial) 56.83 66.83 +10.00
Low-Quality Pleading Intervention FIT (First-Instance Trial) 55.14 28.19 -26.94
Low-Quality Pleading Intervention SIT (Second-Instance Trial) 57.78 29.72 -28.06

Naturally occurring upstream errors further validate this finding: when an upstream drafting stage contains substantive errors, the downstream trial score drops from 57.2 to 53.2 (-4.0) in the first instance and from 59.6 to 51.1 (-8.5) in the second instance, yielding an average pooled degradation of -5.9 points (95% CI [-9.7, -2.5]).

The authors investigated whether life-cycle interaction traces could serve as experience signals for agent self-improvement. Distilling reflective rules into reusable cause-specific Skill cards produced consistent gains on the top three civil causes (0–100 overall score):

Cause of Action Baseline Score (Base) With RLS Score Overall Gain (\(\Delta\)) Reflected Cases Gain Held-Out Cases Gain
Post-Divorce Property Disputes 60.08 63.76 +3.69 +3.81 +3.29
Private Lending Disputes 58.56 61.84 +3.28 +3.63 +2.32
Labor Disputes 66.25 70.47 +4.22 +5.24 +1.42
Average 61.56 65.29 +3.73 +4.20 +2.34

Key Findings

  1. No single model dominates across the life cycle: Aggregate performance hides sharp capability divergences. Kimi-K2.5 excels in consultation, structured drafting, and position consistency; GPT-5.2 lags on formal party extraction (0.53 in first instance) but dominates where dynamic reasoning matters most—first-instance evidentiary advocacy (0.61) and legal reasoning (0.57); Qwen3.5-Plus provides the most reliable claim construction (0.72 / 0.72).
  2. Courtroom advocacy is the shared frontier: Across all evaluated backbones, courtroom advocacy capabilities (position consistency, evidentiary advocacy, legal reasoning) hover substantially lower (0.45–0.65) than document drafting capabilities (0.65–0.85). Managing dynamic adversarial cross-examination and real-time judicial inquiries represents the primary bottleneck.
  3. Structural competencies saturate while reasoning discriminates: Party identification saturates at 0.82–0.85 in second instance due to first-instance judgment scaffolding, offering little model separation. Conversely, evidence marshalling and advocacy require synthesizing novel arguments under appellate scrutiny, widening cross-model gaps.
  4. Large-scale human validation confirms environment fidelity: 18,992 ratings from 217 legal evaluators give LEGALWORLD 8.96/10 on stage authenticity and 8.98/10 on role consistency. Human–LLM agreement (linearly weighted Gwet's AC2 of 0.754 and 0.772) matches the human–human baseline (0.766 and 0.764), verifying benchmark reliability.

Highlights & Insights

  • Authentic Life-Cycle Modeling: Moves past fragmented subtask evaluation by formalizing litigation as a closed-loop causal chain where upstream choices establish binding legal and factual constraints on downstream proceedings.
  • Domain-Tailored Client Persona Framework (LCPF): Replaces generic psychology traits with four domain-grounded dimensions (literacy, disclosure willingness, stability, narrative), generating realistic information friction during fact gathering.
  • Trajectory Reflection as a Training Asset: Proves that complete procedural simulation traces provide high-value grounded experience, enabling agents to distill reusable practice skills that transfer to held-out legal disputes.

Limitations & Future Work

  • Scope Restricted to Chinese Civil Procedure: Currently grounded exclusively in Chinese civil litigation and paired two-instance judgments; criminal, administrative, insolvency, and enforcement procedures are unmodeled.
  • Omission of Procedural Branching Events: Assumes a standard litigation track without branching events like jurisdictional challenges, asset preservation, counterclaims, expert witnesses, or settlement collapses.
  • Reliance on Automated Judicial Rubrics: While extensively validated against human experts, automated judges may miss nuanced tactical shifts in complex, non-standard litigation.
  • vs AgentCourt / AgentsCourt (EMNLP 2024 / ACL 2025): Previous work focuses only on isolated adversarial trial debate; LEGALWORLD connects consultation, pleading drafting, and dual-instance trials into an end-to-end life cycle.
  • vs Ready Jurist One (2026): While covering multiple legal scenes, Ready Jurist One reinitializes each scene from ground truth; LEGALWORLD strictly enforces state handoffs and error propagation across stages.
  • vs LawBench / CAIL / LexEval: Static benchmarks test memorization of statutes and classification; LEGALWORLD tests interactive, embodied legal problem-solving across long procedural horizons.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ [Pioneers life-cycle simulation for legal agents with LCPF personas and dual-layer memory infrastructure]
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ [Extensive 75k corpus, multi-model capability profiling, causal interventions, reflection learning, and large-scale human validation with 217 legal evaluators]
  • Writing Quality: ⭐⭐⭐⭐⭐ [Exceptionally rigorous formulation, pristine architectural structure, and comprehensive appendices]
  • Value: ⭐⭐⭐⭐⭐ [Provides a definitive benchmark and foundation for developing and evaluating autonomous legal agents]