Skip to content

SCALE: Structured Case-law-guided Anticipatory Legal Reasoning

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/khatrimann/SCALE
Area: Information Retrieval & RAG
Keywords: legal reasoning, retrieval-augmented generation (RAG), issue-factor hierarchy, case law guides, European Court of Human Rights

TL;DR

Addressing standard legal RAG's inability to synthesize dense, heterogeneous case law and its reliance on surface factual similarity, SCALE transforms ECHR Case Law Guides into an explicit Issue–Factor Hierarchy, anticipates the legal issues and factual factors implicated by a case, and retrieves specific authority passages to generate faithful, precedent-grounded judicial reasoning.

Background & Motivation

Large language models have demonstrated notable capabilities across various legal NLP applications, including judgment prediction, summarization, and case retrieval. However, generating judicial reasoning that remains strictly grounded in facts, faithful to precedent, and consistent with the argumentative rigor of legal analysis remains a formidable challenge. When prompted to analyze whether a specific Convention article has been violated given the factual record of a case, a model must simultaneously determine the relevant legal questions, isolate legally salient facts from irrelevant narratives, and map those facts to binding or persuasive prior authority. Attempting all of these complex cognitive leaps implicitly within a single-step generation frequently causes models to output superficial factual restatements, unsupported abstractions, or hallucinated precedents.

To improve evidentiary grounding, practitioner pipelines widely adopt retrieval-augmented generation (RAG). Nevertheless, standard legal RAG pipelines suffer from two fundamental architectural limitations. First, retrieved primary legal materials (such as full historical judgments) are notoriously long, heterogeneous, and unorganized relative to the specific reasoning task at hand. This forces the downstream model to bear the heavy burden of filtering, synthesizing, and aligning disparate sources during decoding—an especially fragile process when relevant legal tests are intertwined with peripheral case trivia. Second, standard retrieval is driven primarily by lexical or dense semantic similarity between raw case facts and candidate documents. In jurisprudence, relevance is governed by latent legal issues and normative factual factors rather than surface narrative resemblance; querying with raw facts frequently retrieves cases that share superficial factual overlap but turn on completely irrelevant legal questions.

Effective legal retrieval must therefore be anticipatory: rather than reacting to the surface form of the facts, the system must first identify the underlying issues and factual dimensions required for the ultimate legal analysis and retrieve authority aligned with those specific reasoning needs. The core idea is to externalize article-specific legal structure from European Court of Human Rights (ECHR) Case Law Guides into an Issue–Factor Hierarchy (IFH), using it at inference time to guide anticipatory, issue-aligned, and factor-specific retrieval before generating judicial reasoning.

Method

Overall Architecture

SCALE targets legal reasoning generation in the European Court of Human Rights (ECHR), conditioned on the case facts and an alleged Convention article. The overall workflow consists of three distinct stages: 1. Stage 1: Issue–Factor Hierarchy (IFH) Construction: Offline, article-specific official Case Law Guides are segmented into token chunks. An LLM extracts local issues, factual factors, and supporting paragraph citations for each chunk. These local structures are then iteratively reconciled and merged into a unified, traceable global IFH. 2. Stage 2: Issue and Factor Identification: At test time, given the case facts and an alleged article, the system performs a two-stage anticipatory selection. The LLM first identifies the relevant legal issues from the IFH. For each selected issue, it evaluates its associated factors against the case facts, retaining only supported factors along with their attached guide paragraph references. 3. Stage 3: Reasoning Generation: The full text of the referenced guide paragraphs is fetched from the source guide and appended to the case facts. Acting as a neutral ECHR judge, the LLM synthesizes this structured context to draft the formal judicial reasoning (Law Section) assessing potential violation.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input: Case Facts + Alleged Convention Article"] --> B["Stage 1: Offline Issue–Factor Hierarchy (IFH) Construction<br/>Chunk parsing of case-law guides → Iterative consolidation into global IFH"]
    B --> C["Stage 2: Anticipatory Issue and Factor Identification<br/>Facts match legal issues → Fine-grained selection of factual factors & ¶ citations"]
    C --> D["Stage 3: Precedent-Grounded Reasoning Generation<br/>Fetch guide text excerpts + Synthesize formal ECHR judicial assessment"]
    D --> E["Output: Structured ECHR Judicial Reasoning with Precedent Application"]

Key Designs

1. Issue–Factor Hierarchy Construction and Traceability: Converting Doctrinal Summaries into Structured Authority Graphs

Raw historical judgments are scattered and filled with idiosyncratic factual context. SCALE instead taps into official ECHR Case Law Guides, which synthesize decades of jurisprudence into recurring legal tests, critical factual considerations, and authoritative paragraph citations. To convert these guides into an executable knowledge scaffold, the guide text is partitioned into ~2,000-token chunks. An LLM extracts local hierarchies using precise definitions: a "Legal Issue" is a core legal question under an article (e.g., whether an interference pursues a legitimate aim), while a "Legal Factor" is a recurring factual consideration guiding its resolution (e.g., applicant vulnerability, nature of interference). Each element explicitly records its source paragraph numbers (e.g., ¶15, ¶99). Subsequently, an iterative merging prompt unifies these chunk-level graphs into a coherent master IFH, deduplicating equivalent issues, harmonizing factor granularity, and accumulating paragraph pointers to guarantee complete provenance.

2. Two-Stage Anticipatory Issue-Factor Identification: Structurally Decoupled Precedent Retrieval

To avoid the noise and semantic drift of querying directly with lengthy factual narratives, SCALE implements a top-down, anticipatory identification pipeline over the prebuilt IFH. In the first step, the LLM inspects the case facts alongside the article's candidate issues, selecting only those questions genuinely disputed by the circumstances. In the second step, for every activated issue, the model is presented with its child factors and instructed to identify only those factors concretely substantiated by the factual record. This two-tier filtering bypasses similarity-based vector retrieval over noisy corpora, directly yielding the exact paragraph references needed to support the legal argument and ensuring that precedent retrieval is factor-specific and doctrinally aligned.

3. Judicial Norm-Aligned Reasoning Generation: Neutral Assessment Grounded in Convention Standards

Upon collecting the retrieved guide passages, the model enters the final reasoning phase. The system prompt instructs the model to adopt the persona and formal drafting standards of an ECtHR judge. The generation adheres to a strict judicial structure: first, "The Parties' Submissions", neutrally summarizing applicant grievances and government defenses while treating allegations strictly as unproven claims; second, "The Court's Assessment", interpreting the scope of rights, applying core doctrine (such as proportionality, margin of appreciation, and procedural safeguards), and referencing specific precedents cited in the guide paragraphs to establish whether the domestic measures were justified; and third, "Conclusion", delivering an unambiguous holding on whether a violation occurred.

A Worked Example

Consider Case 237239, an intricate Article 6 dispute regarding civil-party constitution in criminal proceedings: - Case Background: Applicants lodged criminal complaints initiating domestic investigations. The domestic authorities displayed extensive inactivity and eventually discontinued the proceedings due to statutory limitation expiries, compounded by domestic procedural flaws. - Baseline Failure: Standard baselines like PR or CaseRAG over-generalize from investigative delay and inactivity, jumping directly to an Article 6 violation without examining whether civil-party status was a protected civil right under domestic law in that context. - SCALE Pipeline Execution: 1. Identifies the threshold issue from the IFH: whether the applicant's procedural standing engaged the civil limb of Article 6; 2. Pinpoints relevant factors: availability of alternative domestic channels, domestic legislative defects, and the applicant's own due diligence; 3. Retrieves controlling guide paragraphs analyzing Nicolae Virgiliu Tănase v. Romania, which establishes that failure to examine a civil-party application on the merits does not ipso facto constitute an unjustified restriction of access to court; 4. Delivers nuanced reasoning correctly concluding "no violation", recognizing that the applicants failed to exercise available remedies diligently.

Key Experimental Results

Main Results

The evaluation dataset is derived from the ECHR HUDOC database, covering the 10 most frequent Convention articles and randomly sampling 25 cases from 2025 (yielding 37 case–article instances). Experiments evaluate four open-weight models across two families: Llama-3.1 (8B, 70B) and Qwen3 (8B, 14B). SCALE is compared against Parametric Reasoning (PR), CaseRAG, Anticipatory CaseRAG (A-CaseRAG), GuideRAG, and Anticipatory GuideRAG (A-GuideRAG). A former Council of Europe legal expert evaluated all 888 reasoning traces across six dimensions on a 1–5 Likert scale.

Criteria Models PR CaseRAG A-CaseRAG GuideRAG A-GuideRAG SCALE (Ours)
Legal Issue Ident. Llama-3.1 8B 3.54 3.46 3.72 3.88 4.05 4.43
Llama-3.1 70B 3.70 3.61 3.84 4.01 4.15 4.39
Qwen3 8B 3.67 3.60 3.90 4.07 4.22 4.56
Qwen3 14B 3.80 3.72 4.00 4.15 4.28 4.59
Factual Faithfulness Llama-3.1 8B 3.19 3.28 3.42 3.39 3.57 3.91
Llama-3.1 70B 3.33 3.29 3.48 3.56 3.61 3.98
Qwen3 8B 3.41 3.47 3.60 3.55 3.74 4.15
Qwen3 14B 3.53 3.58 3.69 3.74 3.82 4.24
Logical Coherence Llama-3.1 8B 3.14 3.23 3.34 3.31 3.43 3.69
Llama-3.1 70B 3.29 3.38 3.42 3.49 3.54 3.82
Qwen3 8B 3.26 3.35 3.33 3.43 3.48 3.90
Qwen3 14B 3.41 3.47 3.56 3.54 3.65 4.00
Precedent Relevance Llama-3.1 8B 3.00 3.24 3.50 3.66 3.87 4.25
Llama-3.1 70B 3.16 3.36 3.62 3.79 3.99 4.29
Qwen3 8B 3.14 3.34 3.69 3.84 4.06 4.41
Qwen3 14B 3.26 3.45 3.78 3.93 4.13 4.47
Precedent Application Llama-3.1 8B 2.96 3.19 3.43 3.58 3.81 4.19
Llama-3.1 70B 3.14 3.31 3.55 3.71 3.93 4.23
Qwen3 8B 3.08 3.29 3.61 3.77 3.99 4.36
Qwen3 14B 3.21 3.42 3.74 3.88 4.08 4.42
Judicial Alignment Llama-3.1 8B 3.12 3.26 3.39 3.47 3.64 3.93
Llama-3.1 70B 3.26 3.34 3.49 3.63 3.76 4.03
Qwen3 8B 3.28 3.41 3.55 3.69 3.80 4.18
Qwen3 14B 3.40 3.48 3.66 3.78 3.91 4.27

Ablation & Analysis: IFH Quality and Automated Evaluation Reliability

Table 1: Expert evaluation of automatically constructed Issue–Factor Hierarchies (1–5 Likert scale; higher is better, including redundancy where higher indicates lower redundancy)

Level / Criterion Article 5 (Liberty) Article 6 (Fair Trial) Article 8 (Private Life) Article 10 (Expression)
Issue Completeness 5 5 5 5
Issue Precision 4 5 5 4
Issue Redundancy (low) 5 4 4 4
Factor Completeness 5 5 5 4
Factor Correctness 4 5 5 4
Factor Redundancy within Issue (low) 4 4 4 3
Factor Redundancy across Issues (low) 4 4 4 3

Table 2: Spearman rank correlation (\(\rho\)) between legal expert scores and LLM-as-a-Judge (GPT-4.1) scores

Criterion Spearman Correlation (\(\rho\)) Analysis
Judicial Alignment 0.34 Weak-moderate correlation when full ground-truth reasoning is supplied
Logical Coherence 0.31 Moderate correlation; surface structural flow is easier for LLMs to assess
Factual Faithfulness 0.21 Weak correlation; LLM judge struggles to separate legally salient facts
Legal Issue Identification 0.18 Weak correlation; LLM judge easily misled by superficially plausible claims
Precedent Relevance 0.11 Negligible correlation; automated judge cannot verify doctrinal fit
Precedent Application 0.09 Near zero correlation; automated judge fails to evaluate legal contextualization

Key Findings

  1. Explicit Legal Structure Compensates for Scale: The benefits of SCALE are particularly dramatic on smaller models. Both Qwen3 8B and Llama-3.1 8B with SCALE substantially outshine their unconstrained 70B counterparts across Legal Issue Identification (4.56 vs 4.39) and Precedent Application (4.36 vs 4.23), proving that structured domain scaffolding can overcome parameter-scale deficits in implicit planning.
  2. Dense Precedent Guides Outperform Dispersed Case Retrieval: Retrieving from Case Law Guides (GuideRAG / SCALE) achieves a decisive advantage over searching across 17,000 raw historical judgments (CaseRAG). Precedent guides consolidate disparate authority into task-aligned principles, sparing the generation model from the error-prone task of multi-source de-noising and cross-document synthesis.
  3. Severe Limits of LLM-as-a-Judge in High-Stakes Legal Evaluation: While GPT-4.1 preserves the broad rank-ordering of systems, it exhibits poor correlation with legal experts on fine-grained precedent dimensions (\(\rho = 0.09 \sim 0.11\)). Automated judges over-index on rhetorical fluency and formal judicial phrasing, blind to subtle misapplications of precedent or missed threshold issues, showing that human domain expertise remains irreplaceable.

Highlights & Insights

  • Doctrinal Anticipation over Lexical Matching: Rather than searching for documents superficially resembling case facts, SCALE establishes an anticipatory paradigm where retrieval is strictly driven by the latent legal issues and factual factors required for the judicial argument.
  • Auditable Provenance Pipeline: By decomposing reasoning into explicit issue selection, factor matching, and paragraph citation, the entire reasoning trace provides end-to-end transparency, enabling human practitioners to audit every step of the model's legal logic.
  • Uncovering the Precedent Evaluation Blindspot: The paper provides empirical evidence that LLM judges cannot reliably assess precedent relevance or application, establishing a vital benchmark and cautionary finding for legal AI development.

Limitations & Future Work

  • Reliance on Curated Case-Law Guides: SCALE is evaluated using official ECHR Case Law Guides. In jurisdictions lacking standardized guides, deriving such resources automatically (e.g., via LexGenie) could propagate extraction errors directly into the IFH.
  • Jurisdictional Specialization: The ECHR features clean, distinct separation between the Facts and Law sections. In jurisdictions like the US or UK where factual narration and legal analysis are tightly interwoven, isolating input context is substantially more challenging.
  • Future Directions: Extending the issue-factor retrieval paradigm to Long-Form Legal Question Answering (Long-Form Legal QA), grounding answers in statutory provisions and regulatory guidelines.
  • vs. LeCoPCR & LexKeyPlan: While prior works explored anticipatory retrieval using unconstrained generated concepts or forward-looking keywords, SCALE constrains anticipation within a formal, pre-compiled Issue–Factor Hierarchy, preventing hallucinations and thematic drift in legally complex disputes.
  • vs. LegalChainReasoner (Shi et al., 2026a): While LegalChainReasoner decomposes criminal statutes into flat premise-situation-conclusion triplets, SCALE addresses precedent-based balancing tests by organizing recurring factual factors under legal issues with direct paragraph pointers into case-law summaries.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Pioneering framework transforming unstructured case-law guides into an Issue–Factor Hierarchy for anticipatory, precedent-grounded legal reasoning.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Evaluated across 4 open-weight LLMs, 6 methodology baselines, 888 expert-reviewed traces, and in-depth LLM-as-a-Judge correlation analysis.
  • Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally articulate, theoretically grounded, and rigorous in legal methodology and empirical analysis.
  • Value: ⭐⭐⭐⭐⭐ Crucial contribution to domain-specific RAG architectures, legal AI safety, and automated legal evaluation methodology.