Skip to content

JPO: Juris Policy Optimization for Structured Legal Argumentation

Conference: EMNLP 2026
Paper: ACL Anthology
Area: Multimodal VLM
Keywords: Legal Judgment Prediction, Policy Optimization, Reinforcement Learning, Structured Legal Reasoning, Cross-Step Consistency

TL;DR

JPO introduces a two-stage post-training framework that first establishes a standardized four-step legal reasoning scaffold (fact extraction → statutory analysis → charge determination → sentence prediction) via structured SFT, and subsequently optimizes the policy with reinforcement learning under a composite reward balancing legal accuracy, structural completeness, and cross-step logical consistency, augmented by token-level advantage reweighting and adaptive gradient clipping.

Background & Motivation

Criminal judgment prediction (LJP) aims to deduce judicial outcomes—including applicable statutory articles, criminal charges, and sentencing terms—directly from the narrative facts of a case, serving as a critical capability for intelligent legal assistance and automated case processing. Unlike standard classification benchmarks, criminal adjudication represents an intrinsically structured, hierarchical dependency chain. A legitimate judicial decision requires strict inter-stage coherence: cited statutory provisions must find solid factual footing within the established circumstances; the predicted criminal charges must be rigorously qualified and justified by those specific statutes; and the final sentence must be fully compatible with the statutory penalty range and relevant mitigating or aggravating factors. In civil law jurisdictions, high final-label accuracy in isolation is insufficient: a model that guesses the correct charge while relying on irrelevant statutory articles, or predicts an acceptable prison term without a coherent legal rationale, creates severe compliance hazards and poses unacceptable reliability risks in high-stakes judicial environments.

Despite the rapid emergence of large language models for reasoning-intensive legal applications, adapting general-purpose foundation models to criminal judgment prediction remains severely bottlenecked by two fundamental obstacles. First, existing LJP corpora (such as the classical CAIL2018 benchmark) provide ground-truth case facts and final decision labels, but lack granular supervision for intermediate legal deductions and interpretive reasoning traces. As a consequence, standard supervised fine-tuning prompts models to exploit shallow statistical co-occurrences and spurious linguistic correlations between facts and outcomes rather than mastering legally grounded reasoning principles. Second, conventional post-training techniques like generic RLHF, DPO, or standard PPO optimize almost exclusively for final output correctness or broad stylistic preferences. While recent works have explored reasoning quality evaluation, they predominantly rely on LLM-as-a-judge rubrics; such evaluators inevitably reflect the internal subjective biases and format idiosyncrasies of proprietary models rather than the rigorous, rule-bound deductive structure of statutory adjudication.

The central tension lies in the mismatch between the continuous, unconstrained generation dynamics of LLMs and the rigid, causally coupled stage-to-stage dependencies required by legal adjudication. Off-the-shelf post-training fails to translate non-differentiable legal constraints into robust, token-level supervisory signals over long-horizon legal justifications. Core idea: re-conceptualize criminal judgment prediction as an explicit, causally linked four-stage reasoning process (Facts → Statutes → Charges → Sentencing), where structured supervision from a teacher model provides initial reasoning scaffolds, and a composite reinforcement learning objective—incorporating closed-form proxies for legal correctness, structural completeness, and three local cross-step consistency transitions—jointly optimizes the legal policy via entropy- and logic-aware token-level advantage reweighting and adaptive clipping.

Method

Overall Architecture

The input to JPO is a normalized criminal case fact sequence \(F = \{f_1, f_2, \dots, f_n\}\). The learning objective is to infer both the final judicial outcome \(Y = (A, C, S)\) (comprising statutory articles \(A\), criminal charges \(C\), and a continuous normalized sentence length \(S\)) and an explicit four-stage legal reasoning trace \(Z = (G_f, G_a, G_c, G_s)\), which maps directly onto fact extraction, statutory analysis, charge determination, and sentence prediction along the canonical adjudicative path \(F \to A \to C \to S\). The overall framework operates in two sequential stages: Stage I applies supervised fine-tuning (SFT) using structured rationales generated by an instruction-following teacher model (Qwen2.5-72B-Instruct) under a forward-constrained prompt to establish a stable and interpretable reasoning schema. Stage II applies reinforcement learning (PPO) against a frozen reference policy \(\pi_{\theta_{\text{ref}}}\), optimizing the student policy \(\pi_\theta\) using a composite multi-dimensional reward function combined with fine-grained token-level advantage reweighting and token-dependent adaptive clipping.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Input Criminal Case Facts F"] --> B["Stage 1: Teacher-Guided Structured SFT<br/>Standardized Reasoning Chain F → A → C → S"]
    B --> C["Stage 2: Multi-Dimensional Composite Reward RL<br/>Jointly optimizes legal accuracy, completeness & cross-step consistency"]
    C --> D["Stage 3: Token-Level Advantage Reweighting<br/>Combines normalized entropy Ht with legal logic weight Lt"]
    D --> E["Stage 4: Adaptive Policy Gradient Clipping<br/>Dynamic exploration threshold ϵt scaled by token importance"]
    E --> F["Output Final Judicial Decision (A, C, S) & Trace Z"]

Key Designs

1. Teacher-Guided Structured SFT: Anchoring Forward-Causal Reasoning Scaffolds

To overcome the pervasive lack of intermediate reasoning annotations in public judicial datasets, JPO leverages a strong frontier teacher model to generate structured four-step rationales. To eliminate the risk of backward, post-hoc rationalization—where a model sees the gold outcome first and manufactures superficial justifications—the teacher prompt explicitly enforces forward-causal generation: extracting core fact elements (time, location, perpetrators, criminal actions, harm), mapping those elements directly to the constituent elements of penal code articles, deducing the primary charge from the statutory elements, and finally calculating baseline sentences and discretionary mitigating/aggravating factors to arrive at the final term. The student model minimizes standard negative log-likelihood:

\[\mathcal{L}_{\text{SFT}} = - \log P_\theta(Z, Y \mid F)\]

This supervised stage teaches the base language model the required XML/JSON response schemas and seeds the model with an interpretable, structurally aligned reasoning scaffold before entering reinforcement learning.

2. Multi-Dimensional Composite Reward RL: Aligning Final Outcomes with Intermediate Adjudication Chains

To prevent models from sacrificing legal reasoning integrity for final label gaming, JPO structures its sequence-level reward into three foundational pillars: \(R = \alpha R_{\text{legal}} + \beta R_{\text{structure}} + \gamma R_{\text{consistency}}\). The legal accuracy reward \(R_{\text{legal}}\) balances format compliance, set-based precision-recall products for article prediction (\(R_{\text{article}} = \frac{|\hat{A} \cap A|}{|\hat{A}|} \cdot \frac{|\hat{A} \cap A|}{|A|}\)) and charge determination (\(R_{\text{charge}} = \frac{|\hat{C} \cap C|}{|\hat{C}|} \cdot \frac{|\hat{C} \cap C|}{|C|}\)), along with an exponential relative-error term for continuous prison sentencing \(R_{\text{sentence}} = \exp\left(-\xi \frac{|\hat{S} - S|}{S}\right)\). The structural reward \(R_{\text{structure}}\) guarantees segment presence and applies a logarithmic length-penalty to suppress degenerative empty sections.

Crucially, the cross-step consistency reward \(R_{\text{consistency}} = \frac{1}{3}(S_{\text{FA}} + S_{\text{AC}} + S_{\text{CS}})\) provides closed-form, computationally efficient proxies for each local transition: - Fact-to-Article Consistency (\(S_{\text{FA}}\)): Evaluated using a corpus-trained Naive Bayes article estimator \(P(a \mid F)\), scoring the posterior probability that predicted articles are genuinely supported by factual lexical cues: \(S_{\text{FA}} = \frac{1}{|\hat{A}|} \sum_{a \in \hat{A}} P(a \mid F)\). - Article-to-Charge Consistency (\(S_{\text{AC}}\)): Computed via a domain-derived rule-based association matrix \(D \in \mathbb{R}^{|A| \times |C|}\) (assigning 2 for direct statutory codification, 1 for relevant secondary relations, and 0 otherwise), measuring normalized compatibility between \(\hat{A}\) and \(\hat{C}\). - Charge-to-Sentence Consistency (\(S_{\text{CS}}\)): Estimated by modeling empirical sentence distributions per charge as truncated Gaussians \(\mathcal{N}(\mu_c, \sigma_c^2)\) on \([a_c, b_c]\) to assess the relative statistical density \(p(\hat{S} \mid c)\) against the mode. These closed-form proxies eliminate LLM-judge latency while demonstrating strong rank correlation with human legal experts.

3. Token-Level Advantage Reweighting: Mitigating Credit Assignment Failure in Long Legal Rationales

In legal adjudication, sequence-level scalar rewards suffer from severe credit assignment ambiguity: a catastrophic legal deduction error may stem from a single misapplied legal term in early stages, while a correct legal step might be unfairly penalized due to a minor grammatical error at the sequence tail. JPO resolves this by reweighting the sequence advantage \(A_{\text{seq}}\) for each token \(t\) using an importance metric \(\lambda_t \in [0, 1]\):

\[A_t = A_{\text{seq}} \cdot f(\lambda_t, \text{sign}(A_{\text{seq}}), \psi), \quad \text{where } \lambda_t = \zeta \bar{H}_t + (1 - \zeta) L_t\]

Here, \(\bar{H}_t = H_t / \max_{\tau} H_\tau\) denotes vocabulary decoding entropy normalized across the sequence, reflecting model epistemic uncertainty. The legal logic weight \(L_t\) is segment-aware: tokens belonging to statutory analysis \(G_a\) inherit \(S_{\text{FA}}\), tokens in charge determination \(G_c\) inherit \(S_{\text{AC}}\), tokens in sentencing \(G_s\) inherit \(S_{\text{CS}}\), and others take their global average. When \(A_{\text{seq}} > 0\), tokens with above-average importance (\(\lambda_t > \bar{\lambda}\)) receive boosted updates via \(1 + \psi(\lambda_t - \bar{\lambda})\); when \(A_{\text{seq}} < 0\), the same legally salient tokens are penalized more conservatively via \(1 - \psi(\lambda_t - \bar{\lambda})\), effectively shielding foundational legal deductions from over-penalization.

4. Adaptive Policy Gradient Clipping: Breaking Uniform Constraints on Core Judicial Decision Tokens

Standard PPO enforces a rigid, uniform clipping boundary \(\epsilon\) across all token positions, which artificially restrains policy updates on critical decision tokens that carry decisive judicial weight while allowing excessive parameter drift on generic functional syntax. JPO introduces token-dependent dynamic clipping:

\[\epsilon_t = \epsilon \cdot f(\lambda_t, \text{sign}(A_{\text{seq}}), \delta)\]

This mechanism expands the policy exploration corridor for tokens bearing high legal logic relevance and high informational entropy, facilitating decisive updates on key statutory citations, charge designations, and sentencing calculations while strictly preserving conservative updates across repetitive procedural transitions.

Loss & Training

The Stage II optimization objective merges token-reweighted advantages, adaptive clipping envelopes, and KL divergence regularization against the reference policy \(\pi_{\theta_{\text{ref}}}\):

\[\mathcal{L}_{\text{RL}} = \mathbb{E}_t \left[ \min\left( r_t A_t, \, \text{clip}(r_t, 1 - \epsilon_t, 1 + \epsilon_t) A_t \right) - \beta_{\text{KL}} D_{\text{KL}}\big(\pi_\theta(a_t \mid s_t) \parallel \pi_{\theta_{\text{ref}}}(a_t \mid s_t)\big) \right]\]

Hyper-parameter settings are tuned via grid sweeps: reward weights are fixed at \(\alpha = 0.75\), \(\beta = 0.0625\), and \(\gamma = 0.1875\); advantage scaling parameter \(\psi = 0.6\); entropy-logic mixing coefficient \(\zeta = 0.5\); base clipping ratio \(\epsilon = 0.2\); and KL penalty weight \(\beta_{\text{KL}} = 10^{-3}\). Models are trained for 4 epochs with an RL learning rate of \(1 \times 10^{-6}\), an overall batch size of 1024, PPO mini-batch size of 256, and group size 5.

Key Experimental Results

Main Results

JPO is evaluated across five open-source LLM backbones on three legal benchmarks: JPO-Dataset (newly collected from 2024–2026 Chinese judgments; 239.5K SFT train, 9.6K RL train, 20.4K test), CAIL2018 (30K test cases), and LawBench (1.5K test cases). Evaluated metrics include Article Macro-F1, Charge Macro-F1, continuous Sentence Score (\(\xi=3\)), 4-Step Structural Completeness, and Full-Chain Consistency. The main results across representative models are presented below:

Backbone & Method JPO-Dataset Art. F1 JPO-Dataset Charge F1 JPO-Dataset Sent. Score JPO-Dataset 4-Step Comp. JPO-Dataset Full-Chain CAIL2018 Art. F1 CAIL2018 Charge F1 CAIL2018 Sent. Score LawBench Art. F1 LawBench Charge F1 LawBench Sent. Score
Qwen3-4B-Instruct
Pre-trained 0.521 0.468 0.174 0.315 0.208 0.485 0.442 0.158 0.463 0.419 0.142
SFT 0.884 0.858 0.405 0.902 0.652 0.856 0.831 0.381 0.834 0.809 0.364
JPO (Ours) 0.931 0.916 0.542 0.966 0.789 0.903 0.888 0.514 0.872 0.861 0.485
Qwen2.5-7B-Instruct
Pre-trained 0.618 0.573 0.211 0.442 0.267 0.584 0.539 0.194 0.551 0.508 0.182
SFT 0.893 0.871 0.422 0.918 0.681 0.865 0.842 0.399 0.841 0.824 0.377
JPO (Ours) 0.937 0.928 0.551 0.973 0.806 0.908 0.897 0.521 0.879 0.868 0.493
Llama-3-8B-Instruct
Pre-trained 0.594 0.552 0.203 0.418 0.252 0.561 0.524 0.187 0.537 0.495 0.175
SFT 0.871 0.847 0.396 0.897 0.641 0.846 0.822 0.375 0.823 0.798 0.354
JPO (Ours) 0.926 0.912 0.534 0.962 0.781 0.897 0.884 0.505 0.868 0.856 0.477
Qwen2.5-3B-Instruct
Pre-trained 0.451 0.382 0.114 0.284 0.198 0.426 0.359 0.103 0.402 0.334 0.092
RL-only 0.782 0.744 0.295 0.671 0.482 0.755 0.718 0.273 0.732 0.695 0.258
SFT 0.873 0.847 0.391 0.895 0.622 0.848 0.823 0.372 0.825 0.801 0.349
Vanilla PPO 0.896 0.869 0.454 0.911 0.678 0.871 0.845 0.428 0.846 0.822 0.405
LegalΔ 0.904 0.881 0.479 0.924 0.703 0.878 0.856 0.451 0.854 0.834 0.427
Issue Tree Rubrics 0.911 0.896 0.488 0.932 0.721 0.885 0.871 0.459 0.862 0.848 0.435
JPO (Ours) 0.929 0.921 0.536 0.967 0.791 0.904 0.895 0.505 0.877 0.868 0.479

When compared against established SOTA domain-specific architectures on CAIL2018 (using standard per-label averaging), JPO (Qwen2.5-7B) establishes new benchmark records with 90.8 Article F1 and 89.7 Charge F1, outstripping the prior leading multi-agent committee system VERDICT (83.3 / 82.4) by +7.5 and +7.3 points respectively. Even the lightweight JPO (Qwen2.5-3B) achieves 90.4 Article F1 and 89.5 Charge F1, comfortably outperforming complex multi-agent frameworks.

Ablation Study

The modular components of JPO were dissected on Qwen2.5-3B-Instruct using JPO-Dataset:

Configuration / Variant Art. Macro-F1 Charge Macro-F1 Sent. Score 4-Step Comp. Full-Chain Findings & Impact
Full JPO Framework 0.929 0.921 0.536 0.967 0.791 Optimal balanced performance across all axes
w/o \(R_{\text{structure}}\) 0.916 0.909 0.512 0.861 0.755 4-step completeness plummets by -10.6%
w/o \(R_{\text{consistency}}\) 0.918 0.908 0.481 0.958 0.682 Sentence score drops -0.055; chain consistency drops -0.109
w/o token-level advantage 0.922 0.911 0.519 0.961 0.768 Degrades credit assignment across long traces
w/o adaptive clipping 0.924 0.913 0.522 0.963 0.773 Rigid boundaries restrict critical decision exploration
w/ uniform token weights 0.919 0.908 0.501 0.956 0.747 Uniform weighting fails to prioritize core legal arguments
w/o \(S_{\text{FA}}\) (Fact-to-Article) 0.923 0.915 0.520 - 0.768 Compromises statutory grounding in facts
w/o \(S_{\text{AC}}\) (Article-to-Charge) 0.920 0.908 0.513 - 0.755 Largest drop in Charge F1 due to severed statutory link
w/o \(S_{\text{CS}}\) (Charge-to-Sentence) 0.925 0.916 0.497 - 0.763 Sentence score collapses to 0.497
Entropy-only weighting (\(\zeta=1.0\)) 0.923 0.912 0.523 - 0.769 Ignores structured adjudicative importance
Logic-only weighting (\(\zeta=0.0\)) 0.924 0.914 0.525 - 0.771 Ignores internal model prediction uncertainty
w/o KL regularization 0.925 0.915 0.515 - 0.741 Unchecked policy drift harms generation stability

Key Findings

  • Staging Synergy: Applying reinforcement learning directly from pre-trained weights without structured SFT (RL-only) leads to severe performance degradation (Article F1 0.782, Charge F1 0.744, Full-Chain 0.482 on Qwen2.5-3B). SFT provides the critical foundation for structuring output arguments, while Stage II RL is essential for refining cross-step legal calibration.
  • Sentencing Calibration is Driven by Consistency Proxies: Removing the Charge-to-Sentence consistency term \(S_{\text{CS}}\) induces the single largest decline in sentence scoring (from 0.536 down to 0.497). SFT alone only acquires coarse statutory categories, whereas policy optimization with empirical statistical priors accurately tunes the discretionary margin.
  • Empirical Validity of Computable Proxies: In a blind study involving three legal professionals rating 800 test cases, the automatic consistency proxies demonstrated significant rank correlation with human legal experts (Spearman \(\rho = 0.67\) for \(S_{\text{FA}}\), \(\rho = 0.71\) for \(S_{\text{AC}}\), \(\rho = 0.64\) for \(S_{\text{CS}}\), and \(\rho = 0.72\) for aggregate Full-Chain). Under a format-agnostic GPT-5.2 judge evaluation that made no reference to the four-step schema, JPO achieved a 74% win rate over SFT (improving average rationale quality from 3.14 to 3.90 out of 5.0).
  • Causal Faithfulness under Counterfactual Intervention: Across 500 test cases where decisive criminal elements (such as injury severity, embezzlement sums, or narcotics weights) were counterfactually altered by legal annotators, JPO updated its intermediate reasoning and judicial outcomes in the legally expected direction in 83% of cases, substantially outperforming SFT (61%) and pre-trained baselines (43%).

Highlights & Insights

  • Closed-Form Surrogates for Intractable Legal Coherence: Instead of relying on slow, uncalibrated LLM-as-a-judge feedback during training, JPO grounds consistency in lightweight, analytically tractable proxies (Naive Bayes posteriors, discrete statutory incidence matrices, and truncated Gaussian sentencing distributions). This ensures rapid, stable policy optimization (3B training requires only 53 GPU-hours) that correlates strongly with professional legal consensus.
  • Dual-Perspective Token Advantage Shaping: By intersecting intrinsic model uncertainty (normalized entropy) with extrinsic adjudicative domain knowledge (stage-level consistency metrics), JPO effectively resolves the credit assignment dilemma in long-chain legal generation, penalizing flawed intermediate steps without corrupting sound statutory analysis.
  • Compact Models Surpassing Massive Proprietary APIs: With domain-grounded post-training, the 3B-parameter JPO model outperforms zero-shot commercial frontier models (including DeepSeek-V3.2, GPT-5.2, and Claude-Sonnet-4.5) across article citation, charge identification, and sentence prediction, while reducing inference costs by nearly two orders of magnitude ($0.16 vs. $9.70 per 1,000 cases).

Limitations & Future Work

  • Single-Defendant Focus and Multi-Party Attribution Gaps: JPO is primarily tuned for single-defendant criminal proceedings. When tested out-of-distribution on held-out multi-charge (1,000 cases) and multi-defendant (800 cases) scenarios without retraining, performance drops sharply (e.g., sentence score falls to 0.351 on multi-defendant cases) due to lack of explicit per-actor liability disentanglement.
  • Discretionary Nuances and Secondary Article Omission: Error analyses reveal that 27.5% of remaining failures involve missing secondary statutory articles, 25.3% stem from confusion between adjacent property offenses (e.g., fraud vs. contract fraud), and 16.7% reflect incomplete balance of discretionary sentencing factors (e.g., voluntary surrender vs. partial restitution).
  • Substantive vs. Procedural Law Scope: The current pipeline models substantive penal adjudications. Adapting it to civil dispute resolutions, evidentiary admissibility hearings, or procedural motions would require restructuring the transition graph beyond the four-stage framework.
  • vs VERDICT (Liao et al., 2026): While VERDICT deploys an asynchronous multi-agent collegial bench (defense, prosecution, and judges) to iteratively converge on decisions, JPO achieves higher accuracy (CAIL2018 F1 90.8/89.7 vs. 83.3/82.4) within an efficient single-model forward pass by internalizing legal dependency constraints into policy gradient rewards.
  • vs LegalΔ (Dai et al., 2025a) & Issue Tree Rubrics (Cai et al., 2025): Prior RL approaches optimize general CoT information gain or rely on unconstrained LLM tree-search rubrics. JPO replaces subjective reward models with structurally grounded, closed-form consistency penalties derived directly from statutory criminal doctrine.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Formulates legal judgment prediction as a closed-form multi-stage dependency chain, coupled with logic- and entropy-guided token-level policy optimization.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorous evaluation across 3 benchmarks, 5 base LLMs, multi-annotator blind expert trials (κ = 0.73), counterfactual causal interventions, and exhaustive ablations.
  • Writing Quality: ⭐⭐⭐⭐⭐ Clear mathematical formulation, comprehensive error taxonomy, and deep contextualization within both legal theory and NLP post-training.
  • Value: ⭐⭐⭐⭐⭐ Provides a replicable, cost-effective blueprint for aligning language models to high-stakes, logically constrained professional domains.