Skip to content

Benchmarking Patent Drafting from Inventor-Style Disclosures

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/scylj1/Patent-MAF
Area: LLM Reasoning
Keywords: Patent Drafting, Multi-Agent Collaboration, Invention Disclosure, Multimodal Reasoning, Local Deployment

TL;DR

Addressing the disconnect between existing patent benchmarks and real-world drafting practices, this paper establishes Dis2Pat, the first multimodal disclosure-to-patent benchmark from informal inventor-style inputs, and introduces Patent-MAF, a fully locally deployable multi-agent framework that outperforms open-source baselines and rivals state-of-the-art closed-source models in generating legally coherent claims and detailed specifications.

Background & Motivation

Patent drafting is an indispensable yet financially burdensome process that translates an inventor's raw technical ideas into legally enforceable intellectual property documents comprising both claims and a detailed specification. In recent years, large language models (LLMs) have achieved notable milestones across isolated patent tasks. However, a profound mismatch persists between existing academic benchmarks and authentic patenting workflows: the vast majority of prior datasets (such as HUPD-DCG, EPD, PatentFormer, and AutoSpec) assume inputs from later or heavily formalized stages of prosecution. They rely on inputs that are already structured and legally loaded—either deducing claims from fully drafted specifications or generating specifications solely conditioned on pre-existing claims. Consequently, current models circumvent the most demanding bottleneck encountered by patent attorneys: translating informal, unstructured invention disclosures authored by engineers and inventors into legally coherent applications.

Real patent drafting inherently originates from early-stage, de-legalized invention disclosures. These working materials articulate technical problems, core insights, preliminary implementation steps, and accompanying schematic drawings in everyday engineering prose, entirely lacking statutory claim scopes, hierarchical dependencies, or formal specification organization. When standard LLMs are tasked with end-to-end drafting from such informal materials, they struggle to balance multimodal spatial grounding, cross-section terminology consistency, and rigorous legal drafting conventions. Compounding this challenge, invention disclosures contain strictly confidential trade secrets before official publication; enterprise intellectual property departments and law firms categorically prohibit uploading unexamined inventions to proprietary, cloud-hosted model APIs. Thus, any viable drafting assistant must operate reliably on locally deployed open-source architectures.

To bridge this critical operational gap, this paper pioneers a holistic benchmark and a privacy-preserving system for real-world disclosure-to-patent workflows. Core idea: Construct Dis2Pat, the first multimodal benchmark requiring end-to-end patent drafting directly from de-legalized inventor disclosures and drawings, and develop Patent-MAF, a locally deployable multi-agent framework that coordinates structured disclosure routing, specialized drafting, and cross-section consistency polishing to achieve legally robust, privacy-compliant patent generation.

Method

Overall Architecture

Patent-MAF is engineered to mirror the collaborative division of labor practiced in professional patent prosecution. The overall input is an invention disclosure \(D = (T, F)\), where \(T\) represents the de-legalized textual description and \(F\) denotes the set of accompanying technical drawings; the objective is to generate a comprehensive patent application draft \(P = (C, S)\) consisting of a claim set \(C\) and a complete specification \(S\). The architecture coordinates three specialized agent roles: a Manager Agent deconstructs and routes unstructured disclosures into targeted intermediate drafting artifacts; specialized Drafter Agents synthesize independent/dependent claims and descriptive embodiments in parallel; and a Polisher Agent performs cross-section terminology alignment, specification enablement checks, and legal style normalization.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Invention Disclosure & Figures<br/>D = (T, F)"] --> B["Disclosure Structuring & Routing<br/>Manager Agent decomposes D into MC and MS"]
    B --> C["Specialized Drafting & Multimodal Grounding<br/>Fine-tuned Claim Drafter + Vision Specification Drafter"]
    C --> D["Global Consistency & Style Polishing<br/>Polisher Agent eliminates cross-section discrepancies"]
    D --> E["Legally Structured Patent Application<br/>Claims C + Detailed Specification S"]

Key Designs

1. Disclosure Structuring & Routing: Bridging Unstructured Engineering Notes and Legal Drafting Artifacts

Raw inventor disclosures naturally entangle problem background, algorithmic ideas, and low-level implementations without an explicit legal hierarchy. Acting as the central orchestrator, the Manager Agent leverages an open-source Qwen3-32B backbone with explicit JSON-schema prompting to parse raw disclosure \(D\) into two tailored drafting artifacts \(M = (M_C, M_S)\). Specifically, the claim drafting material \(M_C\) extracts scope-defining features, fundamental novelty points, and essential structural limitations while stripping irrelevant background prose. In parallel, the specification drafting material \(M_S\) compiles comprehensive technical background, functional relationships, detailed embodiment steps, and explicit reference links to figure components. This modular parsing decouples long-context generation, preventing downstream models from overlooking technical essentials during long-form synthesis.

2. Specialized Drafting & Multimodal Grounding: Decoupling Statutory Claim Rigor from Multimodal Embodiment Realization

Patent claims and descriptive specifications diverge fundamentally in formatting, length, and legal constraints, rendering single-shot generation suboptimal. Patent-MAF assigns content creation to two distinct agents tailored to their respective modalities: - Claim Drafter Agent: Operating exclusively on \(M_C\), this agent employs a parameter-efficient LoRA fine-tuned Qwen3-32B to generate at least one independent claim and multiple dependent claims. Fine-tuning compels the model to strictly adhere to statutory standards (e.g., proper preamble, "comprising" transitional phrases, precise antecedent basis, and non-fatal definiteness). Multimodal input is intentionally bypassed here because claims formulate an abstract legal monopoly rather than concrete physical geometries. - Specification Drafter Agent: Drafts the complete narrative specification \(S\) (Background, Summary, and Detailed Description of Embodiments) from \(M_S\) and figures \(F\) using a multimodal Qwen3-VL-32B backbone. Vision capabilities ground mechanical configurations, pipeline connections, and part numbering directly from the drawings, resolving structural ambiguities that pure text cannot disambiguate.

3. Global Consistency & Style Polishing: Resolving Cross-Section Terminology Drift and Enablement Mismatches

Independent execution between the claim and specification drafters inevitably introduces cross-section discrepancies: the claim drafter may recite a generic term (e.g., "drive mechanism") while the specification drafter refers specifically to a "servo actuator", or claims might introduce limitations lacking descriptive support in the specification (violating statutory enablement rules). The Polisher Agent, powered by Qwen3-32B, functions as a senior partner reviewer by executing three non-generative refinement passes: unifying terminology across both sections, verifying bidirectional claim-specification support, and enforcing formal legal style while pruning informal conversational language. Crucially, the polisher is forbidden from hallucinating new technical mechanisms, operating strictly to harmonize and refine existing drafts.

Benchmark & Evaluation Protocol

To establish reliable, privacy-preserving research, the authors built Dis2Pat, comprising 9,433 granted patents from Google Patents and PatentDesc (split into 8,490 training and 943 test samples, partitioned without patent family overlap). Because genuine pre-filing disclosures are protected by non-disclosure agreements, pseudo-disclosures were extracted via GPT-5-mini under strict constraints, reorganizing technical disclosures into 7 non-legal, inventor-centric categories (reducing average document length from 11,207 to 1,196 tokens and technical term density from 0.05 to 0.001). Licensed patent professionals validated 100 sample disclosures, confirming high fidelity (Hallucination 9.8/10, Contradiction 10.0/10). All fine-tuning and inference experiments were conducted on NVIDIA A100 GPUs (~700 GPU hours total) with decoding temperature 0.3. Evaluation combines lexical overlap (BLEU, ROUGE), semantic similarity (BERTScore, BERT-for-Patent), LLM-as-a-judge scoring via DeepSeek-V3 with Chain-of-Thought (covering Completeness, Clarity, and Legality), and blind pairwise comparisons by licensed patent attorneys on 60 test cases.

Key Experimental Results

Main Results

On the Dis2Pat benchmark, comparative performance across open-source and proprietary models is reported in Table 1 and Table 2.

Table 1: Model performance on patent specification generation (from paper Table 3)

Model / Setting Modality BLEU ROUGE-1 ROUGE-L BERTScore BERT-for-Patent Completeness Clarity Style Overall
Qwen3-32B Text-Only 0.8 28.8 12.1 81.5 94.4 82.8 81.0 72.9 78.9
Qwen3-VL-32B Text-Only 0.7 30.3 12.3 82.0 94.6 83.3 85.9 73.7 81.0
Qwen3-VL-32B Text+Image 0.8 30.5 12.5 82.0 94.7 83.6 86.1 74.3 81.3
Qwen3-VL-72B Text+Image 1.8 32.4 12.9 81.6 94.7 84.8 83.6 77.1 81.8
LLaMA-3.2-90B-Vision Text+Image 0.6 27.7 12.1 81.8 94.6 68.9 73.9 58.6 67.1
GPT-4o Text+Image 0.4 13.4 6.8 81.9 89.2 75.5 79.6 65.9 73.7
GPT-5 Text+Image 2.3 34.8 13.9 81.8 96.4 84.7 88.2 82.3 85.1
Patent-MAF (Ours) Text+Image 2.1 32.5 13.8 82.3 96.1 85.6 88.5 82.0 85.4

Table 2: Model performance on patent claim generation (from paper Table 4)

Model / Setting Modality BLEU ROUGE-1 ROUGE-L BERTScore BERT-for-Patent Scope Clarity Legality Overall
Qwen3-32B Text-Only 13.5 48.0 25.3 83.2 94.5 82.8 81.2 78.6 80.9
Qwen3-32B-FT Text-Only 19.1 54.3 30.4 87.3 96.1 83.6 85.2 81.2 84.4
Qwen3-VL-72B Text+Image 12.4 50.1 25.2 83.7 94.2 82.9 80.0 76.5 79.8
LLaMA-3.2-90B-Vision Text+Image 9.1 43.3 23.2 82.3 93.5 78.0 78.1 78.9 78.3
GPT-4o Text-Only 1.0 24.9 15.3 82.4 86.6 67.8 72.7 71.6 70.7
GPT-5 Text-Only 19.0 53.8 26.8 83.8 95.9 85.8 95.1 92.3 91.1
Patent-MAF (Ours) Text-Only (Claims) 21.1 55.2 32.3 88.3 96.4 84.6 90.2 85.2 86.7

Ablation Study

The contribution of each design component was scrutinized through systematic ablations (Table 3 and Table 4):

Configuration Variant Spec Overall Claim Overall Key Metric Changes & Primary Observations
Patent-MAF (Full System) 85.4 86.7 BERTScore: 82.3 / 88.3; BERT-for-Patent: 96.1 / 96.4
w/o Manager Agent 83.4 (-2.0) 84.2 (-2.5) Unstructured inputs harm claim scope (83.8) and spec completeness (85.0)
w/o Vision (for Spec Drafter) 83.4 (-2.0) - BERT-for-Patent drops sharply to 93.6; spatial component layout errors surge
w/o Fine-tuning (for Claim Drafter) - 81.6 (-5.1) Legality collapses from 85.2 to 79.1; BLEU drops to 13.2 without domain tuning
w/o Polisher Agent 84.4 (-1.0) 80.1 (-6.6) Catastrophic drop in claim score; cross-section terminology mismatches persist

Key Findings

  • Multi-Agent Orchestration Outperforms Pure Model Scaling: Built on a 32B open-source parameter foundation, Patent-MAF achieves 85.4 on specification generation, eclipsing closed-source GPT-5 (85.1) and far exceeding LLaMA-3.2-90B (67.1). On claims, Patent-MAF scores 86.7, outperforming Qwen3-32B-FT (84.4) and Qwen3-VL-72B (79.8).
  • Asymmetric Modality Demands in Patent Drafting: Visual figure inputs provide tangible gains for specification drafting (+2.5 BERT-for-Patent), grounding mechanical structures and part connectivity. Conversely, providing figures to claim drafters degrades performance (GPT-5 drops from 91.1 to 90.7; Qwen3-VL-32B drops from 78.9 to 78.6), as claims encode abstract statutory scopes where concrete visual features introduce restrictive biases.
  • Polishing Is the Crucial Legal Safeguard: Removing the Polisher Agent precipitates a 6.6-point drop in claim overall quality. Expert attorney annotations reveal that independent drafting agents frequently produce conflicting terms, which directly compromises patent validity unless harmonized by global polishing.

Highlights & Insights

  • Scalable De-Legalization Methodology: Rather than exposing confidential inventions or using synthetic toy examples, the authors reverse-engineered granted patents into plain, non-legal inventor notes, preserving complete technical truth while eliminating legal phrasing.
  • Privacy-Preserving Local IP Infrastructure: By proving that coordinated 32B open-source models can match proprietary frontier models, Patent-MAF provides a deployable template for law firms and corporate R&D units bound by strict secrecy.
  • Rigorous Human-in-the-Loop Validation: Evaluation combines automated metrics with blinded pairwise comparisons by licensed patent attorneys, establishing high inter-annotator agreement (\(\kappa = 0.79\) on completeness) and proving that multi-agent drafting matches human prosecution standards.

Limitations & Future Work

  • Information-Complete Disclosure Assumption: Dis2Pat pseudo-disclosures assume that the inventor has provided complete technical information. Real inventor notes are notoriously incomplete, requiring future systems to integrate interactive interview and elicitation mechanisms.
  • Jurisdiction-Specific Generalization: The framework is exclusively tuned to USPTO standards in English. Expanding to the European Patent Office (EPO) or CNIPA requires modeling two-part claim formats and stricter added-matter doctrines.
  • Computational Overhead: Coordinating multiple intermediate reasoning agents increases total token expenditure by approximately 2.5x relative to direct single-shot prompting, introducing higher inference latency.
  • vs. HUPD-DCG / EPD / PatentFormer: Prior datasets isolate narrow downstream tasks (specification-to-claim or claim-to-specification) on already formalized legal texts; Dis2Pat tackles the holistic, unassisted transition from raw engineering disclosures to full filings.
  • vs. Draft2Patent / Pap2Pat: Earlier end-to-end setups utilized structured Q&A answers or formal scientific papers. Dis2Pat handles authentic non-legal inventor language and systematically evaluates the impact of multimodal figures.
  • Takeaways for Complex LLM Reasoning: For long-form generation subject to dual constraints (technical factuality and statutory compliance), monolithic prompting frequently degenerates. Decomposing the pipeline into structured routing, modular generation, and global consistency audits offers a robust, reproducible blueprint for high-stakes enterprise AI.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Establishes the first authentic disclosure-to-patent multimodal benchmark, addressing an open challenge in legal NLP.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Thorough evaluation across diverse open-source and proprietary backbones, ablation models, and licensed patent attorneys.
  • Writing Quality: ⭐⭐⭐⭐⭐ Cohesive prose, disciplined structure, detailed experimental transparency, and precise legal terminology.
  • Value: ⭐⭐⭐⭐⭐ High industrial and intellectual property utility, establishing an open-source, privacy-first baseline for legal text automation.