PatentAgentBench：用专业专利任务评测 LLM 智能体
PatentAgentBench: Benchmarking LLM Agents with Professional Patent Task Suites
这篇把专利检索、撰写、答复审查意见拆成 510 个真实任务来考 7 个大模型，想看 agent 到底能不能干专业活的可以看看差距在哪。
论文提出 PatentAgentBench，包含 510 个实例、覆盖 8 类专利任务：novelty search、FTO search、无效检索、权利要求对比表、专利撰写、审查意见答复和翻译等。数据来自真实专利记录，配有专家审核的标注和评分标准。检索类任务要求智能体自行搜索和比较专利集合，不提供候选文档池。评测涵盖 7 个 LLM 和 4 种 agent 框架，结果显示当前系统在识别新颖性、评估风险和理解从业者意图上仍有明显差距。
PatentAgentBench: Benchmarking LLM Agents with Professional Patent Task Suites
Patent practice lies at the intersection of technical, legal, and linguistic work, making it challenging yet crucial to technological innovation. Recently, rapid advances in large language models (LLMs) and agentic systems have demonstrated the potential to complete realistic business workflows and even patent works. This raises a question: can current AI systems perform patent work reliably? Existing benchmarks cover a limited range of patent tasks and simplify evidence acquisition by providing reference materials or candidate documents. In this paper, we introduce PatentAgentBench, a benchmark comprising 510 instances across eight task suites: novelty search, freedom-to-operate (FTO) search, design FTO, invalidity search, claim charting, patent drafting, office-action response, and patent translation. Curated from real patent records and professional materials, the benchmark combines task-specific metrics with expert-reviewed annotations and rubrics. Retrieval tasks such novelty search, FTO, etc., require agents to search and compare patent collections without a supplied candidate pool. The drafting and OAR tasks assess the quality of agents' work given technical disclosure and office action notice. We evaluate seven LLMs across four agent harnesses on PatentAgentBench, finding that current systems still struggle to identify novelty, assess risk, and understand practitioner intent throughout patent workflows.