FRAME 基准：如何在保留旧格式的同时生成新报告
Frame: Form-Restricted Authoring with Model-Based Evaluation
想给公司写季度报告或学术综述的可以看看，这论文把“照着旧版格式填新内容”做成了 120 道题的基准，还测出了开源模型和 GPT 这类闭源模型的差距在哪。
arXiv 论文提出 FRAME 基准，形式化定义了表单受限报告生成任务：给定样例报告和新的依据文档，生成保留样例格式并准确反映新证据的报告。基准包含 120 个来自 EDGAR 和 arXiv 的实例，覆盖 Financial Quarterly、Financial Annual、Academic Simple、Academic Complex 四类，采用三快照时间管线构建。评估使用三个经人工验证的指标：语义树编辑距离衡量结构相似性、信息单元召回衡量覆盖率、claim 级 grounding 验证衡量忠实度。在五个 LLM 的测试中，开源模型在结构上接近专有模型，但在覆盖率和忠实度上落后；所有模型在金融报告上的表现好于学术报告，输出越长、依据越复杂，性能越差。
Frame: Form-Restricted Authoring with Model-Based Evaluation
Many professional reports reuse the structure of prior editions while updating the content. We formalize this setting as form-restricted report generation: given a sample report and new grounding documents, generate a report that preserves the sample's form while accurately reflecting the new evidence. We introduce FRAME, a benchmark of 120 EDGAR- and arXiv-derived instances across Financial Quarterly, Financial Annual, Academic Simple, and Academic Complex categories, built with a temporal three-snapshot pipeline. We evaluate outputs with three human-validated metrics: structural similarity via semantic tree-edit distance, coverage via information unit recall, and faithfulness via claim-level grounding verification. Across five LLMs, open-source models nearly match proprietary models on structure but lag on coverage and faithfulness; all models perform better on financial than academic reports, and performance declines with longer outputs and more complex grounding.