论文多源确认

MechAlign:用 LLM 审查 Lean 数学形式化的项目级语义对齐

MechAlign: Evaluating Semantic Alignment at the Project Level in Mathematical Formalization

精选理由

有人给 Lean 形式化做了个语义对齐检查器,用依赖图让 LLM 找形式化定理和论文原文的出入,还放了个 501 条的基准。

Lean 只能验证形式化证明在其声明假设下成立,无法判断命题是否忠实表达了论文中的数学断言。MechAlign 框架结合其提取工具 LeanTrace 构建语句依赖图,再由 LLM 在该上下文中检查对齐并给出带定位证据的差异报告。团队对来自 173 个 OpenAI Math 项目的 201 个主要结果做了初步审查,并以双曲群案例展示了依赖检查如何定位缺失的维度约束。论文同时发布了 LeanReviewBench,包含来自 OpenAI Math、Fermat's Last Theorem 和 Kakeya 的 501 个固定 claim-statement-context 三元组。

原文 · arXiv: OpenAI

MechAlign: Evaluating Semantic Alignment at the Project Level in Mathematical Formalization

Lean verifies that a formal proof establishes its stated proposition under declared assumptions, but does not check whether that proposition faithfully expresses the mathematical claim in a manuscript. Assessing this correspondence requires interpreting the main theorem through its project dependencies, which may be scattered throughout a large development. We introduce MechAlign, a framework for project-level semantic alignment review that combines formal dependency extraction with LLM-based assessment. Its extraction tool, LeanTrace, constructs a statement dependency graph from elaborated declarations, retaining definition bodies and structure constraints while excluding proof bodies and stopping at an explicit library boundary. The reviewer uses this context to investigate alignment and report discrepancies with localized evidence. We conduct a preliminary review of 201 main results from 173 OpenAI Math projects, using supplied informal descriptions and available paper context. A hyperbolic-group case illustrates how dependency inspection localizes a missing dimensional constraint relative to the manuscript, consistent with the upstream declaration of formalization scope. We also construct LeanReviewBench, comprising 501 fixed claim-statement-context triples from OpenAI Math, Fermat's Last Theorem, and Kakeya, with description provenance recorded. The resource supports controlled comparisons of more project-level semantic alignment evaluations.