SemantEq:形式化命题的双向等价性判定新方法
SemantEq: Faithful Bidirectional Equivalence of Formal Statements
自动形式化里老方法判定等价太宽松,强证明器反而容易作弊。这篇给出更严格的判定规则,精确率能到100%,做形式化数学的可以看看。
SemantEq 针对 Lean 4 自动形式化中命题是否忠实翻译原命题的问题,提出比逻辑等价更严格的双向等价关系,要求证明只能通过逻辑重排和库内等式改写等允许步骤完成。SemantEq-Eval 无需模型即可在 ProofNetVerif 测试集上达到 100% 精确率和 65.6% 召回率,而 BEq+ 为 95.8% 和 61.8%。SemantEq-ProofAudit 审计模型生成的证明时,拒绝全部 104 个非等价命题对,并将 407 个等价命题对的最佳召回率从 65.4% 提升至 81.1%。
SemantEq: Faithful Bidirectional Equivalence of Formal Statements
Statement autoformalization translates mathematics from natural language into a proof assistant such as Lean 4, and models now do it at scale. As provers get stronger, the burden of correctness shifts from the proof to the statement, that is, to whether the formal statement faithfully translates its source. Given a trusted reference formalization, the natural test is logical equivalence, and it proves too much. $P \leftrightarrow Q$ holds trivially for any two statements $P$ and $Q$ that the library proves, because a prover can prove each separately and pass through $P \leftrightarrow \mathrm{True} \leftrightarrow Q$. The stronger the prover, the more unrelated pairs pass. Earlier methods restrict the prover and check that each proof names the other statement. We introduce SemantEq, a more principled relation that instead requires each statement to turn into the other only through a chain of allowed steps, such as logical rearrangement and rewriting with the library's own equations. Our programs judge a proof by the proof term Lean checks, not by its tactic script. SemantEq-Eval, a search without a model, reaches 100% precision and 65.6% recall on the ProofNetVerif test split under our adjudicated labels, against 95.8% and 61.8% for BEq+, and an 8.9-fold increase in its CPU time adds at most three pairs. A language model finds more proofs, but under two of four prompts it proves both directions of about two in five of the non-equivalent pairs. SemantEq-ProofAudit audits each model-written proof under a policy derived from the relation and accepts none of the 104 non-equivalent pairs, and it raises recall on 407 equivalent pairs from 65.4% to 81.1% under the best prompt. Separating the generation of equivalence proofs from their acceptance lets a stronger prover raise recall without changing what an acceptance means. We release code and data at: https://github.com/hlxr89/SemantEq