arXiv:四层可执行验证协议评估 LLM 生成的 HVAC 数字孪生规格
A four-level executable validation protocol for evaluating LLM-generated topology specifications of physics-based HVAC digital twins
8 个大模型写空调数字孪生规格,仿真跑通了不代表物理正确,这篇论文给出四层校验方法把它们区分开。转轮除湿是重灾区,做工业数字孪生的开发者值得照抄这套协议。
arXiv 论文提出一套四层验证协议,用于评估 LLM 生成的空调系统物理数字孪生规格是否物理有效。四层依次为静态有效性、模型构建、仿真收敛、以及与隐藏参考模型的行为等价性校验,等价带宽由 ±20% 尺寸扰动校准,解决了以往仅凭解析成功或仿真跑通就通过评估、无法识别仿真收敛但物理错误的模型的问题。
协议应用于 8 个 LLM 和 14 种 HVAC 拓扑,其中 5 种来自真实低湿度测试室。结果显示,三次修复内 8 个模型中有 7 个在 98.6% 至 100% 的试验中完成构建,仿真收敛率至少 80.0%,但行为等价率仅 50.0% 至 98.6%。约一半失败试验仿真收敛却未达参考稳态,主要失败点集中在转轮除湿系统。为排除作者偏差,工程师和另一模型家族也参与了拓扑描述,混合通过率在不同描述作者间差异不足五个百分点。
作者指出,每个领域中最难建模的物理环节(此处为转轮除湿)应作为验证层级纳入生成框架,该协议可推广到其他从声明式规格生成可执行数字孪生的领域。
A four-level executable validation protocol for evaluating LLM-generated topology specifications of physics-based HVAC digital twins
Physics-based digital twins of air conditioning, a principal energy consumer in manufacturing and buildings, can be generated automatically from a declarative specification of its equipment, connections, and sizing. Large language models (LLMs) now write such specifications from plain-language descriptions. Evaluations of these specifications stop at parse success, completed runs, or textual similarity, and cannot reject a model that converges in simulation but is physically wrong. In this study, a four-level protocol is proposed that tests whether an LLM-written specification not only executes but also yields a physically valid digital twin. Each specification is executed through static validity, model build, simulation convergence, and behavioral equivalence with a hidden reference model within bands calibrated by $\pm$20% sizing perturbations. The protocol was applied to eight LLMs and fourteen HVAC topologies, five describing an existing low-humidity test chamber. To test for authorship bias, engineers and a second model family also described six topologies. Within three repairs, seven of eight models built in 98.6 to 100% of trials, with simulations converging in at least 80.0%, but reached behavioral equivalence in only 50.0 to 98.6%. About half of the failed trials converged, but not to the reference steady state, mostly in desiccant rotor systems. Pooled pass rates varied by under five percentage points across description authors. These results imply that the physics hardest to model in each field, here the desiccant rotor, must enter the generation framework as validation levels. The protocol can be adapted to other fields that generate executable digital twins from declarative specifications.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:LLM、数字孪生、HVAC、仿真验证,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Younghwan Joo, Jeonghoon Han, Sanghyun Oh, Soosik Bang, Sung-il Kim),经 AITOP 多维评分管线结构化提炼。