ALLMTS：让 LLM 执行电话分诊指南，准确率达 0.95
ALLMTS: Automated LLM-based Medical Triage System
让 LLM 照着护士用的分诊指南一步步执行，566 个场景测出 0.95 准确率，直接问 LLM 只有 0.47，过度分诊也从 0.5 压到 0.02。
电话分诊要求护士根据简短的接诊记录判断呼叫者需要救护车、急诊、预约还是居家自理。ALLMTS 不直接让 LLM 做分诊决策，而是让它执行护士使用的商用远程医疗指南，根据首个肯定回答确定性得出处置结果。在 566 个经专家验证的合成场景、覆盖 14 种处置类别上，ALLMTS 达到 0.95 准确率，而原始 LLM 基线仅 0.47。在 50 个场景的开放基准子集上，ALLMTS 达到 0.98，接近五名护士专家小组 0.944 的正确处置率，同时把过度分诊率从基线最高 0.50 降到不超过 0.02。
ALLMTS: Automated LLM-based Medical Triage System
Telephone triage is a safety critical task: from a brief intake note, a nurse must decide whether a caller needs an ambulance, an emergency department, an appointment, or self care at home. Queried directly, large language models (LLMs) are unreliable: they over-escalate routine cases and cannot justify their recommendations against an auditable clinical standard. We present the Automated LLM-based Medical Triage System (ALLMTS), which grounds every disposition in the same commercially maintained telehealth guidelines nurses use. Rather than asking an LLM to triage, ALLMTS makes it execute the guideline: it selects the appropriate guideline and evaluates its triage assessment questions. The disposition follows deterministically from the first positive answer, using the same traversal rule a nurse follows. We evaluate ALLMTS on 566 synthetic, expert-validated scenarios spanning 14 dispositions and designed to test specific guideline questions. ALLMTS achieves 0.95 accuracy, compared with 0.47 for a raw-LLM baseline. On a curated 50-scenario subset released as an open benchmark, ALLMTS reaches 0.98 accuracy, on par with the 0.944 correct disposition rate that a five nurse expert panel recorded on those same scenarios. Critically, on that subset the baselines have over-triage rates of up to 0.50 (0.44 on the full dataset), whereas ALLMTS reduces the over-triage rate to at most 0.02 while maintaining low under-triage. These results support coupling LLM language understanding with executable clinical guidelines for triage from intake notes.
