不改模型只改推理程序：科学图表问答准确率从16.6%升至41.3%
Improving VLMs on Scientific Chart Analysis via Autonomous Evolution
冻结的 VLM 不重训，光进化推理代码就能把图表问答从 16.6% 拉到 41.3%，还快 3.8 倍，AlphaEvolve 那套玩法很实用。
一篇 arXiv 论文针对开源 VLM 在科学图表问答上的低准确率，尝试完全不改动模型权重，只用自主进化方法优化推理程序。方法借鉴 AlphaEvolve：LLM judge 每轮查看代码和训练题上的失败案例，提出进化修改，候选程序按训练准确率排名保留。在 3 个 VLM 和 4 个图表基准上，描述性问题的精确匹配准确率平均从 16.6% 提升到 41.3%，单次最大提升 55.2 个百分点。进化后的程序还比初始版本快 3.8 倍。
Improving VLMs on Scientific Chart Analysis via Autonomous Evolution
Understanding and operating with scientific charts is a standard approach in data analysis, and while humans are good at interpreting such charts, open vision-language models (VLMs) are known to perform poorly when it comes to question answering based on such data types. Motivated by an initial observation that many of their errors are due to inexact answering, we raise the following natural question: how much accuracy can we recover without changing the model at all? In this paper, we study this question by searching over inference programs and the code around a "frozen" version of the model. Using ideas from autonomous evolution (famously used in AlphaEvolve), we first start from a baseline inference program, and in each round we show an LLM judge the code together with a few failures on training questions, and ask for an improving evolutionary move. As usual in autonomous evolution approaches, the candidate programs are kept in a small pool ranked by training accuracy, and the next edit is performed on top-scorers from this pool. We run this approach on three VLMs and four popular benchmarks on scientific charts, and we measure the fraction of descriptive questions whose answer exactly matches the ground truth. The accuracy improves in all experimental settings. On average, it rises from 16.6% with the starting program to 41.3% with the evolved one, and the largest single increase is 55.2 percentage points. As a side benefit, our evolved program is 3.8x faster than the starting program, because short answers are easier to grade and faster to generate. Our results demonstrate that a large part of the observed errors arise due to the inference pipeline, and that optimizing it can improve the "frozen" VLM without the need to retrain it.
