研究分析 5398 个仓库：开发者实际只用到一小部分 LLM 推理配置项
How Developers Configure LLM Inference: A Large-Scale Study of Real-World Practices
三个后端、五千多个仓库被扒了一遍配置，想知道大家平时真的调哪些推理参数、哪些基本没人碰，看这篇就够了。
一项大规模实证研究分析了 5398 个使用 Hugging Face Transformers、vLLM 或 OpenAI API 的 GitHub 开源仓库，共识别出 345117 次显式配置的选项。结果显示，虽然 98.8% 的后端文档选项至少出现一次，但 78.9% 的选项只在不超过 10% 的相关仓库中被配置。配置规模很小，Transformers 仓库中位数为 2 个选项，vLLM 和 OpenAI 为 4 个，但选项组合差异极大，56.8%-61.6% 的多选项组合只出现在一个仓库中。不同应用场景之间的选项采用率差异最高达 30.7 个百分点。
How Developers Configure LLM Inference: A Large-Scale Study of Real-World Practices
LLMs are increasingly integrated into software systems, making deployment and inference configuration a recurring software engineering task. Inference backends expose many options that affect model loading, generation, serving, and request processing. Yet little is known about which of these options developers explicitly configure in practice. In this paper, we present a large-scale empirical study of LLM inference configuration practices in open-source software using Hugging Face Transformers, vLLM, and the OpenAI API. We analyze 5,398 GitHub repositories using at least one of these backends and identify 345,117 explicitly configured option occurrences. Our results show that developers use only a small part of the available configuration space regularly. Although 98.8% of the documented backend-specific options appear at least once, 78.9% are configured in at most 10% of repositories using the corresponding backend. Configurations are also small, with a median of 2 options for Transformers and 4 for vLLM and OpenAI, but the combinations of options vary widely: 56.8-61.6% of multi-option signatures appear in only one repository. We also observe recurring option values as well as substantial variation in how values are chosen. Finally, configuration practices differ across application contexts, with option-adoption differences reaching 30.7 percentage points. These findings characterize the configuration surface developers make explicit rather than the importance or optimality of individual options. They provide empirical evidence for prioritizing future studies of inference configuration and for improving backend documentation and configuration support.
