LLM 何时能帮上特征选择？一项系统性实证研究
When Do LLMs Help with Feature Selection? A Systematic Empirical Study
这篇论文实测了 LLM 选特征到底靠不靠谱，结论挺实在：7B 的模型还不如传统统计方法，但特征名字本身对回归任务确实有用。
arXiv 论文在 18 个数据集上系统比较了 LLM 与传统统计方法做表格特征选择的效果，框架包含 10 个 GRPO 训练数据集、2 个验证集和 6 个留出评估集。测试对象包括 5 个经典基线、冻结的 Qwen2.5-7B 和 Llama-3.3-70B 选择器，以及 2 个 GRPO 适配的 Qwen2.5-7B 模型。结果显示经典选择器在分类 ROC-AUC 和回归 R² 上普遍优于 7B 级 LLM/RL 选择器，即使按约 33% 特征预算严格只用训练数据拟合也是如此。但语义敏感性实验发现 Zero-shot Qwen-7B 在回归任务上受益于真实特征名，相比匿名和打乱版本 R² 分别提升 0.1964 和 0.1606，分类任务提升则较小且依赖具体数据集。
When Do LLMs Help with Feature Selection? A Systematic Empirical Study
Large language models (LLMs) can leverage semantic feature names alongside summary statistics for tabular feature selection, yet it remains unclear when such information improves subset selection relative to established statistical methods. We investigate this question within an 18-dataset framework comprising 10 datasets for Group Relative Policy Optimization (GRPO) training, 2 for checkpoint validation, and 6 held-out evaluation datasets. We compare five classical baselines, frozen Qwen2.5-7B and Llama-3.3-70B selectors, and two GRPO-adapted Qwen2.5-7B models, reporting classification ROC-AUC and regression R^2 separately and using dataset-wise ranks for cross-task comparison. Across five seeded evaluation partitions, classical selectors generally achieve higher predictive performance than the tested 7B LLM/RL selectors, including under a pre-specified ~33% feature budget with strict train-only classical fitting, while the 7B selectors retain compact subsets. A cardinality-matched semantic sensitivity experiment shows that, for Zero-shot Qwen-7B, informative feature names improve regression performance over anonymized and shuffled alternatives (+0.1964 and +0.1606 mean R^2), whereas aggregate classification differences are smaller (+0.0159 and +0.0084 ROC-AUC) and dataset-dependent. These results characterize when semantic information benefits LLM-based feature selection and show that classical selectors remain strong baselines for predictive tabular feature selection.