RAG 提升大模型竞赛编程解题能力的研究
Assessing the Impact of Retrieval-Augmented Generation on Algorithmic Problem-Solving
有篇论文给 LLM 做竞赛编程题时加了相似题检索，CodeForces Elo 和 pass rate 都涨了，还顺手排查了是不是靠背题，做 RAG 的可以看看
一项 arXiv 论文提出面向竞赛编程的 RAG 方法，用 embedding 检索加相似度搜索从知识库找类似题目，并用动态阈值降低检索噪声。实验基于 CodeContests 数据集，对比原始描述、随机示例、检索相似示例三种条件，用 pass rate 和 CodeForces Elo 评分衡量。团队还加入 Min-K membership inference 分析，区分检索带来的泛化与对训练数据的记忆。结果显示相似题检索能明显提高大模型在竞赛编程上的表现。
Assessing the Impact of Retrieval-Augmented Generation on Algorithmic Problem-Solving
Competitive programming problems pose distinctive challenges for Large Language Models, as they demand not only proficient code generation but also advanced algorithmic reasoning. This study presents a Retrieval-Augmented Generation approach specifically designed to enhance the capacity of Large Language Models to address such problems. Our approach integrates an embedding-based retrieval mechanism and a similarity search algorithm to identify analogous problems within a knowledge database, employing a dynamically adjusted threshold to mitigate retrieval noise. We conducted an evaluation using the CodeContests dataset, focusing on the test split problems that exhibit similarity to instances from the training split. The experimental framework considered three conditions: (i) problems provided solely with their original descriptions, (ii) problems supplemented with randomly selected examples from the training split, and (iii) problems augmented with similar examples retrieved through the proposed method. In addition, we incorporated a Min-K membership inference analysis to examine whether the generated code solutions were influenced by potential memorization of training data within the evaluated Large Language Models. This complementary perspective enables a deeper understanding of model behavior by distinguishing genuine generalization through retrieval from implicit recall of pretraining samples. Performance was assessed using both pass rates and the CodeForces Elo rating of the problems. The results provide empirical evidence that similarity-based retrieval substantially enhances Large Language Models performance in competitive programming scenarios, offering an scalable framework for improving code generation in algorithmic problem-solving contexts.
