MiST模型在网络安全基准测试中表现优异
MiST: Mid-Training LLMs for Cybersecurity
这个研究很有意思，他们开发了一个叫MiST的模型，专门针对网络安全领域，效果比之前的Qwen模型好很多，对安全分析应该有帮助。
我们介绍了MiST（Mid-trained Security Transformer）模型，它是一套8B和32B大小的模型，在网络安全领域取得了显著成果。该模型通过在通用预训练和网络安全训练之间引入中间的微调阶段，并使用专家审核过的种子语料库生成高质量的合成训练数据，最终在网络安全基准测试中，8B和32B模型分别比Qwen基线高出13.1和8.6个百分点。
MiST: Mid-Training LLMs for Cybersecurity
Cybersecurity combines high-stakes analysis with complex technical language, making it an impactful and challenging domain for LLMs. We present MiST (Mid-trained Security Transformer), a suite of 8B and 32B models that achieve strong performance on public cybersecurity benchmarks. We use mid-training as an intermediate adaptation stage between general pre-training and cybersecurity training. Rather than performing continual pre-training over large volumes of raw domain text, we curate a compact, expert-vetted seed corpus, and transform it into high-quality domain-specific synthetic training data. The final MiST checkpoints improve mean cybersecurity accuracy by +13.1 and +8.6 absolute percentage points over the corresponding Qwen baselines for 8B and 32B, respectively, corresponding to relative gains of +27.0% and +15.8%. Ablation results further show that these cybersecurity gains arise in the mid-training and supervised fine-tuning stages through a combination of the synthetic data generation flows. Furthermore, we show that MiST provides a stronger initialization for downstream task-specific fine-tuning adaptation and reinforcement learning.