BrainLoRA 用参数高效微调刷新脑编码模型性能
Scaling Parameter-Efficient Fine-Tuning for Brain Encoding
把 LoRA 微调用在脑编码上，连 104B 的 Qwen3-Omni 都能高效适配，还拿了 Algonauts 2025 第一，做法挺巧的。
研究团队提出 BrainLoRA，用 LoRA 低秩适配器对预训练多模态骨干做参数高效微调，用于预测大脑神经响应。方法利用骨干的因果结构单次前向预测多个时间点，省去滑动窗口的特征提取开销，成功适配了最大达 104B 参数的模型。在与此前 SoTA 相同的 Qwen3-Omni 骨干上，该方法在 Algonauts 2025 Challenge 上取得已公开的最优成绩，并在九个多模态骨干上一致提升表现。团队还开发了 multi-LoRA kernel 加速共享骨干上的多适配器联合训练，用 200+ 个适配模型组成的集成在 Algonauts 公开排行榜排名第一。
Scaling Parameter-Efficient Fine-Tuning for Brain Encoding
Brain encoding models aim to predict neural responses to stimuli. The most widely used approach fits readouts from frozen pretrained backbones. Using neural data to train large models from scratch or fully fine-tune them after task-pretraining has proven difficult because neural data is costly and limited. We show that parameter-efficient fine-tuning (PEFT) enables us to effectively adapt very large pretrained backbones for brain encoding with a small number of trainable parameters. To make it computationally practical, we exploit the backbone's causal structure to predict multiple timepoints in a single pass instead of using sliding windows, significantly reducing feature-extraction cost. This enables efficient fine-tuning of multimodal backbones with up to 104B parameters, the largest model adapted for brain encoding to our knowledge. Trained with a low-rank adapter (LoRA) and a lightweight readout, our approach consistently improves brain encoding performance across nine diverse multimodal backbones. On the same Qwen3-Omni backbone as the prior state-of-the-art (SoTA), our method BrainLoRA achieves the best published performance on the Algonauts 2025 Challenge. We also show the adapted multimodal backbones remain interpretable: modality attribution recovers the canonical sensory topography that prior work often obtains by construction from separate unimodal encoders, and neural adaptation increases the contribution of audio inputs and cross-modal mixing within the backbone. To scale out the number of adapted models, we develop a multi-LoRA kernel that accelerates joint training of multiple LoRAs on a shared backbone. This enables an ensemble of 200+ adapted models that ranks first on the public Algonauts post-challenge leaderboard at the time of writing. Together, our results highlight PEFT as an efficient and powerful paradigm for brain encoding.
