Yalda-Embedding：595.8M 参数的波斯语检索模型开源
Yalda-Embedding: Open Development of a Competitive Persian Retrieval Model
一个开源的波斯语检索模型，0.6B 大小打平 4B 的 Qwen3 原版，做小语种 RAG 的可以看看它的蒸馏配方。
波斯语密集检索长期受限于语言专用训练数据不足。Yalda-Embedding 基于 Qwen3-Embedding-0.6B 适配，用超过 100 万查询-文档对训练，参数量 595.8M。在 FaMTEB 的 8 个波斯语检索任务上，它以 0.5488 的宏平均在所有 0.6B 模型中排第一，每个任务都超过 Qwen3-Embedding-0.6B 和 F2LLM-v2-0.6B，并接近 Qwen3-Embedding-4B（宏平均仅差 0.0075）。消融实验显示，从 F2LLM-v2-8B 蒸馏后均值从 0.4813 提升到 0.5488，且推理时模型大小不变。模型权重、训练代码、数据集和教师嵌入版本均已开源。
Yalda-Embedding: Open Development of a Competitive Persian Retrieval Model
Despite advances in multilingual text representations, effective dense retrieval for Persian remains constrained by the limited availability of language-specific training resources and compact embedding models. We present Yalda Embedding, a 595.8-million-parameter Persian retrieval model adapted from Qwen3-Embedding-0.6B using more than one million query-document pairs. On eight Persian retrieval tasks from FaMTEB, the model achieves the highest macro-average among the evaluated 0.6B models, scoring 0.5488 and outperforming Qwen3-Embedding-0.6B and F2LLM-v2-0.6B on every task. It also approaches Qwen3-Embedding-4B, winning on four tasks, tying on one, and trailing by 0.0075 in macro-average. An incremental ablation study examines contrastive batch size, optimization stabilization, and relational knowledge distillation from F2LLM-v2-8B. Under the one-epoch training setup, contrastive fine-tuning alone yields a modest mean improvement over initialization and regressions on two tasks. The final distilled configuration improves all eight tasks over the non-distilled model, raising the mean from 0.4813 to 0.5488 without increasing inference-time model size. These gains include hard-negative evaluation tasks despite training without explicitly mined or supplied hard negatives. We release model weights, training and evaluation code, the pairwise dataset, and its teacher-embedded counterpart, enabling inspection of data composition and training procedures, independent evaluation, and reuse without repeating expensive teacher inference.
