Milvus 3.0 External Collection 发布

A new file lands in S3. Should production search expose it before its indexes are ready? ...

A new file lands in S3. Should production search expose it before its indexes are ready? 𝗠𝗶𝗹𝘃𝘂𝘀 𝟯.𝟬 𝗘𝘅𝘁𝗲𝗿𝗻𝗮𝗹 𝗖𝗼𝗹𝗹𝗲𝗰𝘁𝗶𝗼𝗻 avoids that partial state through an explicit Refresh boundary. When new files, updated metadata, or new embeddings arrive in the lake, Refresh reads the external metadata and identifies which source fragments changed. Milvus can reuse the segment and index work for unchanged fragments, while processing only the new or modified ones. During that work, queries continue using the previous published version. Once Refresh completes, the new version becomes available as a whole, so applications do not see a mixture of old and partially prepared data. External Collection also works with Milvus Tiered Storage to reduce the serving footprint. QueryNodes can initially load lightweight metadata, then fetch fields and indexes at the chunk level as queries need them. Frequently accessed data stays cached, while colder data can remain remote or be evicted. Together, incremental Refresh and Lazy Loading let Milvus keep lake-resident data searchable without rebuilding everything after each batch or loading the entire dataset into memory. See ho milvus.io/blog/milvus-3-… 0: https://t.co/VWDpFH8iAV 💬 0 🔄 0 ❤️ 0 👀 55 ⚡