LoRAForge：压缩与执行协同设计的多 LoRA 高效服务系统
LoRAForge: Co-Designing Post-Hoc Compression and Execution for Efficient Multi-LoRA Serving
跑一堆 LoRA 适配器做服务的朋友看这个：85 个 Qwen 和 Llama 适配器压到 7.7% 参数，分数基本不掉，吞吐还涨了。
LoRAForge 针对大量 LoRA 适配器共享同一基座模型时的存储、加载和执行开销，把批量因子构建与按秩专用执行通过显式保留秩表示连接起来。系统提供两种分配策略：统一保留比例的 LoRAForge-U 和谱能量预算分配的 LoRAForge-S，二者共用 FlashTSQR 因子构建，相比匹配的库级 QR 最多加速 2.28 倍。在 Qwen 固定工作负载控制实验中，共享运行时为两种压缩策略带来 1.17–1.22 倍吞吐提升。在 85 个开源 Qwen 与 Llama 适配器上，谱策略配置仅保留约 7.7% 的适配器参数，舰队平均分数下降不超过 1.2 分（0–100 分制）。
LoRAForge: Co-Designing Post-Hoc Compression and Execution for Efficient Multi-LoRA Serving
Multi-LoRA serving allows many task-specific adapters to share a common base model, but scaling to large adapter fleets introduces substantial storage, loading, and execution overheads. Post-hoc compression can reduce the memory footprint of trained LoRA adapters, yet smaller ranks do not automatically translate into faster serving. For instance, padding, factor access, and generic kernels can obscure the retained low-rank structure, while independently recompressing thousands of modules also incurs significant preparation overhead. We present LoRAForge, a system connecting batched factor construction to rank-specialized execution through an explicit retained-rank representation. It supports two allocation policies: uniform retention ratios (LoRAForge-U) and spectral-energy budget allocation (LoRAForge-S). Both share FlashTSQR factor construction and the same serving backend. FlashTSQR exploits parallelism across modules and row tiles, accelerating factor construction by up to $2.28\times$ over matched library QR. In a fixed-work Qwen control, the shared runtime adds $1.17$--$1.22\times$ throughput across both compressed policies, with negligible gains on original adapters. Across 85 trained open-weighted Qwen and Llama adapters, the spectral policy's floored quality configuration retains about 7.7% of adapter parameters with fleet-average score drops of at most 1.2 points on a 0--100 scale. The same rank-floored $1/12$ spectral exports achieve substantial token-throughput gains on both AMD and NVIDIA GPUs.
