视频生成中解码器长期被忽视，RefDecoder 用轻量级条件注入解决了细节丢失的痛点，做视频生成或编辑的团队可以直接替换现有系统试试，效果立竿见影。
RefDecoder 是一种参考条件视频 VAE 解码器，通过将高保真参考图像信号直接注入解码过程来改善视频生成中的细节丢失和不一致问题。它使用轻量级图像编码器将参考帧映射为高维 token，并在解码器每个上采样阶段与去噪后的视频潜在 token 协同处理。在 Inter4K、WebVid 和 Large Motion 基准测试上，RefDecoder 相比无条件基线实现了最高 +2.1dB PSNR 的提升。该方法可直接替换现有视频生成系统中的解码器而无需额外微调，并在 VBench I2V 基准上全面提升了主体一致性、背景一致性和整体质量分数。此外，RefDecoder 还能泛化到风格迁移和视频编辑优化等多种视觉生成任务。
RefDecoder: Enhancing Visual Generation with Conditional Video Decoding
Video generation powers a vast array of downstream applications. However, while the de facto standard, i.e., latent diffusion models, typically employ heavily conditioned denoising networks, their decoders often remain unconditional. We observe that this architectural asymmetry leads to significant loss of detail and inconsistency relative to the input image. To address this, we argue that the decoder requires equal conditioning to preserve structural integrity. We introduce RefDecoder, a reference-conditioned video VAE decoder by injecting high-fidelity reference image signal directly into the decoding process via reference attention. Specifically, a lightweight image encoder maps the reference frame into the detail-rich high-dimensional tokens, which are co-processed with the denoised video latent tokens at each decoder up-sampling stage. We demonstrate consistent improvements across several distinct decoder backbones (e.g., Wan 2.1 and VideoVAE+), achieving up to +2.1dB PSNR over the unconditional baselines on the Inter4K, WebVid, and Large Motion reconstruction benchmarks. Notably, RefDecoder can be directly swapped into existing video generation systems without additional fine-tuning, and we report across-the-board improvements in subject consistency, background consistency, and overall quality scores on the VBench I2V benchmark. Beyond I2V, RefDecoder generalizes well to a wide range of visual generation tasks such as style transfer and video editing refinement.