论文

FlowTranslator:免训练的图像内机器翻译框架

FlowTranslator: Training-Free In-Image Machine Translation with Flow-Based Editing

精选理由

不用重新训练模型,靠 OCR 加 LLM 加扩散模型就能把图片里的文字换成译文,翻成中文的场景还能跑赢现有编辑系统,做图像翻译的可以看看这篇。

论文提出 FlowTranslator,一个免训练的图像内机器翻译(IIMT)框架,用 OCR 模型和 LLM 生成翻译提示词,注入 T2I 扩散模型完成文字替换。方法引入 RCR 和 CE 两种轨迹控制来提升译文生成质量。在 EasyOCR 读回协议下,PRIM 的 BLEU 和 COMET 分别达到 20.6 和 57.2,超过基于训练的基线 UniTranslator。在更接近真实场景的 ScenePair 整图照片翻译成中文任务上,精确匹配准确率为 40.1%,超过现有编辑系统。

原文 · arXiv cs.AI

FlowTranslator: Training-Free In-Image Machine Translation with Flow-Based Editing

In-image machine translation (IIMT) replaces text in an image with its translation while preserving the surrounding visual context. Existing task-specific models achieve convincing performance but require massive paired training images before and after translation. As advanced text-to-image (T2I) models have reached a practically usable level of multilingual text generation, we argue that this capability can be transferred to IIMT and present FlowTranslator. It is a training-free IIMT framework that first employs an OCR model and an LLM to generate the translation prompt, which is injected into the T2I-based diffusion model. To optimize the accurate and natural generation of translated text, we introduce two trajectory controls: Region Coupling Release (RCR) partially decorrelates target-branch noise inside the text region early in the trajectory, while Coupling Expansion (CE) amplifies the emerging edit displacement mid-trajectory. Under the EasyOCR read-back protocol, FlowTranslator achieves average PRIM BLEU and COMET scores of 20.6 and 57.2, respectively, substantially outperforming the strong training-based baseline UniTranslator. On the more challenging and realistic whole-image ScenePair photographs translated into Chinese, it reaches 40.1\% exact-match accuracy and surpasses existing editing systems, some of which only address the text-rendering stage rather than the complete IIMT task. These results establish flow-based editing as a strong, fully training-free baseline for IIMT.