论文

ProtoDiffusion:逐层生成多图层设计的潜在扩散框架

ProtoDiffusion: Token-per-Layer Generative Modeling of Multi-Layer Images

精选理由

做设计工具或分层图像生成的开发者可以看看这篇。它把多层设计的表示压缩到 HW+N 个 token,text-to-design 的 FID 比 FLUX.2 [klein] 低了 20 多个点,还快 19 倍,对比现有的先生成再分解方案明显更省。

一篇新论文提出 ProtoDiffusion,用于生成由 RGBA 图层组成的图形设计。核心组件是 ProtoVAE,一种多层变分自编码器:每层编码为一个候选向量,与全画布共享一个空间原型,N 层设计的表示只需 HW+N 个 token,而传统逐层网格表示需要 NHW 个。

正文

模型以图像或文本为条件,联合去噪原型和固定数量的候选槽位,可选边界框作布局引导;无需输入框时模型自行决定层数和布局,闲置槽位留空。同一架构同时支持 image-to-design 和 text-to-design 两种任务。

实验数据方面:ProtoVAE 在图层裁剪上重建达 40.43 dB PSNR;LayerD 评测中 image-to-design 的 RGB L1 为 0.040(有框)/0.057(无框),框引导 alpha soft IoU 0.850 接近最强微调基线的 0.871;text-to-design 在合成图像上 FID 为 66.41,优于 FLUX.2 [klein] 的 86.84,text-accuracy 达 0.860,且比同骨干网络的先生成再分解级联快 19.1 倍。

原文 · arXiv cs.AI

ProtoDiffusion: Token-per-Layer Generative Modeling of Multi-Layer Images

Graphic designs are ordered compositions of RGBA layers whose separation enables selection, movement, and reuse. Existing layered models represent each layer with a full-canvas or region-sized spatial latent grid, so representation size grows with the cumulative spatial extent of the layers. We present ProtoDiffusion, a latent-diffusion framework built on ProtoVAE, a multi-layer variational autoencoder that encodes a design into one shared spatial prototype and one candidate vector per layer. Each candidate conditions a shared decoder to reconstruct its corresponding RGBA layer. For $N$ layers, the representation contains $HW+N$ tokens before backbone-specific packing, compared with $NHW$ for full-grid-per-layer representations. ProtoDiffusion jointly denoises the prototype and a fixed set of candidate slots, conditioned on an image or text, with bounding boxes as optional layout guidance. The model leaves unneeded slots empty and determines the number of layers and, in absence of boxes, their layout, requiring neither as input. The same architecture supports both image-to-design and text-to-design. ProtoVAE achieves high-fidelity reconstruction from its compact shared latent representation, reaching 40.43 dB PSNR on layer crops. On the LayerD evaluation split, image-to-design achieves RGB L1 scores of 0.040 with bounding boxes and 0.057 without, with box-guided alpha soft IoU (0.850) comparable to the strongest fine-tuned baseline (0.871). On merged images, text-to-design reaches 66.41 FID versus 86.84 for FLUX.2 [klein], achieves a 0.860 text-accuracy score, and runs $19.1\times$ faster than the same-backbone generation-then-decomposition cascade.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心追踪栏目。涉及核心实体与技术议题包括:ProtoDiffusion、FLUX.2、扩散模型、图像生成,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Adrià Ciurana Lanau, Harleen Hanspal, Lusine Abrahamyan, Emilio Tylson Baixauli, Yu Cao, Danai Triantafyllidou, Stefano Zorzi, Gustavo Zanoni Felipe, Peter Griger, Stefan Sietzen, Petru-Daniel Tudosiu),经 AITOP 多维评分管线结构化提炼。