论文

GrapeSplat:无位姿图像单次前向重建可渲染 3D 高斯场景

GrapeSplat: Geometry-Grounded Reconstruction via Amalgamated Pose-Free Encoding for Feed-Forward 3D Gaussian Splatting

精选理由

无位姿照片丢进去,一次前向就出可渲染的 3D 高斯场景,权重也开源了,做 3D 重建的可以去看看。

arXiv 论文 GrapeSplat 针对前馈式 3D Gaussian Splatting 只靠光度一致性监督、逐像素预测高斯的缺陷,把多视角线索融合进体素对齐的场景表征,无需逐场景优化或后处理。其 Atlas Encoder 将各视角提升为锚定在预测 3D 点上的逐像素几何与外观特征,PEACH-Vox 通过带精确闭式逆映射的平滑逐轴压缩把无界场景收进有界稀疏网格,Sparse Decoder 再用稀疏卷积从每个被占据体素解码出多个高斯。高斯数量跟随被占据体素并随视角覆盖饱和,由网格分辨率决定上限,不再绑定图像分辨率与视角数。模型在 8 视角序列上以 2D 与 3D 监督训练,零样本泛化到 4 至 64 视角,覆盖室内与无界场景,代码和训练权重已在 GitHub(VAISR/GrapeSplat)开源。

原文 · arXiv cs.AI

GrapeSplat: Geometry-Grounded Reconstruction via Amalgamated Pose-Free Encoding for Feed-Forward 3D Gaussian Splatting

Feed-forward 3D Gaussian Splatting now reconstructs renderable scenes from unposed, uncalibrated images. Yet, most models supervise only photometric consistency and predict Gaussians pixel by pixel, which leaves global structure fragile and ties primitive count to image resolution and view count. To this end, GrapeSplat amalgamates multi-view cues into a voxel-aligned scene representation and decodes Gaussians directly from the learned grid, requiring no per-scene optimization or post-processing. An Atlas Encoder lifts all views into pixel-wise geometry-and-appearance features anchored at predicted 3D points. PEACH-Vox compands the unbounded scene into a bounded sparse grid through a smooth per-axis map with an exact closed-form inverse. The Sparse Decoder then consolidates the grid with sparse convolutions and decodes the full scene as multiple Gaussians per occupied cell. This amalgamated representation exploits sparse voxel occupancy, where the Gaussian count follows the occupied cells and saturates as views cover the scene, while grid resolution sets its ceiling. GrapeSplat turns unposed images into a renderable Gaussian scene in a single forward pass. Trained with 2D and 3D supervision on 8-view sequences, it generalizes zero-shot from 4 to 64 views across indoor and unbounded scenes. Code and trained weights are available at https://github.com/VAISR/GrapeSplat