论文

HCG 提出层级协作 GFlowNets,动态划分搜索空间提升探索覆盖

Hierarchical Cooperative GFlowNets

精选理由

多个 GFlowNet 各自探索容易撞在同一片高奖励区,这篇论文用共享分类器动态分地盘,采样覆盖更广,做生物序列设计的可以看看。

arXiv 论文提出 Hierarchical Cooperative GFlowNets(HCG),用于解决 GFlowNets 探索不足、采样集中于少数高奖励区域的问题。HCG 让多个 GFlowNets 通过共享分类器动态划分搜索空间:排名最高的模型可探索全部空间,排名较低的模型被推向未被覆盖的区域。论文给出 normalizer 加权聚合规则,并证明在给定假设下该集成方法能恢复与奖励成比例的目标分布。实验覆盖多模态分布、序列生成、组合优化和生物序列设计,显示模式覆盖与目标逼近能力随协作采样器数量增加而提升。

原文 · arXiv cs.LG

Hierarchical Cooperative GFlowNets

Generative Flow Networks (GFlowNets) are trained to sample objects from compositional spaces with probability proportional to a reward function, but ineffective exploration can cause learned samplers to concentrate on a few high-reward regions. An intuitive remedy is to train multiple GFlowNets independently in parallel and combine their samples. However, this does not prevent multiple models from collapsing onto the same high-reward areas, leaving large regions of the state space unexplored. With this in mind, we propose Hierarchical Cooperative GFlowNets (HCG), an approach that dynamically partitions the search space to coordinate discovery without relying on hand-crafted boundaries. HCG imposes an ordering in which the highest-ranked model may explore the entire space, while lower-ranked models are progressively pushed toward regions not covered by higher-ranked ones. We provide a theoretical characterization of how specific learning objectives can shift probability mass away from designated areas, instantiating this mechanism within our architecture. A shared classifier coordinates the hierarchy by partitioning the search space, keeping the GFlowNets independently parameterized and enabling asynchronous training. Furthermore, we derive a normalizer-weighted aggregation rule for the ensemble and prove that it recovers the target reward-proportional distribution under the stated assumptions. Finally, experiments across multimodal distributions, sequence generation, combinatorial optimization, and biological sequence design show that HCG improves mode coverage and target approximation, translating increases in the number of coordinated samplers into broader, complementary exploration.