General Compute 与 Positron 将 prefill 和 decode 拆分到不同芯片上推理
Prefill 跑 GPU、decode 交给 Positron 的内存优先芯片，合成一个端点卖，思路挺聪明的，讲清了为啥推理瓶颈在内存不在算力。
General Compute 在 GPU 上运行 prefill（提示词读取），把逐 token 的生成交给 Positron 的内存优先芯片处理，并以单一端点对外提供服务。拆分的依据是两种负载的物理特性：prefill 是大规模并行计算任务，decode 则主要瓶颈在于内存带宽，单一芯片只能取舍其一。Positron 的芯片专门针对 decode 优化，且不依赖 HBM 内存。对客户而言仍是一个合同和一套 SLA，芯片分工由 General Compute 负责。
Splitting prefill and decode across different silicon is one of the most sensible ideas in inference right now.
@general_compute is running prompt reads on GPUs and handing token-by-token generation to @positron_ai’s memory-first systems, then selling both as one endpoint.
i.e. run prefill on GPUs and decode on Positron hardware, giving customers one endpoint for a request that crosses 2 chip types.
These chips are built to write AI answers faster and cheaper by focusing on memory speed (the real bottleneck when AI generates text) instead of raw processing power.
The split follows the physics. Prefill is a large parallel compute job. Decode is mostly the chip waiting to pull weights and cache out of memory for every new token. A single chip has to compromise on at least one of those.
Positron builds for the second job, and builds it without relying on HBM, the memory GPUs are all competing for.
From the customer’s side it stays one contract and one set of SLAs. Deciding which chip handles which phase is General Compute’s problem.
