Together 发布 Together Compiler:为生产流量持续优化推理
Together 搞了个自动优化推理的东西,跑在你家生产流量上边跑边调,号称比 InferenceX 快到 2.6 倍,做部署的可以看看。
Together 推出 Together Compiler,定位为能在生产流量上持续优化推理性能的系统。该系统每天服务数万亿 token,官方称其速度可达领先 InferenceX 配置的 2.6 倍,在部分客户工作负载上快 50%。它由 Reyna Abhyankar、Dylan Lim、Simran Arora 等人参与开发。
Introducing Together Compiler: agents that continually optimize inference for production traffic.
Serving trillions of tokens/day. Up to 2.6x faster than leading InferenceX configs, up to 50% faster on customer workloads.
Congrats @reyna_abhyankar @dylan__lim @simran_s_arora and the team!