EdgeRock-4B 开源权重模型用固定大小记忆控制长推理内存占用
开源的 4B 小模型 EdgeRock-4B，靠固定大小记忆让长推理时内存可预测，同硬件能跑更多请求，不过可能丢信息，部署也有软件要求。
EdgeRock-4B 是一个支持文本和图像的 4B 开源权重模型，面向长推理任务。它把上下文历史保持在固定大小，通过保留近期信息、维护压缩摘要以及学习哪些信息值得保留来控制内存。普通模型随对话变长内存持续增长，而 EdgeRock-4B 让同一硬件能承载更多请求。代价是模型自主决定记忆内容可能丢失关键信息，且需要兼容其专用记忆架构的部署软件。
This new open-weight model is pretty interesting:
EdgeRock-4B works with text and images, and it's designed to handle long reasoning tasks, but there's a twist:
The model keeps its history at a fixed size, making memory usage more predictable as reasoning continues.
As conversations get longer, regular models use more memory on your hardware. This makes it hard for people building AI applications to predict how much memory they will need.
EdgeRock-4B does something different:
• It keeps recent information available • It maintains a compressed summary • It learns which information to keep around
Basically, the model tries to keep the most recent details together with the most important information in context.
This is good when you want predictable memory. Basically, you can handle more requests on the same hardware.
But there are trade-offs.
1. The model decides what to remember, so it can lose important information.
2. Accuracy will vary across different tasks. This model will work well for some problems, but not for others.
3. You'll need compatible software to deploy the model because of its specialized memory architecture.
You can see the model's architecture attached to this post.
