个人本地AI实验环境配置：包含Qwen 3.8 27B和Deepseek v4模型
current homelab setup for local AI experimentation: - hermes box hosted on a Framework Desktop Main...
朋友分享了自己用Qwen 3.8 27B和Deepseek v4搭建的本地AI实验环境，适合想快速测试不同大模型性能的用户参考。
用户搭建了包含Framework桌面、eGPU、树莓派5和Mac mini的多设备本地AI实验环境。其中，Qwen 3.8 27B模型在eGPU上运行，可实现150+ tok/s的吞吐量；Deepseek v4 Flash 0731模型则部署在两台DGX Spark服务器上，性能更好但速度较慢。系统还集成了Hermes盒子和Arch-Router路由插件，用于决定本地或云端处理请求。
current homelab setup for local AI experimentation: - hermes box hosted on a Framework Desktop Main...
current homelab setup for local AI experimentation: - hermes box hosted on a Framework Desktop Mainboard AI Max+ 395 - 5090 eGPU running Qwen 3.8 27B for fast tok/s LLM use - sometimes 150+ tok/s - 2x DGX Spark: running Deepseek v4 Flash 0731 - better but slower model - pi 5 for monitoring - Mac mini as a dev box - use Herdr and ohmypi/codex/claude depending on the use case - housed in a 10" DeskPi mini rack (mostly) Hermes is defaulted to local AI but with a homegrown routing plugin hitting a small low TTFT model (Arch-Router) to decide whether to go local or upgrade to cloud/frontier. Trying to get to 100% local over time, but right now probably more like 60-70% The Sparks are for batch processing background runs (all the cron jobs, longer dev builds, etc) do you need all of this? Absolutely not lol. I started with the mac mini and couldn't help myself but to add over time! Also I regret getting the eGPU so I wouldn't recommend that to anyone 💬 18 🔄 6 ❤️ 45 👀 5394 📊 25 ⚡
