Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s

<a href="https://news.ycombinator.com/item?id=49953495">Comments</a>

<a href="https://news.ycombinator.com/item?id=47476422">Comments</a>

<!-- SC_OFF --><div class="md"><p>I’ve managed to get it to run and, with some patches to llama.cpp, I get ~8 tok/s decode and ~30 tok/s prefill on a M1 Max 64GB MacBook.</p>

<!-- SC_OFF --><div class="md"><p>DeepSeek-V4.1-Flash's prompt state is only 0.9 KB/token, so you can split it at layer 20 across CUDA/Metal. All you need is 1/10GbE network.

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1t6x2za/ds4c_deepseek_4_flash_local_inference_engine_for/"> <img alt="ds4.c - DeepSeek 4 Flash local

<a href="https://news.ycombinator.com/item?id=49098510">Comments</a>

<!-- SC_OFF --><div class="md"><p>I'm genuinely unsure if I'm doing something wrong, or if the model is just worse, despite the community enthusiasm. I tend to use it with pi

<!-- SC_OFF --><div class="md"><p>I could not find good data points on what speed one could get with a single 5090 and enough DDR4 RAM. </p> <p>My system: AMD EPYC 7532 32core