What speeds are everyone getting with deepseek v4 flash 0731?

<!-- SC_OFF --><div class="md"><p>DeepSeek-V4.1-Flash's prompt state is only 0.9 KB/token, so you can split it at layer 20 across CUDA/Metal. All you need is 1/10GbE network.

<!-- SC_OFF --><div class="md"><p>I’ve managed to get it to run and, with some patches to llama.cpp, I get ~8 tok/s decode and ~30 tok/s prefill on a M1 Max 64GB MacBook.</p>

<!-- SC_OFF --><div class="md"><p>Got Qwen 3.5 27B to 1,103,941 tok/s on 12 nodes with 96 B200 GPUs.</p> <p>At that rate you process 50,000 insurance policy documents in hours

<!-- SC_OFF --><div class="md"><p>So i recently updated LMstudio after a long pause and updated my llama.cpp runtimes too.. i was shocked.. i thought maybe something like

<!-- SC_OFF --><div class="md"><p>I'm getting ~13 tps on Q8_0, with a context window of 128000, K Q8_0, V Q8_0</p> <p>this is on 3x GPUS (1x2060super 8gb, 2x5060ti 16gb), via

<!-- SC_OFF --><div class="md"><p>Few days ago I posted about my <a href="https://www.reddit.com/r/LocalLLaMA/comments/1t94ito/i_have_deepseek_v4_pro_at_home/">DeepSeek V4 Pro</a>

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1x8qvaj/muse_spark_on_m3u_256_gb/"> <img alt="Muse Spark on M3U 256 GB"

<!-- SC_OFF --><div class="md"><p>If anyone has testing results or any results can you please share performance and or effects of KV Cache precision with Deepseek V4 Flash