Deepseek V4 Flash 0731 KV Cache precision

<!-- SC_OFF --><div class="md"><p>This is one more vibe slopped custom optimization for, in this case, my hardware (m2 ultra 60 cores, 192gb). It is just a fork from llama.cpp

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1x8qvaj/muse_spark_on_m3u_256_gb/"> <img alt="Muse Spark on M3U 256 GB"

<!-- SC_OFF --><div class="md"><p>I’ve managed to get it to run and, with some patches to llama.cpp, I get ~8 tok/s decode and ~30 tok/s prefill on a M1 Max 64GB MacBook.</p>

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1su3fle/deepseek_v4_flash_pro_now_out_on_api/"> <img alt="DeepSeek V4 Flash &amp; Pro Now out on API"

<!-- SC_OFF --><div class="md"><p>What speeds are everyone getting with deepseek v4 flash 0731?</p> <p>I’m getting~200 tps prompt processing / ~11 tps token gen, on 4x5060ti16gb

<!-- SC_OFF --><div class="md"><p><a href="https://huggingface.co/collections/deepseek-ai/deepseek-v4">https://huggingface.co/collections/deepseek-ai/deepseek-v4</a></p>

<!-- SC_OFF --><div class="md"><p>I remember seeing something the other day about someone running deepseek locally because of some deepseek engine program or some shit and I can't

<table> <tr><td> <a href="https://www.reddit.com/r/singularity/comments/1sux16b/deepseek_v4_is_out_16_trillion_parameters_mit/"> <img alt="DeepSeek V4 is out. 1.6 trillion