Gemma 4 and Qwen 3.6 with q8_0 and q4_0 KV cache: KL divergence results

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1seua77/gemma_4_31b_gguf_quants_ranked_by_kl_divergence/"> <img alt="Gemma 4 31B GGUF quants ranked by KL

<!-- SC_OFF --><div class="md"><p>I'm continuing to play around with local llms on my framework13 laptop. </p> <p>So, limited memory bandwith and processing power means exploring

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1s5cggh/google_turboquant_blew_up_for_kv_cache_heres/"> <img alt="Google TurboQuant blew up for KV cache.

<!-- SC_OFF --><div class="md"><p>I applied video compression to LLM inference and got <strong>10,000x less quantization error at the same storage cost</strong></p> <p><a

<!-- SC_OFF --><div class="md"><p><a href="https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF">https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF</a></p> <p><a

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1sa4lzt/i_benchmarked_quants_of_qwen_3_6b_from_q2q8_heres/"> <img alt="I benchmarked quants of Qwen 3 .6b

<table> <tr><td> <a href="https://www.reddit.com/r/singularity/comments/1s7rcwr/qwen35_omni_qwens_latest_generation_of_fully/"> <img alt="Qwen3.5 Omni - Qwen’s latest generation

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1szrbub/qwenscope_official_sparse_autoencoders_saes_for/"> <img alt="Qwen-Scope: Official Sparse