Gemma 4 31B GGUF quants ranked by KL divergence (unsloth, bartowski, lmstudio-co

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1suh3sz/gemma_4_and_qwen_36_with_q8_0_and_q4_0_kv_cache/"> <img alt="Gemma 4 and Qwen 3.6 with q8_0 and q4_0

<!-- SC_OFF --><div class="md"><p><a href="https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF">https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF</a></p> <p><a

<!-- SC_OFF --><div class="md"><p>Got inspired to try and crack this egg without using heretic.</p> <p>FP16, Q8_0 and Q4_K_M quants, plus the abliteration script for

<!-- SC_OFF --><div class="md"><p>I'm continuing to play around with local llms on my framework13 laptop. </p> <p>So, limited memory bandwith and processing power means exploring

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1sf1hbd/gemma_4_31b_kimi_k25_grok_420_on_duellabs_highest/"> <img alt="Gemma 4 31B &gt; Kimi K2.5 &gt; Grok

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1sja4rc/gemma_4_insists_its_not_running_locally/"> <img alt="Gemma 4 insists it’s not running locally"

<!-- SC_OFF --><div class="md"><p>I see lot of people prefer to stick different quants, like bartowski, LMstudio, gglm-org and other ones</p> <p>but why if unsloth does the job

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1sk6l63/unsloth_minimaxm27gguf_in_broken_udq4_k_xl_avoid/"> <img alt="unsloth - MiniMax-M2.7-GGUF in BROKEN