Quantization from the Ground Up

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1s5cggh/google_turboquant_blew_up_for_kv_cache_heres/"> <img alt="Google TurboQuant blew up for KV cache.

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1s5qdf6/qwopus_v2_nvfp4_quantization/"> <img alt="Qwopus v2 nvfp4 quantization"

<!-- SC_OFF --><div class="md"><p>Can anyone guide me generally on how to make your own quantized versions of models? </p> </div><!-- SC_ON --> &#32; submitted by &#32; <a

<!-- SC_OFF --><div class="md"><p>Been running a few models locally at different quant levels and honestly the jump from Q5 to Q4 sometimes feels like nothing and other times it

<!-- SC_OFF --><div class="md"><p>I applied video compression to LLM inference and got <strong>10,000x less quantization error at the same storage cost</strong></p> <p><a

<!-- SC_OFF --><div class="md"><p><a href="https://arxiv.org/pdf/2607.13511">https://arxiv.org/pdf/2607.13511</a></p> <p>the core idea is, we cannot have ternary PTQ with fixed

<!-- SC_OFF --><div class="md"><p>which model quantization technique is best suitable for smartphones at this point...specially if the model is finetuned as that tends to amplify

<p>I'm a 10th grade student teaching myself Rust by building something instead of just reading theory. The project is called Green-AI — an attempt to explore more energy-efficient