Has anyone tested the quantization quality (AWQ/GPTQ/FP8/NVFP4) for Qwen3.5 9B &

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1s5cggh/google_turboquant_blew_up_for_kv_cache_heres/"> <img alt="Google TurboQuant blew up for KV cache.

<!-- SC_OFF --><div class="md"><p>I applied video compression to LLM inference and got <strong>10,000x less quantization error at the same storage cost</strong></p> <p><a

<!-- SC_OFF --><div class="md"><p>Been running a few models locally at different quant levels and honestly the jump from Q5 to Q4 sometimes feels like nothing and other times it

<!-- SC_OFF --><div class="md"><p>I could not find good data points on what speed one could get with a single 5090 and enough DDR4 RAM. </p> <p>My system: AMD EPYC 7532 32core

<!-- SC_OFF --><div class="md"><p>With practically all of LocalLlama glazing Qwen 3.5/3.6 for it's coding skills. Along with the fact that Alibaba themselves are focusing on

<!-- SC_OFF --><div class="md"><p>What's the difference between FP8 or INT8 ? For nvidia you would go FP8 but on ampere you would rely on INT8. On the other side new intel gpu

<table> <tr><td> <a href="https://www.reddit.com/r/singularity/comments/1s7rcwr/qwen35_omni_qwens_latest_generation_of_fully/"> <img alt="Qwen3.5 Omni - Qwen’s latest generation

<!-- SC_OFF --><div class="md"><p>I'm genuinely unsure if I'm doing something wrong, or if the model is just worse, despite the community enthusiasm. I tend to use it with pi