Need Info on quality benchmarks to run on DeepSeek V3.2 different quant levels [

<!-- SC_OFF --><div class="md"><p>Been running a few models locally at different quant levels and honestly the jump from Q5 to Q4 sometimes feels like nothing and other times it

<!-- SC_OFF --><div class="md"><p>Working on evaluating some AI-generated outbound (SDR-style emails along with follow-ups), and I’m running into a weird problem. Everyone talks

<!-- SC_OFF --><div class="md"><p>I’m planning to deploy the 9B and 27B parameter models using vLLM and was wondering if anyone has done some thorough testing on the non-GGUF

<!-- SC_OFF --><div class="md"><p>Title says it. What are the current standard benchmarks to test model perplexity?</p> <p>I want to play around with different quantization

<!-- SC_OFF --><div class="md"><p>So beellama allows you to only reduce quantization on older cache, this should allow better performance than reducing quantization on all