Why we cannot “compress” number of weights down?

<!-- SC_OFF --><div class="md"><p>is it possible that a moe model can decide how many billion parameters to activate per token according to the task. eg if qwen 3.6 35b a3b - if a

<!-- SC_OFF --><div class="md"><p>Hi. We recently had MoE models as big as 1T and 1.6T total parameters. My expectation on the proportion between total and active parameters so

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1s5cggh/google_turboquant_blew_up_for_kv_cache_heres/"> <img alt="Google TurboQuant blew up for KV cache.

<!-- SC_OFF --><div class="md"><p>I’m interested in dual Xeon setup with AMX support for ktransformers and CPU sglang backend. Let’s say I have 512gb RAM in 8x channel for each

<!-- SC_OFF --><div class="md"><p>Forgetting the villainization of overfitting, what happens when you just take a massive amount of training data and overfit the model on that. Is

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1st8l2u/kimi_k26_takes_1st_overall_of_open_weights_models/"> <img alt="Kimi K2.6 takes 1st overall of open

<table> <tr><td> <a href="https://www.reddit.com/r/MachineLearning/comments/1t2zy4h/torchnvenccompress_gpu_nvenc_silicon_as_a_pcie/"> <img alt="torch-nvenc-compress: GPU NVENC