Supercharging LLM inference on Google TPUs: Achieving 3X speedups with diffusion

<p>Building a production-ready, fault-tolerant Retrieval-Augmented Generation system is an exercise in managing harsh tradeoffs. You want massive context, lightning-fast hybrid

<p>Today Inception Labs dropped <strong>Mercury 3</strong>: a diffusion-based LLM with a 1M-token context window doing <strong>1,068 tokens/sec on NVIDIA GPUs</strong>, claiming

<!-- SC_OFF --><div class="md"><p>If the claims presented in the paper are true, this will be very big for multi-user local inference</p> </div><!-- SC_ON --> &#32; submitted by

<!-- SC_OFF --><div class="md"><p>Google DeepMind dropped Gemma 4 today:</p> <p><strong>Gemma 4 31B:</strong> dense, 256K context, redesigned architecture targeting efficiency and

<!-- SC_OFF --><div class="md"><p>Hey everyone, I've been working on getting Multi-Token Prediction (MTP) working with quantized GGUFs for Qwen3-27B and the results are pretty

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1s5cggh/google_turboquant_blew_up_for_kv_cache_heres/"> <img alt="Google TurboQuant blew up for KV cache.

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1x75bme/alodlm_8b_looped_diffusion_language_model_3x/"> <img alt="ALoDLM 8b looped diffusion language model:

<!-- SC_OFF --><div class="md"><p><strong>Cost &amp; Performance Efficiency</strong></p> <ul> <li><strong>Training Cost-Performance (8t):</strong> +170% to +180% gain