Real-time LLM Inference on Standard GPUs: 3k tokens/s per request

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1t4jehc/supercharging_llm_inference_on_google_tpus/"> <img alt="Supercharging LLM inference on Google TPUs:

<a href="https://news.ycombinator.com/item?id=49202852">Comments</a>

<a href="https://news.ycombinator.com/item?id=48024540">Comments</a>

<a href="https://news.ycombinator.com/item?id=48328184">Comments</a>

<!-- SC_OFF --><div class="md"><p>Google DeepMind dropped Gemma 4 today:</p> <p><strong>Gemma 4 31B:</strong> dense, 256K context, redesigned architecture targeting efficiency and

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1s5g2zh/cohere_transcribe_webgpu_stateoftheart/"> <img alt="Cohere Transcribe WebGPU: state-of-the-art

<!-- SC_OFF --><div class="md"><p>Wrote a deep dive on <strong>FlashAttention-4 (03/05/2026)</strong> that's relevant for anyone thinking about inference performance.</p>

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1s2e0d5/built_an_opensource_llm_router_for_consumer_gpus/"> <img alt="Built an open-source LLM router for