Running Gemma 4 26B at 5 tokens/SEC on a 13-year-old Xeon with no GPU

<a href="https://news.ycombinator.com/item?id=49098510">Comments</a>

<!-- SC_OFF --><div class="md"><p>Got Qwen 3.5 27B to 1,103,941 tok/s on 12 nodes with 96 B200 GPUs.</p> <p>At that rate you process 50,000 insurance policy documents in hours

<p>The Gemma 4 MoE stack on TPU v6e-4 has reached its definitive production state. By applying the "Turbo-Stable" low-level optimizations<br /> (512-token padding gap and 90% HBM

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1sarlb8/gemma_4_running_on_raspberry_pi5/"> <img alt="Gemma 4 running on Raspberry Pi5"

<!-- SC_OFF --><div class="md"><p><a href="https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF">https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF</a></p> <p><a

<a href="https://news.ycombinator.com/item?id=49524447">Comments</a>