Thousands of tokens per second?

<!-- SC_OFF --><div class="md"><p>We're obsessed with raw tokens per second. Every hardware post leads with it. Every quantization comparison is ranked by it. It's the one number

<a href="https://news.ycombinator.com/item?id=49941327">Comments</a>

<!-- SC_OFF --><div class="md"><p>I need to create an analysis of how many tokens it takes for various type of agents - single to multi agent, simple to complex ones. Is there any

<!-- SC_OFF --><div class="md"><p>i'm wondering if there is a certain way or model or third party tool that would allow me to accelerate the tps for openai models?</p> </div><!--

<!-- SC_OFF --><div class="md"><p>Got Qwen 3.5 27B to 1,103,941 tok/s on 12 nodes with 96 B200 GPUs.</p> <p>At that rate you process 50,000 insurance policy documents in hours

<!-- SC_OFF --><div class="md"><p>We currently have an LLM that does over 14000 tk/s. <a href="https://chatjimmy.ai/">https://chatjimmy.ai/</a></p> <p>OpenAI’s ultrafast mode does

<table> <tr><td> <a href="https://www.reddit.com/r/singularity/comments/1thw41i/cerebras_is_running_a_trillion_parameter_model/"> <img alt="Cerebras is running a trillion

<!-- SC_OFF --><div class="md"><p>I often hear FANG employees discussing LLM budgets and token usage.</p> <p>So, a token is like a payment, right? Why don't they just say