Used the RT Cores on my RTX 5070 Ti for LLM routing — 218x speedup on a single c

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1si7xzy/spectralai_a_project_to_use_nvidia_rt_cores_to/"> <img alt="Spectral-AI - a project to use Nvidia RT

<!-- SC_OFF --><div class="md"><p>Got Qwen 3.5 27B to 1,103,941 tok/s on 12 nodes with 96 B200 GPUs.</p> <p>At that rate you process 50,000 insurance policy documents in hours

<!-- SC_OFF --><div class="md"><p>Hardware-----------------<br /> 4070 12gb - gen 4.0 x16<br /> v100 16gb - gen 3.0 x4</p> <p>32gb 3200mhz ram</p> <p>it5-12400f<br /> ssd read -

<!-- SC_OFF --><div class="md"><p>Today I set up a full coding toolbox on a single RTX 5080 (with RAM offloading) that's actually viable.</p> <p><strong>Autocomplete</strong>:

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1t0jd95/poor_mans_guide_to_servicing_a_used_rtx_3090_for/"> <img alt="Poor man's guide to servicing a used

<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1tq35a0/zai_replaced_the_network_architecture_running/"> <img alt="Zai replaced the network architecture