Local AI Model Selection

<!-- SC_OFF --><div class="md"><p><a

<!-- SC_OFF --><div class="md"><p>Currently using glm 4.7 flash, it’s very meh </p> <p>Heard omnicoder or Crow 9b are good, are they any better?</p> <p>Or Qwen3.5 27b?</p>

<!-- SC_OFF --><div class="md"><p>Tomorrow, I am getting delivery of a 13&quot; M5 macbook air with 32gb RAM and 1T SSD. Currently, I have pro subscriptions to Gemini, Perplexity

<!-- SC_OFF --><div class="md"><p>Today I set up a full coding toolbox on a single RTX 5080 (with RAM offloading) that's actually viable.</p> <p><strong>Autocomplete</strong>:

<!-- SC_OFF --><div class="md"><p>Went down the local LLM rabbit hole. Looked at P40s, V100s (almost bought an SXM2 version that doesn’t even plug into a normal motherboard lmao),

<!-- SC_OFF --><div class="md"><p>Running Llama-3-70B with long contexts and hitting HBM limits constantly. 128K context burns approx 40GB just for KV cache — half the H100 gone

<!-- SC_OFF --><div class="md"><p>So this is probably a pretty common thing, but I just want to ask in case am not missing something. I have pretty much no knowledge but trying to

<!-- SC_OFF --><div class="md"><p>I could fit the Q3 model not sure if it's worth over 27B ? </p> </div><!-- SC_ON --> &#32; submitted by &#32; <a