<!-- SC_OFF --><div class="md"><p>Hardware: RTX 5070Ti + RTX 5060Ti</p> <p>llama.cpp command:</p> <p>./llama.cpp/build/bin/llama-server -m
<!-- SC_OFF --><div class="md"><p>Hi,</p> <p>I taught Computer Science for 30 years in a French School of Electrical Engineering, Computer Science Department.</p> <p>I recently
<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1s2bne8/context_shifting_sliding_window_rag/"> <img alt="Context Shifting + sliding window + RAG"
<!-- SC_OFF --><div class="md"><p>So i recently updated LMstudio after a long pause and updated my llama.cpp runtimes too.. i was shocked.. i thought maybe something like
<!-- SC_OFF --><div class="md"><p>How do you guys deal with long context, for example while coding, when you’re going back and forth for adjustments or fixing some errors and
<!-- SC_OFF --><div class="md"><p>Every output token the model generates requires reading the entire KV cache from High Bandwidth Memory.</p> <p>Here's what that actually looks
<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1s2jd8a/using_anythingllm_with_ollama_but_when_i_do/"> <img alt="Using AnythingLLM with Ollama, but when i
<!-- SC_OFF --><div class="md"><p>I noticed on the same hardware that I can get 24 x 128k contexts with muse glimmer (30b q8_0 + mmproj+dflash) only gets me 3x 256k or 6x 128k
NewsHive monitors these sources continuously. All signal titles above link to the original reporting.
Intelligence by NewsHive. Need help navigating what this means for your business? Contact GeekyBee →