Show HN: Fine-tune an 8B model on a 4 GB laptop GPU

<a href="https://news.ycombinator.com/item?id=47476422">Comments</a>

<a href="https://news.ycombinator.com/item?id=47504695">Comments</a>

<a href="https://news.ycombinator.com/item?id=49346135">Comments</a>

<table> <tr><td> <a href="https://www.reddit.com/r/MachineLearning/comments/1unl62q/if_your_gpu_can_run_inference_it_should_be_able/"> <img alt="If your GPU can run inference, it

<!-- SC_OFF --><div class="md"><p>This is nuts.</p> <p><a href="https://huggingface.co/prism-ml/Bonsai-8B-gguf">prism-ml/Bonsai-8B-gguf · Hugging Face</a></p> <p>has anyone tested

<!-- SC_OFF --><div class="md"><p>Every output token the model generates requires reading the entire KV cache from High Bandwidth Memory.</p> <p>Here's what that actually looks

<a href="https://news.ycombinator.com/item?id=48935687">Comments</a>

<a href="https://news.ycombinator.com/item?id=49898931">Comments</a>