What's the best way to run an embedding model on the Apple Neural Engine?

<!-- SC_OFF --><div class="md"><p>Hello!<br /> Does anyone tried text embedding models on Jetson Nano 2/4Gb? I need it for the RAG.<br /> I want to know the speed. For example

<!-- SC_OFF --><div class="md"><p>Trying to auto categorize a former employees photos from personal and work related. It’s a lot of photos and I don’t want the guy to loose

<!-- SC_OFF --><div class="md"><p>Hi everyone, so I have been playing around with the software side and an RTX 3090, but I'm wondering what hardware I could experiment with to get

<!-- SC_OFF --><div class="md"><p>I need SOTA ai at like 2k TPS with tiny latency so that I can get time to first answer token under 3 seconds for real time replies with full COT

<!-- SC_OFF --><div class="md"><p>Hi. What do you think is the smallest Open model, as of Spring 2026, that can match/beat OpenAI GPT 4o Mini in the broadest sense possible?</p>

<!-- SC_OFF --><div class="md"><p>Hi everyone,</p> <p>I’m currently working on a project to build a <strong>100% local AI chatbot</strong> for a government-related use case

<a href="https://news.ycombinator.com/item?id=48794803">Comments</a>

<!-- SC_OFF --><div class="md"><p>I have experience using llama.cpp on Windows/Linux with 8GB NVIDIA card (384 GB/s bandwidth) and offloading to CPU to run MoE models. I typically