Is there any way how to run NVFP4 model on Windows without WSL?
Someone on LocalLLaMA asked a question that sounds deceptively simple: is there any way to run an NVFP4 model on Windows without going through WSL? The post scored a modest 2.0 and drew little wider attention, which tells you something about where this sits in the community's priority list right now. NVFP4 — NVIDIA's 4-bit floating point quantisation format — has been generating quiet interest among local inference enthusiasts since NVIDIA began positioning it as a path to running larger models on consumer hardware. The problem is that most of the accessible tooling, the pipelines that actually work, have been built with Linux-first assumptions. WSL2 bridges the gap for many users, but it adds friction, it adds memory overhead, and for anyone running a tightly configured Windows system, it can feel like bolting a bicycle wheel onto a car. The question being asked is whether that detour is actually necessary, or whether native Windows support has quietly caught up.
If confirmed as a genuine gap rather than a solved problem, this matters more than the thread's low score suggests. Windows remains the dominant desktop operating system. The local AI movement has grown precisely because enthusiasts want to run models on hardware they already own, on operating systems they already use. Every additional step between a user and a working inference setup is a dropout point. If NVFP4 support genuinely requires WSL on Windows, NVIDIA and the broader open-source tooling community are leaving a significant slice of motivated users in a frustrating middle ground — hardware capable, software blocked. The second-order effect is that Windows users may simply default to older quantisation formats that do have native support, which slows adoption of newer, more efficient formats and fragments the ecosystem at exactly the moment it should be consolidating.
Watch for any updates to llama.cpp, Ollama, or LM Studio announcing native Windows CUDA support for FP4 inference — that would resolve this cleanly, and the silence around it right now is itself worth noting.
<!-- SC_OFF --><div class="md"><p>I have been using it through vllm and faster than other quant types for my RTX 5060ti. Do we have this in llama.cpp yet ?</p> </div><!-- SC_ON
<!-- SC_OFF --><div class="md"><p>so recently i got to know that we can use vscode terimal + claude code + ollama models<br /> and i tried doing that it was great but im running
<!-- SC_OFF --><div class="md"><p>All repositories either have compiling bugs in Windows or there's zero instructions to compiling at all.</p> </div><!-- SC_ON -->   submitted
<!-- SC_OFF --><div class="md"><p>Anyone running into CUDA issues on newer GPUs (sm120)?</p> <p>Tried building llama.cpp with CUDA targeting sm_120 and couldn’t get a clean
<!-- SC_OFF --><div class="md"><p>so as it says, I am not a FE eng, but want to do some light FE work<br /> I don't need the smartest model but need to get some work done.<br /> I
<!-- SC_OFF --><div class="md"><p>I am amazed by the development of locallm in the coding area. Right now im Testing Qwen3.6 27b and it works quite well, even tho this is not made
<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1t4uj9h/mtp_on_strix_halo_with_llamacpp_pr_22673/"> <img alt="MTP on strix halo with llama.cpp (PR #22673)"
<!-- SC_OFF --><div class="md"><p>So I’m testing several models on macOS and I’d like to understand if NVFP4 is the best option to run 4bit models quantized models using mlx. From
NewsHive monitors these sources continuously. All signal titles above link to the original reporting.
Intelligence by NewsHive. Need help navigating what this means for your business? Contact GeekyBee →