Tag: quantization
All the articles with the tag "quantization".
-
How we got Qwen3.8-27B to 92.9 tok/s and a 360K window on one RTX 5090
The engineering behind our NVFP4 build of Qwen3.8-27B: why decode is a bandwidth problem, why lm_head was the tensor that mattered, and the exact environment it was built and measured in.
-
830K downloads later: what quantizing Qwen3.8-27B to NVFP4 for one RTX 5090 actually taught me
Our NVFP4 build of Qwen3.8-27B passed 830K monthly downloads. Then NVIDIA and two others published their own. I benchmarked all three on one 5090 — quality was a tie, and what separated them was precision allocation, a drafter trained against the checkpoint, and the runtime underneath.