Tag: performance
All the articles with the tag "performance".
-
Developing a custom Qwen2.5 72B inference engine from scratch — what is the real bottleneck?
Custom Qwen2.5-72B engine on 8×H200: the real bottleneck isn't the 72B matmuls — it's the 160 all-reduce barriers per token. H200 roofline math, barrier vs. bandwidth anatomy, and why the engine anti-scales at TP=8.
-
genie-ai-runtime: a Jetson edge LLM inference engine that beats llama.cpp — designing for unified memory
A CUDA LLM engine built for Jetson's unified memory: ~2× llama.cpp short-prompt prefill on Qwen3-4B, decode ceiling math, and a transformer vs Mamba vs hybrid edge benchmark.
-
Seven hard-won lessons from putting a zk-STARK prover on the Blackwell RTX PRO 6000 — 245 hand-written CUDA kernels
245 hand-written CUDA kernels on the RTX PRO 6000 Blackwell: seven performance-engineering lessons from GPU-accelerating a zk-STARK prover — profiling, memory, PTX, registers, and what Blackwell changed.
-
Why I dropped UltraScale+ FPGAs for a single L40S — TIP5 mining on a GPU in one month, not six
After six months on FPGAs, a single NVIDIA L40S hit the TIP5 mining target in one month — and why the Goldilocks field maps so naturally onto CUDA.