Posts
All the articles I've posted.
-
genie-ai-runtime v1.1 → v1.3.1: Gemma 4 on a Jetson, from llama.cpp parity to beating it at depth
From v1.1 to v1.3.1: Gemma 4 E2B on the Jetson Orin Nano — decode past llama.cpp via split-K attention plus a TTFT win, all driven by the profiler.
-
I implement the papers, I don't just read them — a speculative-decoding field journal
How I actually learn new inference tech: build it. Training EAGLE3 and DFlash drafters for a 72B, the traps reading never warns you about, and DFlash v2.
-
The three layers of efficient AI — my roadmap from inference kernels to a physical AI chip
Efficient AI splits into algorithm, system, and application — the co-design loop across all three, and why my roadmap ends in a custom AI inference chip.
-
Developing a custom Qwen2.5 72B inference engine from scratch — what is the real bottleneck?
Custom Qwen2.5-72B engine on 8×H200: the real bottleneck isn't the 72B matmuls — it's the 160 all-reduce barriers per token. H200 roofline math, barrier vs. bandwidth anatomy, and why the engine anti-scales at TP=8.
-
Deploying VLA models on Jetson — what fits, what doesn't, and what TensorRT buys you
VLA deployment on Jetson Orin — what models fit in 8–64 GB unified memory, ORT-TRT EP 5.55×, SnapFlow distillation to get pi0.5 down to 6.5 GB, and the engineering traps that bite every fresh Jetson setup.
-
genie-ai-runtime: a Jetson edge LLM inference engine that beats llama.cpp — designing for unified memory
A CUDA LLM engine built for Jetson's unified memory: ~2× llama.cpp short-prompt prefill on Qwen3-4B, decode ceiling math, and a transformer vs Mamba vs hybrid edge benchmark.