Posts
All the articles I've posted.
-
Developing a custom Qwen2.5 72B inference engine from scratch — what is the real bottleneck?
Custom Qwen2.5-72B engine on 8×H200: the real bottleneck isn't the 72B matmuls — it's the 160 all-reduce barriers per token. H200 roofline math, barrier vs. bandwidth anatomy, and why the engine anti-scales at TP=8.
-
Deploying VLA models on Jetson — what fits, what doesn't, and what TensorRT buys you
VLA deployment on Jetson Orin — what models fit in 8–64 GB unified memory, ORT-TRT EP 5.55×, SnapFlow distillation to get pi0.5 down to 6.5 GB, and the engineering traps that bite every fresh Jetson setup.
-
genie-ai-runtime: a Jetson edge LLM inference engine that beats llama.cpp — designing for unified memory
A CUDA LLM engine built for Jetson's unified memory: ~2× llama.cpp short-prompt prefill on Qwen3-4B, decode ceiling math, and a transformer vs Mamba vs hybrid edge benchmark.
-
genie-claw: building a private on-device AI agent for Jetson — what is limited context size, and why it's the real engineering problem
On-device private home agent on Jetson Orin Nano 8 GB — what context size actually means, the KV cache memory math, 4096-token budget engineering, and BFCL accuracy from grounding.
-
AI agents vs. AI Inference: how they evolved and competed in 2026 — the Claw War, the fastest token, and what the race really is
2026: the OpenClaw wave, the Claw War between agent frameworks, and the inference arms race underneath — TTFT, throughput, and why fast tokens became the real competitive moat.
-
Nvidia Jetson as an intelligent Thread border router: ESP32-C6 driver port and the connected-home vision
Porting the ESP-Hosted-NG kernel driver to Jetson Orin Nano 8 GB: WiFi + BLE 5.0 over SPI from an ESP32-C6, and why 67 TOPS + IEEE 802.15.4 = an intelligent Thread border router.