Archives
All the articles I've archived.
-
Proof you can't fake: shipping dual-vendor Intel TDX + NVIDIA CC attestation for AI training claims
I closed every trust gap in a real proof-of-training system: TDX measured-VM quotes, Intel DCAP signature verification, and NVIDIA JWKS-verified GPU tokens — live, not claimed.
-
Distilling frontier reasoning into an edge model — Unsloth + W&B in Colab, Axolotl on Blackwell
Distill Opus 4.6 / Qwen3.5 reasoning into ~14k SFT samples; prototype on Llama 3.2 3B in Colab with Unsloth + W&B, then Axolotl on RTX PRO 6000 Blackwell.
-
TRT-LLM vs vLLM vs llama.cpp on Blackwell — sparkinfer v0.3.7 owns decode on the RTX 5090
sparkinfer v0.3.7: ~30% faster decode than llama.cpp on RTX 5090 at the same LLM quality — 2.5 MB native binary. I develop and maintain it. Plus TRT-LLM vs vLLM vs llama.cpp on PRO 6000.
-
The confidential API: sending a prompt to a cloud GPU the provider can't read
HTTPS protects the wire, not the server. A confidential API binds an attested TEE to an ECDH session key — so prompts stay encrypted even from the provider.
-
Trust before secrets: confidential AI inference with CPU TEEs and GPU Confidential Computing
Renting a GPU means trusting someone else's host. Confidential computing flips it: attest the CPU TEE and GPU, then release API keys only to a verified CVM.
-
Training a DFlash drafter on a B200 — the real grind behind block-diffusion speculative decoding
I trained a DFlash drafter for a 72B on 2×B200 with the latest SGLang: the version-drift tax, the warm-start trap, accept 1.49→1.71, and what's next.
-
genie-ai-runtime v1.1 → v1.3.1: Gemma 4 on a Jetson, from llama.cpp parity to beating it at depth
From v1.1 to v1.3.1: Gemma 4 E2B on the Jetson Orin Nano — decode past llama.cpp via split-K attention plus a TTFT win, all driven by the profiler.
-
I implement the papers, I don't just read them — a speculative-decoding field journal
How I actually learn new inference tech: build it. Training EAGLE3 and DFlash drafters for a 72B, the traps reading never warns you about, and DFlash v2.
-
The three layers of efficient AI — my roadmap from inference kernels to a physical AI chip
Efficient AI splits into algorithm, system, and application — the co-design loop across all three, and why my roadmap ends in a custom AI inference chip.
-
Developing a custom Qwen2.5 72B inference engine from scratch — what is the real bottleneck?
Custom Qwen2.5-72B engine on 8×H200: the real bottleneck isn't the 72B matmuls — it's the 160 all-reduce barriers per token. H200 roofline math, barrier vs. bandwidth anatomy, and why the engine anti-scales at TP=8.
-
Deploying VLA models on Jetson — what fits, what doesn't, and what TensorRT buys you
VLA deployment on Jetson Orin — what models fit in 8–64 GB unified memory, ORT-TRT EP 5.55×, SnapFlow distillation to get pi0.5 down to 6.5 GB, and the engineering traps that bite every fresh Jetson setup.
-
genie-ai-runtime: a Jetson edge LLM inference engine that beats llama.cpp — designing for unified memory
A CUDA LLM engine built for Jetson's unified memory: ~2× llama.cpp short-prompt prefill on Qwen3-4B, decode ceiling math, and a transformer vs Mamba vs hybrid edge benchmark.
-
genie-claw: building a private on-device AI agent for Jetson — what is limited context size, and why it's the real engineering problem
On-device private home agent on Jetson Orin Nano 8 GB — what context size actually means, the KV cache memory math, 4096-token budget engineering, and BFCL accuracy from grounding.
-
AI agents vs. AI Inference: how they evolved and competed in 2026 — the Claw War, the fastest token, and what the race really is
2026: the OpenClaw wave, the Claw War between agent frameworks, and the inference arms race underneath — TTFT, throughput, and why fast tokens became the real competitive moat.
-
Nvidia Jetson as an intelligent Thread border router: ESP32-C6 driver port and the connected-home vision
Porting the ESP-Hosted-NG kernel driver to Jetson Orin Nano 8 GB: WiFi + BLE 5.0 over SPI from an ESP32-C6, and why 67 TOPS + IEEE 802.15.4 = an intelligent Thread border router.
-
Seven hard-won lessons from putting a zk-STARK prover on the Blackwell RTX PRO 6000 — 245 hand-written CUDA kernels
245 hand-written CUDA kernels on the RTX PRO 6000 Blackwell: seven performance-engineering lessons from GPU-accelerating a zk-STARK prover — profiling, memory, PTX, registers, and what Blackwell changed.
-
Why I dropped UltraScale+ FPGAs for a single L40S — TIP5 mining on a GPU in one month, not six
After six months on FPGAs, a single NVIDIA L40S hit the TIP5 mining target in one month — and why the Goldilocks field maps so naturally onto CUDA.
-
A UART-driven FPGA miner in pure PL — TIP5 on three VU35P UltraScale+, no soft processor, programmed through a Zynq 7010
A pure-PL TIP5 miner on three VU35P UltraScale+ FPGAs (Osprey E300): soft UART, AXI master, and control FSM with no processor, programmed via a Zynq 7010.
-
Parallel TIP5 hashing on AMD Virtex UltraScale+ FPGAs — designing a Neptune Cash miner on AWS EC2 F2
Designing a TIP5 hash accelerator on AWS EC2 F2 Virtex UltraScale+ — the hard part was the Shell/CL PCIe interface: AppPF BARs and an AXI4-Lite register map.