Skip to content

TVM Deep Dives — compile, inspect, and test a computation

Learn how an ML compiler represents a tensor computation, changes its execution schedule, and packages a runnable artifact. Start on a CPU with tiny inputs whose answers you can check. GPU tuning, custom accelerator integration, and embedded deployment are extensions that require their own hardware and toolchain evidence.

The executable compiler lab compiles a 37-element affine operation and a small matmul+bias+ReLU model with Apache TVM's LLVM CPU backend. It checks vector tails, wrong shapes, output values, and real shared-library export/reload. No pretrained weights or GPU are required.

Prerequisites and route

You should be comfortable with Python, matrix multiplication, array shapes, and a loop. If compiler IR is new, first complete the CPU compiler introduction. Allow roughly one session per lecture; spend longer on the optional tuning/hardware extensions when available.

Lesson Question Artifact
01 — compilation flow What changes between graph, tensor program, and machine code? Original/lowered IR plus checked CPU output
02 — schedules Which loop changes preserve meaning and handle tails? Vectorized 37-element kernel and boundary tests
03 — tuning When does search repay its cost? Candidate timings, correctness checks, and reuse assumptions
04 — graph optimization When is fusion legal and worthwhile? Shape/layout contract and a fusion/BYOC decision
05 — deployment What must travel with a compiled artifact? Exported/reloaded CPU module and deployment manifest

The outcome is a reproducible compiler experiment, not a required win over a framework. A framework may already call an excellent library kernel or use its own graph compiler. A result showing that TVM loses on a shape is useful when explained and measured fairly.

Version map, checked 2026-10-18

The lab pins apache-tvm 0.27.0.post1, installed from PyPI and actually executed. Its API uses tvm.tirx, tvm.s_tir, from tvm.script import tirx as T, T.sblock, and Schedule.get_sblock. The moving upstream Quick Start currently uses the separately named tvm_tirx package for several core pieces. That package is not present in the tested release wheel. These are different snapshots, not interchangeable import examples.

Material How to use it
This lab Install its exact requirements and run its complete files
Latest website examples Match the documented source/package revision before adapting
Older tvm.tir, tvm.meta_schedule, T.block, get_block examples Historical APIs; namespace replacement alone may be insufficient
Relay/GraphExecutor/AutoTVM/Ansor/microTVM tutorials Use an archived compatible checkout and explain the historical executor/toolchain
MLC-LLM Use its own documented compiler dependency and target packages; arbitrary TVM wheel compatibility is not established

Inspect tvm.__version__, tvm.__file__, build options, and exported IR before debugging a tutorial. A build option such as USE_CUDA=ON in a wheel does not mean a CUDA device was present or executed.

What makes an experiment convincing

Use deterministic inputs, an independent numerical reference, odd shapes and tails, and an explicit dtype/layout contract. Record compiler/package versions, target, compile time, warmup, timing method, output error, and artifact hashes. Keep candidate correctness separate from performance ranking. Tiny kernels may measure Python/FFI overhead rather than useful hardware throughput.

TVM is one compiler family among several. XLA has GPU autotuning and persisted tuning caches; describing it as “no autotuning” is incorrect. PyTorch can compile graphs, and TensorRT, IREE, and MLC have their own supported workloads and deployment boundaries. Compare concrete versions and tasks rather than assigning a universal maturity ranking.

Finish by explaining one transformation, one caught failure, and one measured limitation. Then continue to AI chip design or inference engineering with those contracts intact.