Skip to content

PyTorch — Industry-Standard Deep Learning Framework

PISD

Deep Dive · AI Workloads

Specialized course identity for PyTorch — Industry-Standard Deep Learning Framework.

Artifact: model or workload study · Measure: accuracy, latency, memory, throughput

Parent: Module 2 — Deep Learning Frameworks

The framework most models are written in. You need fluency here because many deployment workflows begin with a PyTorch reference.


What to Master

Concept Why it matters for hardware
torch.Tensor The data structure accelerators process. Shape, dtype, layout (contiguous, channels-last).
nn.Module How models are structured. Layers → forward pass → computational graph.
Autograd (loss.backward()) Generates the backward graph that training hardware executes.
Data loading (DataLoader) CPU-GPU pipeline. Bottleneck if not overlapped with compute.
torch.onnx.export() How models leave PyTorch and enter the compiler/runtime stack (Phase 4C).
torch.compile() Compilation interface; backend/device determine generated code and library calls.
torch.profiler Where is time spent? Kernel launches, memory copies, CPU overhead.
Mixed precision (torch.amp) FP16/BF16 training — what tensor cores accelerate.
Quantization (torchao) Low-precision representations and supported kernels; verify dtype, operator, and device coverage.

API currency, reviewed 2026-10-18: PyTorch documents the migration of quantization development to torchao. Treat older torch.ao.quantization recipes as version-specific migration material. Pin compatible PyTorch/torchao versions and check backend support; a smaller weight file does not establish a faster runtime.

Start with a correct reference

Before the larger projects, reproduce the CPU policy lab, then port its three-parameter model to nn.Linear(2, 1). Preserve the episode split, normalization, and held-out metrics. Match predictions against the plain-Python reference on a fixed batch before benchmarking.

Keep these operations separate:

  • model.train() / model.eval() change behavior of modules such as dropout and BatchNorm.
  • torch.no_grad() or torch.inference_mode() controls gradient recording. eval() does not disable autograd.
  • optimizer.zero_grad() clears accumulated parameter gradients; backward() computes them; step() updates parameters.

The autograd documentation explains the distinction. Self-check: evaluating under no_grad() while leaving dropout in training mode still produces training-mode dropout behavior. Use the correct model mode and gradient context together.

First artifact: predictions and parameter shapes, one numerical tolerance check, a train/validation/test manifest, and a deliberately swapped feature column that the reference comparison catches. The CPU route validates model semantics; GPU timing remains a separate experiment.

Run the complete CPU practical

The robot-stop classification lab uses actual PyTorch 2.14 CPU training and torch.export. It includes an exact dependency lock for the tested platform, reproducible synthetic episodes, a nonlinear model, two baselines, threshold selection, error slices, serialization parity, and a deliberate distribution shift. Start here before the larger CNN projects.

For AMP, use torch.amp.autocast(device_type=...) and a device-appropriate scaler where needed; do not mechanically enable FP16 on every CPU. Current AMP documentation describes supported devices/dtypes. torch.export captures an operator graph with input constraints; torch.compile prepares optimized execution; ONNX export adds another representation/runtime boundary. Validate outputs at each boundary and include preprocessing in the contract.

Projects

  1. Train ResNet-18 on CIFAR-10 from scratch. Profile with torch.profiler. Identify top-3 time-consuming ops.
  2. Export to ONNX. Visualize graph with Netron. Count total ops and parameters.
  3. Post-training quantization to INT8. Measure accuracy drop and inference speedup.
  4. torch.compile() on a transformer block. Compare eager vs compiled execution time.

Resources

Carry this course into the physical-AI system

Start from: A reference model and fixed input batch.

Build and challenge: Compare eager evaluation with one supported export/compile path. Include a second input shape and noncontiguous input; record guards, graph breaks, and numerical tolerance.

Pass forward: An executable reference plus an operator/shape inventory for compiler work. Continue with the connected course and keep the artifact in the Physical AI Design Studio.