Skip to content

Models, robotics, and ML systems — course review

Reviewed October 14, 2026. 39 course entry pages: 21 in Phase 3, 5 in robotics, and 13 in ML systems (including one lab index). Each entry received a prerequisite/output/sequence review and a specific transfer exercise. Long guides and lecture collections received targeted checks, not a claim that every command, model claim, or hardware lab was independently reproduced.

Substantive corrections include scalar/PyTorch gradient behavior, image-format bandwidth, Kalibr installation boundaries and geometric assumptions, edge latency claims, MLOps terminology, TF32/INT4/NVFP4 semantics, overlapped timing, scaling efficiency, unsupported quantization speedup targets, and VLA action/control distinctions. The rows below state the reviewed teaching boundary and handoff.

Course entry Reviewed exercise and handoff
Neural Networks Derive one forward pass and one gradient by hand. Compare finite differences away from nondifferentiable points, then deliberately transpose one weight to explain the shape failure. Handoff: A numerical reference, tensor-shape table, and measured training/inference distinction.
Module 2 — Deep Learning Frameworks Trace it in a scalar autograd engine, PyTorch, and a pinned tinygrad revision. Separate backward-graph construction from inference compilation; compare outputs before timing. Handoff: A framework-to-operator trace with shapes, dtypes, layouts, and live tensors.
PyTorch — Industry-Standard Deep Learning Framework Compare eager evaluation with one supported export/compile path. Include a second input shape and noncontiguous input; record guards, graph breaks, and numerical tolerance. Handoff: An executable reference plus an operator/shape inventory for compiler work.
PyTorch and micrograd — Insight for Mastering tinygrad Check shared-parameter gradient accumulation and the ReLU-at-zero convention. Construct a fresh scalar graph each training step; distinguish this educational engine from production autograd. Handoff: A gradient check and an explanation of forward, backward, and parameter-update costs.
tinygrad — The Hackable Compiler Framework Trace one matmul-plus-activation to IR and generated code. Change a dimension so a tile has a tail and verify output before comparing kernel count. Handoff: A minimal tensor program and correctness vectors for a custom backend.
Phase 3: Artificial Intelligence — The Workloads Your Hardware Must Run Complete the neural-network/framework core, then select the perception or application branch required by the task. Explain which courses can wait until a measured need appears. Handoff: A workload brief connecting model inputs, outputs, state, and deployment limits.
Computer Vision — Complete Guide Compare the reference preprocessing with one optimized implementation. Inject RGB/BGR reversal and wrong normalization; distinguish input-pipeline faults from model faults. Handoff: A preprocessing manifest and golden images feeding the policy observation adapter.
BEVFusion — Camera + LiDAR Fusion in Bird's Eye View Compare matched single-modality and fused runs on one split. Inject a transform offset and a missing sensor; report quality, memory, and latency rather than importing another board’s FPS. Handoff: A calibrated fusion workload and a modality-ablation report.
Sensor Fusion Work through estimation, calibration, association, then fusion. Inject time offset and spatial error separately so their residual patterns can be distinguished. Handoff: A frame/time/uncertainty contract that the robot observation pipeline can validate.
Kalibr — Multi-Sensor Calibration for Autonomous Vehicles Fit on one trajectory and validate on another. Verify transform direction, units, clock convention, and timestamp preservation through any bag conversion. Handoff: Calibration YAML, held-out residuals, and provenance for the fusion pipeline.
Kalman Filter Learning Series Compare prediction-only, measurement-only, and filtered estimates. Double the sample interval and introduce dropout; inspect innovation and covariance as well as state error. Handoff: An estimator with documented time-step and uncertainty assumptions.
Multi-Object Tracking: Hungarian Algorithm + Kalman Filter Hold detector outputs fixed while changing assignment gating. Create a crossing/occlusion sequence; measure identity switches and track recovery rather than only detector mAP. Handoff: Timestamped tracks with lifecycle rules and identity metrics.
Module 6A — Voice AI Measure buffering, endpointing, recognition, and dispatch separately. Test silence, noise, and an expired command; a transcript must pass an intent adapter before a robot task is requested. Handoff: An optional language-command interface with latency, expiry, and rejection behavior.
Edge AI Optimize one supported component and repeat under sustained load. Include preprocessing, queueing, and action dispatch; a fast kernel alone does not establish a fresh action. Handoff: A quality/timing/memory report ready for target deployment.
AI Agent Development 2026 Build a planner that requests task intents with typed arguments and timeouts. Replay malformed or stale requests and retrieved text that attempts to alter permissions. Handoff: An optional supervisory planner with an auditable trace; local controller execution remains separately scheduled.
AI Agent Development 2026 — Lecture Index Use the flat index to locate a lecture, then return to the guide for prerequisites. Preserve one trace across model call, tool request, result, and retry rather than building unrelated demos. Handoff: A linked set of agent lab artifacts that can be evaluated as one runtime.
MCP for AI Agents — The Model Context Protocol in Depth Expose observation metadata and task-status tools first. Test schema errors, cancellation, authorization, and duplicate requests; protocol support alone does not authorize an actuator operation. Handoff: A versioned supervisory tool interface and failure/retry transcript.
Module 4B — ML Engineering & MLOps Build a release candidate with quality, timing, memory, and compatibility gates. Rehearse rollback and prove that restoring weights also restores normalization and runtime configuration. Handoff: A reproducible release manifest and rollback result for the robot policy.
Practical Machine Learning — adapted from Stanford CS329P Split by the unit that must generalize. Compare with a random-frame split to expose leakage; fit normalization only on training data, then evaluate distribution shift. Handoff: A defensible held-out evaluation protocol used by VLA training.
Module 5B — LLM Application Development Separate retrieval quality, model quality, and tool-result correctness. Keep long-context planning outside the controller tick and test an unavailable remote endpoint. Handoff: A measured optional planning service and a local fallback contract.
Qwen3.5-4B-Base Fine-Tuning with Unsloth Compare the unchanged base, adapter, and exported artifact on the same split. Record which graph components are trained; text instruction tuning alone does not create an action policy. Handoff: An adapter/export report and a list of missing action data, head, and embodiment contracts before VLA use.
Deep RL for Robot Learning — Special Course Compare training methods at a fixed environment-step and compute budget. Keep evaluation seeds/scenes separate, check reward hacking, and report multiple seeds rather than a best run. Handoff: A frozen policy and training provenance for independent deployment evaluation.
Phase 5 — Track D: Robotics (12-18 months) Learn ROS 2 integration, establish an Isaac baseline, then study the matching VLA family. Add RL or multi-robot work only when it answers a demonstrated task limitation. Handoff: One working robot loop with explicit model/controller timing boundaries.
Isaac Sim and Isaac Lab — Special Course Vary physics step, sensor rate, and policy decimation independently. Compare simulated time with wall time, then inject delayed observations while measuring task outcome. Handoff: An environment contract, reset test, and observation-to-action traces.
VLA Foundations to Frontier 2026 — Professional Course Follow history and architecture into action representation, data, training, evaluation, and deployment. Trace a complete action chunk through normalization, frame conversion, expiry, and controller consumption. Handoff: A workload dossier and closed-loop reference that constrain the future accelerator.
VLA Optimization and Action-Parity Harness — Special Course Optimize one component at a time. Compare same-observation numerics and paired closed-loop outcomes separately; test stale observations and recovery as well as average success. Handoff: A passing candidate plus measured critical-path operator shapes for compiler and accelerator design.
Part 1 — Fundamentals of AI Inference / MLSys Trace prefill and decode for explicit batch/context shapes. Derive weight and state traffic, then explain when compute, bandwidth, or scheduling dominates. Handoff: A baseline benchmark with numerical and resource assumptions for dense serving.
Part 2 — Dense Decoder-Only Inference at Hopper Compare supported quantization and parallelism candidates on matched requests. Do not force AWQ to win; use data to choose. Without large GPUs, complete config/math work and mark throughput unmeasured. Handoff: A dense-model quality/capacity/latency comparison before studying sparse routing.
Part 3 — MoE Inference at Blackwell Separate total parameter storage from active expert compute. Compare one supported placement and trace communication; reject unsupported model/format/kernel combinations before benchmarking. Handoff: Per-device capacity and traffic budgets that transfer to a named frontier checkpoint.
Part 4 — Optimizing a Real Engine Treat published PR numbers as historical evidence. Reproduce one mechanism on a tractable subgraph, checking numerical tolerance and paired performance rather than inferring correctness from greedy decoding. Handoff: An optimization report with attribution, measured changes, and unresolved portability.
Part 5 — DeepSeek V4.1 Flash NVFP4: inference optimization engineering Audit payload/scales, total versus active parameters, hybrid state, and parallel placement. A text-output multimodal model is an inference case study or optional planner; it is not a native robot action policy. Handoff: A checkpoint/memory/kernel dossier; transfer supported numerical and memory lessons to the selected VLA.
Inference lab tools Run metadata, storage, placement, and benchmark-validation exercises on CPU. Use malformed scale or placement data as a negative control; distinguish fixture validation from loading the full checkpoint. Handoff: Machine-readable audit results and explicit hardware experiments still to run.
AI Inference Engineer 2026 — Special Course Progress from fundamentals to dense/MoE cases and one real-engine optimization, then audit DeepSeek. Carry the same quality and timing definitions through all five parts. Handoff: An inference evidence bundle reusable by the physical-AI design studio.
GLM-5.3-Flash Architecture Mastery Compare recurrent and chunked execution, including final state. Test speculative rejection/rollback for all state types; output agreement on one prefix is insufficient. Handoff: A hybrid-state ABI and reference vectors informing programmable memory/state support.
Phase 5 — Track G: ML Systems Engineering Build a dependency graph and identify its critical path. Change a tile schedule to expose overlap; keep scaling efficiency, device utilization, and application throughput distinct. Handoff: A bottleneck explanation that selects the next kernel, compiler, or serving experiment.
Hardware-Aware LLM Quantization & Inference Optimization Derive packed bytes including scales and unchanged tensors. Pair each precision change with quality and dispatch evidence; combine optimizations only after measuring them individually. Handoff: A precision/layout contract with error budgets for the compiler and accelerator.
Logprobs, Perplexity & KL Divergence — The Information Theory of LLM Inference Compute masked-token NLL, perplexity, and KL with an explicit direction. Separate teacher-distribution KL from empirical data loss; add task metrics because low KL does not prove robot success. Handoff: A numerical quality grader reused across quantization candidates.
MLSys Deep Dives — The 2026 Machine-Learning-Systems Landscape Choose one kernel, scheduling, architectural, or speculative change to test. Count cost over the same interval as accepted outputs; distinguish commercial prices from engineering cost. Handoff: A measured optimization ladder with combined results and costs.
TVM Deep Dives — Apache TVM for MLSys Engineers Lower to a pinned compiler/runtime target, preserve shape/layout semantics, and test odd sizes. Inspect generated code and buffer lifetimes before claiming fusion or reuse. Handoff: A compiler/runtime ABI, generated kernel, and golden vectors for the chip-design capstone.