Models, robotics, and ML systems — course review¶
Reviewed October 14, 2026. 39 course entry pages: 21 in Phase 3, 5 in robotics, and 13 in ML systems (including one lab index). Each entry received a prerequisite/output/sequence review and a specific transfer exercise. Long guides and lecture collections received targeted checks, not a claim that every command, model claim, or hardware lab was independently reproduced.
Substantive corrections include scalar/PyTorch gradient behavior, image-format bandwidth, Kalibr installation boundaries and geometric assumptions, edge latency claims, MLOps terminology, TF32/INT4/NVFP4 semantics, overlapped timing, scaling efficiency, unsupported quantization speedup targets, and VLA action/control distinctions. The rows below state the reviewed teaching boundary and handoff.
| Course entry | Reviewed exercise and handoff |
|---|---|
| Neural Networks | Derive one forward pass and one gradient by hand. Compare finite differences away from nondifferentiable points, then deliberately transpose one weight to explain the shape failure. Handoff: A numerical reference, tensor-shape table, and measured training/inference distinction. |
| Module 2 — Deep Learning Frameworks | Trace it in a scalar autograd engine, PyTorch, and a pinned tinygrad revision. Separate backward-graph construction from inference compilation; compare outputs before timing. Handoff: A framework-to-operator trace with shapes, dtypes, layouts, and live tensors. |
| PyTorch — Industry-Standard Deep Learning Framework | Compare eager evaluation with one supported export/compile path. Include a second input shape and noncontiguous input; record guards, graph breaks, and numerical tolerance. Handoff: An executable reference plus an operator/shape inventory for compiler work. |
| PyTorch and micrograd — Insight for Mastering tinygrad | Check shared-parameter gradient accumulation and the ReLU-at-zero convention. Construct a fresh scalar graph each training step; distinguish this educational engine from production autograd. Handoff: A gradient check and an explanation of forward, backward, and parameter-update costs. |
| tinygrad — The Hackable Compiler Framework | Trace one matmul-plus-activation to IR and generated code. Change a dimension so a tile has a tail and verify output before comparing kernel count. Handoff: A minimal tensor program and correctness vectors for a custom backend. |
| Phase 3: Artificial Intelligence — The Workloads Your Hardware Must Run | Complete the neural-network/framework core, then select the perception or application branch required by the task. Explain which courses can wait until a measured need appears. Handoff: A workload brief connecting model inputs, outputs, state, and deployment limits. |
| Computer Vision — Complete Guide | Compare the reference preprocessing with one optimized implementation. Inject RGB/BGR reversal and wrong normalization; distinguish input-pipeline faults from model faults. Handoff: A preprocessing manifest and golden images feeding the policy observation adapter. |
| BEVFusion — Camera + LiDAR Fusion in Bird's Eye View | Compare matched single-modality and fused runs on one split. Inject a transform offset and a missing sensor; report quality, memory, and latency rather than importing another board’s FPS. Handoff: A calibrated fusion workload and a modality-ablation report. |
| Sensor Fusion | Work through estimation, calibration, association, then fusion. Inject time offset and spatial error separately so their residual patterns can be distinguished. Handoff: A frame/time/uncertainty contract that the robot observation pipeline can validate. |
| Kalibr — Multi-Sensor Calibration for Autonomous Vehicles | Fit on one trajectory and validate on another. Verify transform direction, units, clock convention, and timestamp preservation through any bag conversion. Handoff: Calibration YAML, held-out residuals, and provenance for the fusion pipeline. |
| Kalman Filter Learning Series | Compare prediction-only, measurement-only, and filtered estimates. Double the sample interval and introduce dropout; inspect innovation and covariance as well as state error. Handoff: An estimator with documented time-step and uncertainty assumptions. |
| Multi-Object Tracking: Hungarian Algorithm + Kalman Filter | Hold detector outputs fixed while changing assignment gating. Create a crossing/occlusion sequence; measure identity switches and track recovery rather than only detector mAP. Handoff: Timestamped tracks with lifecycle rules and identity metrics. |
| Module 6A — Voice AI | Measure buffering, endpointing, recognition, and dispatch separately. Test silence, noise, and an expired command; a transcript must pass an intent adapter before a robot task is requested. Handoff: An optional language-command interface with latency, expiry, and rejection behavior. |
| Edge AI | Optimize one supported component and repeat under sustained load. Include preprocessing, queueing, and action dispatch; a fast kernel alone does not establish a fresh action. Handoff: A quality/timing/memory report ready for target deployment. |
| AI Agent Development 2026 | Build a planner that requests task intents with typed arguments and timeouts. Replay malformed or stale requests and retrieved text that attempts to alter permissions. Handoff: An optional supervisory planner with an auditable trace; local controller execution remains separately scheduled. |
| AI Agent Development 2026 — Lecture Index | Use the flat index to locate a lecture, then return to the guide for prerequisites. Preserve one trace across model call, tool request, result, and retry rather than building unrelated demos. Handoff: A linked set of agent lab artifacts that can be evaluated as one runtime. |
| MCP for AI Agents — The Model Context Protocol in Depth | Expose observation metadata and task-status tools first. Test schema errors, cancellation, authorization, and duplicate requests; protocol support alone does not authorize an actuator operation. Handoff: A versioned supervisory tool interface and failure/retry transcript. |
| Module 4B — ML Engineering & MLOps | Build a release candidate with quality, timing, memory, and compatibility gates. Rehearse rollback and prove that restoring weights also restores normalization and runtime configuration. Handoff: A reproducible release manifest and rollback result for the robot policy. |
| Practical Machine Learning — adapted from Stanford CS329P | Split by the unit that must generalize. Compare with a random-frame split to expose leakage; fit normalization only on training data, then evaluate distribution shift. Handoff: A defensible held-out evaluation protocol used by VLA training. |
| Module 5B — LLM Application Development | Separate retrieval quality, model quality, and tool-result correctness. Keep long-context planning outside the controller tick and test an unavailable remote endpoint. Handoff: A measured optional planning service and a local fallback contract. |
| Qwen3.5-4B-Base Fine-Tuning with Unsloth | Compare the unchanged base, adapter, and exported artifact on the same split. Record which graph components are trained; text instruction tuning alone does not create an action policy. Handoff: An adapter/export report and a list of missing action data, head, and embodiment contracts before VLA use. |
| Deep RL for Robot Learning — Special Course | Compare training methods at a fixed environment-step and compute budget. Keep evaluation seeds/scenes separate, check reward hacking, and report multiple seeds rather than a best run. Handoff: A frozen policy and training provenance for independent deployment evaluation. |
| Phase 5 — Track D: Robotics (12-18 months) | Learn ROS 2 integration, establish an Isaac baseline, then study the matching VLA family. Add RL or multi-robot work only when it answers a demonstrated task limitation. Handoff: One working robot loop with explicit model/controller timing boundaries. |
| Isaac Sim and Isaac Lab — Special Course | Vary physics step, sensor rate, and policy decimation independently. Compare simulated time with wall time, then inject delayed observations while measuring task outcome. Handoff: An environment contract, reset test, and observation-to-action traces. |
| VLA Foundations to Frontier 2026 — Professional Course | Follow history and architecture into action representation, data, training, evaluation, and deployment. Trace a complete action chunk through normalization, frame conversion, expiry, and controller consumption. Handoff: A workload dossier and closed-loop reference that constrain the future accelerator. |
| VLA Optimization and Action-Parity Harness — Special Course | Optimize one component at a time. Compare same-observation numerics and paired closed-loop outcomes separately; test stale observations and recovery as well as average success. Handoff: A passing candidate plus measured critical-path operator shapes for compiler and accelerator design. |
| Part 1 — Fundamentals of AI Inference / MLSys | Trace prefill and decode for explicit batch/context shapes. Derive weight and state traffic, then explain when compute, bandwidth, or scheduling dominates. Handoff: A baseline benchmark with numerical and resource assumptions for dense serving. |
| Part 2 — Dense Decoder-Only Inference at Hopper | Compare supported quantization and parallelism candidates on matched requests. Do not force AWQ to win; use data to choose. Without large GPUs, complete config/math work and mark throughput unmeasured. Handoff: A dense-model quality/capacity/latency comparison before studying sparse routing. |
| Part 3 — MoE Inference at Blackwell | Separate total parameter storage from active expert compute. Compare one supported placement and trace communication; reject unsupported model/format/kernel combinations before benchmarking. Handoff: Per-device capacity and traffic budgets that transfer to a named frontier checkpoint. |
| Part 4 — Optimizing a Real Engine | Treat published PR numbers as historical evidence. Reproduce one mechanism on a tractable subgraph, checking numerical tolerance and paired performance rather than inferring correctness from greedy decoding. Handoff: An optimization report with attribution, measured changes, and unresolved portability. |
| Part 5 — DeepSeek V4.1 Flash NVFP4: inference optimization engineering | Audit payload/scales, total versus active parameters, hybrid state, and parallel placement. A text-output multimodal model is an inference case study or optional planner; it is not a native robot action policy. Handoff: A checkpoint/memory/kernel dossier; transfer supported numerical and memory lessons to the selected VLA. |
| Inference lab tools | Run metadata, storage, placement, and benchmark-validation exercises on CPU. Use malformed scale or placement data as a negative control; distinguish fixture validation from loading the full checkpoint. Handoff: Machine-readable audit results and explicit hardware experiments still to run. |
| AI Inference Engineer 2026 — Special Course | Progress from fundamentals to dense/MoE cases and one real-engine optimization, then audit DeepSeek. Carry the same quality and timing definitions through all five parts. Handoff: An inference evidence bundle reusable by the physical-AI design studio. |
| GLM-5.3-Flash Architecture Mastery | Compare recurrent and chunked execution, including final state. Test speculative rejection/rollback for all state types; output agreement on one prefix is insufficient. Handoff: A hybrid-state ABI and reference vectors informing programmable memory/state support. |
| Phase 5 — Track G: ML Systems Engineering | Build a dependency graph and identify its critical path. Change a tile schedule to expose overlap; keep scaling efficiency, device utilization, and application throughput distinct. Handoff: A bottleneck explanation that selects the next kernel, compiler, or serving experiment. |
| Hardware-Aware LLM Quantization & Inference Optimization | Derive packed bytes including scales and unchanged tensors. Pair each precision change with quality and dispatch evidence; combine optimizations only after measuring them individually. Handoff: A precision/layout contract with error budgets for the compiler and accelerator. |
| Logprobs, Perplexity & KL Divergence — The Information Theory of LLM Inference | Compute masked-token NLL, perplexity, and KL with an explicit direction. Separate teacher-distribution KL from empirical data loss; add task metrics because low KL does not prove robot success. Handoff: A numerical quality grader reused across quantization candidates. |
| MLSys Deep Dives — The 2026 Machine-Learning-Systems Landscape | Choose one kernel, scheduling, architectural, or speculative change to test. Count cost over the same interval as accepted outputs; distinguish commercial prices from engineering cost. Handoff: A measured optimization ladder with combined results and costs. |
| TVM Deep Dives — Apache TVM for MLSys Engineers | Lower to a pinned compiler/runtime target, preserve shape/layout semantics, and test odd sizes. Inspect generated code and buffer lifetimes before claiming fusion or reuse. Handoff: A compiler/runtime ABI, generated kernel, and golden vectors for the chip-design capstone. |