Lecture 10 — Capstone: defend a programmable VLA accelerator¶
Previous: Prototype and feasibility · Course: Overview · Execution: Lab guide
The capstone turns the original proposal into a falsifiable design study. The product hypothesis is: a robot completes the required tasks, within its response-time constraints, using less compute-system energy. Select an evidence level you can complete; a sound analysis is useful, but cannot be relabeled measured silicon performance.
1. Freeze a scope that can be reviewed¶
Select a task family, an embodiment or simulator, and a small diverse policy set. Begin with one usable checkpoint; add a second architecture only after the first reference works. Flow-based, autoregressive-action, and cascade-style policies test different assumptions. Use the model source ledger to check accessible weights, dependencies, and licenses.
Write a one-page requirements sheet:
| Requirement | Example form, to replace with your task |
|---|---|
| Behavior | Named tasks/scenes, success predicate, timeouts, recovery and intervention rules |
| Freshness | Capture-to-usable deadline and maximum source-observation age at consumption |
| Power | Measurement boundary and sustained budget, including host/DRAM |
| Capacity | Peak live system memory and supported concurrent workloads |
| Compatibility | Required model/operator/shape/precision set; documented fallback |
| Programming | Import path, compiler IR, generated commands, versioned runtime/driver ABI |
| Connectivity | Telemetry/task-request role, expiry, authentication, link-loss behavior |
| Quality | Allowed loss, paired evaluation, confidence procedure, held-out conditions |
The teaching design envelope of 0.5–4B parameters, 20–40 W, 16–32 GB external memory, 16–64 MiB SRAM, and 10–30 Hz are a search region. Replace each with evidence. Do not combine all upper bounds into a claimed feasible product.
2. Build an evidence chain¶
| Gate | Artifact | Reason to return upstream |
|---|---|---|
| A — Reference | Pinned model/data/processor and valid action loop | Unknown action semantics or unavailable dependency |
| B — Workload | Trace, shapes, lifetimes, repeated work, cold/warm traffic | Analytical counts disagree with trace |
| C — Architecture | Tile/vector/memory candidates and contention model | Bandwidth, SRAM, or timing budget fails |
| D — Software | Numerical reference, lowering, command ABI, fallback | Unsupported operator or incorrect layout/state |
| E — Prototype | Simulator/RTL/FPGA evidence at a declared level | Tail shapes, ownership, or recovery fail |
| F — Evaluation | Quality, energy, freshness, sustained behavior | Advantage vanishes at matched task quality |
| G — Integration decision | Linux/wireless plan and physical feasibility memo | Missing IP, unrealistic power/area, or unbounded shared-resource interference |
Keep rejected candidates. They explain why the final architecture needs each feature. If a small programmable baseline meets the requirements, justify additional area by the measured limitation it removes.
3. A concrete design-space experiment¶
Run the CPU memory sweep before collecting hardware data. It uses a synthetic 3B packed-4-bit backbone once and a 300M 8-bit expert ten times. The zero-residency payload-only traffic is 4.5 GB; at 120 GB/s the weight-transfer floor is 37.5 ms.
Then compare:
- SRAM capacities 16, 32, and 64 MiB, reserving 8 MiB for other live data in this toy model.
- Cold versus already-valid warm weight residency.
- Sustained bandwidth 60, 120, and 180 GB/s.
- Optional 1D NVFP4 metadata for one synthetic backbone tensor.
- Small versus larger padded matrix tiles for the same operator shapes.
The curve omits compute, activation traffic, banks, power, and area. It is not a measured speedup. PNG. Explain why 64 MiB does not hold the 300 MB expert and why its 8 MiB reserve reduces useful residency.
Use these results to prioritize a real trace experiment. Iteration reduction, model compression, caching, and bandwidth changes require separate correctness and feasibility evidence; the sweep does not decide them automatically.
4. Proposal audit¶
For each claim in your design proposal, record one of four outcomes:
- Verified context: supported by a dated primary specification, paper, or official repository; preserve its conditions.
- Engineering hypothesis: plausible and now paired with a falsifying experiment.
- Analytical example: reproducible arithmetic with units and omitted costs.
- Unresolved: inaccessible artifact, inconsistent source, missing metadata, or experiment not run.
Examples: Thor capacity is a specification; GR00T replanning rate is a vendor-reported result; a 2× energy target is a hypothesis; the 37.5 ms calculation is a conditional bandwidth floor. A public paper does not establish that its checkpoint, kernel implementation, or chip is available.
DeepSeek connection: use DeepSeek V4.1 Flash NVFP4 to practice exact format, state, and placement auditing. Its text-output workload is not automatically an on-robot VLA action policy. Transfer the relevant mechanisms and retain the different capacity, latency, and action contracts.
5. Review packet¶
Submit a single index linking:
- Requirements and architecture diagram, including host, memory, controller, and wireless boundaries.
- Immutable artifact manifest and reproduction commands.
- Model/operator inventory, numerical reference, and invalid-input tests.
- Capacity and traffic worksheets, candidate sweeps, and model-to-measurement discrepancies.
- Compiler/ABI specification and one executable lowered subgraph.
- Scheduling/freshness analysis and overload/recovery evidence.
- Paired task outcomes, power-window definition, raw energy/timing data, and uncertainty.
- Implementation maturity, missing experiments, PPA assumptions, and product decision.
Keep large weights and recordings outside the source repository unless redistribution is permitted. Provide retrieval instructions, checksums, and small shareable fixtures. The current course labs are deliberately small and do not download model weights.
6. Assessment¶
| Dimension | Weight | Strong evidence |
|---|---|---|
| Workload and numerical correctness | 25 | Real graph semantics, complete formats, valid actions, negative cases |
| Memory and architectural reasoning | 20 | Distinct capacity/traffic; explicit reuse and bottleneck assumptions |
| Compiler/runtime integration | 20 | Executable mapping, ABI, ownership, fallback and recovery |
| Experimental quality | 25 | Controlled comparisons, failed attempts retained, uncertainty and energy boundary |
| Reproduction and critique | 10 | Another learner reruns a bounded result; limitations identify next experiments |
A teaching pass is 75/100 with no missing correctness or reproduction evidence. This is a course rubric, not tapeout approval or product certification. An architecture hypothesis that is convincingly falsified can pass; an unsupported efficiency claim cannot.
Oral defense with answer guidance¶
- What survives a new action-head architecture? Identify programmable vector/tensor operations, supported state/layouts, and explicit fallback; do not answer only “the compiler.”
- Where did each saved byte go? Point to actual removed loads or retained allocations, including fills, evictions, scale data, and buffer reserves.
- What happens during network loss? Trace the local controller and expiry behavior independently of retries or planner availability.
- What would make you abandon the chip proposal? Examples include no advantage against a tuned baseline, insufficient model coverage, quality regression, or infeasible memory/power implementation.
Continue: pass the reviewed evidence into the full SoC capstone and Physical AI Design Studio. The next design decision should answer the most consequential unresolved question in your report.