Part 5 — DeepSeek V4.1 Flash NVFP4: inference optimization engineering¶
2026-10-18 update: read the 1017 release evidence and CPU exercise. The ten lectures retain the earlier pinned conversion case; 1017 has new weight hashes and requires its own conversion and runtime validation.
Outcome: build an auditable deployment decision for a named checkpoint, then optimize a supported execution path while preserving model quality.
Prerequisites: Part 1 (execution, rooflines, precision), Part 3 (MoE and parallelism), and Part 4 Lecture 02 (measurement). You should be able to distinguish checkpoint size from peak GPU memory and TTFT from decode throughput.
Effort: ten lectures; approximately 25–40 hours for the analysis route, plus installation and hardware time for full serving experiments. Stop a deployment experiment at its compatibility gate if the exact format is unsupported.
Access: Lectures 01–03 and the supplied fixtures run on a CPU with Python 3.10+. Full-checkpoint work requires a capacity plan for hundreds of GiB of weights, runtime memory, and the chosen GPU topology. An FP4-capable GPU alone does not establish model support.
Checkpoint identity and evidence¶
Checked 2026-10-13. These are distinct artifacts:
| Artifact | Role in this course | Evidence boundary |
|---|---|---|
| deepseek-ai/DeepSeek-V4.1-Flash | Official architecture and source-format reference | Its evaluations apply to the upstream model |
| s-zaizen/DeepSeek-V4.1-Flash-NVFP4 | Default community conversion for the checkpoint audit | Backbone routed experts converted; other components retain source formats. Publisher reports no quality/throughput benchmark or validated launch recipe |
| LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4 | Optional comparison of format and quality tradeoffs | Also reduces Engram precision; publisher explicitly leaves end-to-end quality unestablished |
Neither community conversion is an official NVIDIA checkpoint. Upstream experts already use FP4; “convert to NVFP4” does not imply halving the model. Read each actual config, index, and loader. The s-zaizen card identifies its upstream source revision; resolve and record a separate immutable revision for the conversion you inspect.
The current vLLM architecture documentation contains a DeepSeek V4.1 implementation. That is evidence of architecture work, not proof that either community serialization works on a particular released NVIDIA backend. Do not repeat the older claim that no engine supports the architecture; verify the exact runtime commit and checkpoint together.
Lecture map¶
| Lecture | Engineering question | Artifact |
|---|---|---|
| 01 — Checkpoint identity and compatibility | What did you actually download, and can a loader read it? | Metadata manifest and support matrix |
| 02 — Execution and memory budgets | What is resident, active, cached, or gathered? | Per-rank memory and traffic budget |
| 03 — NVFP4 representations and numerical checks | Which values and scales did the conversion change? | Reconstruction checks and error cases |
| 04 — Establishing a serving baseline | Is a successful launch also correct inference? | Pinned smoke-test report |
| 05 — Prefill, decode, and kernel profiling | Which stage limits this workload? | Trace-backed bottleneck hypothesis |
| 06 — MoE placement and communication | When does sharding save memory but lose time? | Topology and EP/TP sweep |
| 07 — Long context and cache behavior | What does cache reuse save in your runtime? | Context, cache, and memory curve |
| 08 — Scheduling, load, and goodput | How much useful work meets the latency target? | Reproducible load experiment |
| 09 — Quality and speculative decoding | Does the optimized system still perform the task? | Quality gate and speculation ablation |
| 10 — Capstone deployment decision | Should this exact deployment ship? | Reproduction package and decision |
Start on a laptop¶
From this directory:
python3 labs/checkpoint_audit.py --demo
python3 labs/inference_metrics.py --demo
python3 labs/fp4_layout.py
python3 -m unittest discover -s labs -p 'test_*.py'
These standard-library exercises use synthetic fixtures, not DeepSeek performance measurements. See the lab guide for real inputs and expected outputs.
Completion levels¶
Analysis route: submit inspected metadata, the memory derivation, format checks, a compatibility decision, and a benchmark design. Mark full-model execution “not run.” Unsupported deployment is an acceptable evidence-based decision.
Deployment route: additionally submit a working pinned launch, reference-quality comparisons, at least three repeated load runs, per-rank memory, and one successful or rejected optimization backed by traces. Apply the shared rubric; an integrity check alone cannot satisfy model-quality assessment.
Back to the inference course · Start Lecture 01
Carry this course into the physical-AI system¶
Start from: Exact DeepSeek V4.1 Flash checkpoint identity and loader compatibility.
Build and challenge: Audit payload/scales, total versus active parameters, hybrid state, and parallel placement. A text-output multimodal model is an inference case study or optional planner; it is not a native robot action policy.
Pass forward: A checkpoint/memory/kernel dossier; transfer supported numerical and memory lessons to the selected VLA. Continue with the connected course and keep the artifact in the Physical AI Design Studio.