Skip to content

Foundation sources and evidence boundaries

This dossier records primary references used for Lectures 01–08. It supports the course's October 13, 2026 evidence cutoff. A source's first publication date, a repository's current contents, and an independently reproduced result are different records.

The historical sequence intentionally ends at the foundations needed for later frontier lectures. It does not imply that FAST, OpenVLA, or any other historical example is the newest or best available model in October 2026. Model rankings require a specified task, evaluation protocol, and compared implementation revisions.

Reading this record

  • Paper evidence: establishes what authors proposed and reported under their experimental setup. It is not a local measurement.
  • Official implementation evidence: establishes what maintainers document or expose at the inspected URL. A moving branch is not an immutable reproduction pin.
  • Original teaching material: the numerical examples, synthetic fixtures, evaluation proposals, and CPU exercises in these lectures were constructed for this course. They are not paper results.
  • Local verification: arithmetic and Markdown structure were checked during authoring. Upstream VLA training, checkpoint inference, physical robots, and Isaac execution were not run for these eight lectures.

When starting an upstream lab, record the actual paper version, code commit, checkpoint revision, dataset revision, processor/tokenizer revision, normalization statistics, environment lock, and robot/controller configuration. Do not substitute the date of this course for those pins.

Historical and decision-making foundations

Primary source Date and role Facts used Scope and limitations
Brooks: A Robust Layered Control System for a Mobile Robot MIT technical report, September 1985; associated journal publication 1986 Lecture 02: reactive layered control as a historical design point A control architecture, not a language-conditioned model; cited for the historical distinction
ALVINN proceedings NeurIPS 1988 conference volume Lecture 02: early neural sensory-input-to-steering mapping Conference-volume date differs from the publication convention below; no claim of modern semantic or cross-embodiment generality
CMU publication catalog Catalog lists ALVINN proceedings in 1989 Lecture 02: explicit date distinction Institutional bibliographic record, not a separate experiment
Kaelbling, Littman, Cassandra: Planning and Acting in Partially Observable Stochastic Domains 1998 journal article; earlier technical report exists Lecture 01: partial observability, belief-state formulation Formal model assumes specified transition/observation structure; a learned VLA need not compute an exact belief
Ross, Gordon, Bagnell: A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning AISTATS 2011 Lecture 02: DAgger and learner-induced state distributions Guarantees depend on assumptions; the scalar lab is inspired by the idea and does not reproduce the paper
Levine et al.: End-to-End Training of Deep Visuomotor Policies JMLR 2016; earlier preprint exists Lecture 02: guided policy search and learned image-to-torque policies Historical manipulation setup; no broad-language or generalist-VLA claim

Language grounding and early large-model robotics

Primary source First arXiv submission Facts used Scope and limitations
CLIP February 26, 2021 Lectures 03 and 07: image-language supervision and representations Representation learning; image-text similarity alone does not establish physical feasibility
CLIPort September 24, 2021 Lecture 03: complementary semantic and spatial pathways Reported tabletop manipulation settings; not an unrestricted continuous-control interface
SayCan April 4, 2022; revised August 16, 2022 Lectures 01 and 03: language-informed selection grounded by available skills and feasibility Planner/skill boundary; the course's scoring table is synthetic and its scores are not a paper benchmark
RT-1 paper and official project December 13, 2022 Lecture 04: visual/language conditioning, compressed visual tokens, discretized actions, scale/diversity study Paper architecture and project description; no local execution or assumed complete reproduction bundle
PaLM-E March 6, 2023 Lecture 04: sensory embeddings in an embodied multimodal language model and manipulation-planning use Grounded reasoning output must be distinguished from direct motor commands
RT-2 paper and official project July 28, 2023 Lectures 01 and 04: action tokens and vision-language transfer to robot control Semantic transfer in evaluated settings; no claim that web knowledge alone supplies arbitrary motor skills

Open policies and data

Primary source Date and role Facts used Scope and limitations
Open X-Embodiment / RT-X paper First submission October 13, 2023; revision history extends into 2025 Lecture 05: multi-embodiment data and transfer question Dataset counts/results can differ by version; the lecture avoids treating a moving paper as a fixed count ledger
Official OXE repository Official dataset/project entry point Lecture 05: dataset-access and implementation provenance Constituent datasets retain their own access, schema, and licensing details
Octo paper First submission May 20, 2024 Lecture 05: 800,000-trajectory pretraining and adaptable generalist policy A reported research configuration; does not prove zero-shot operation on arbitrary robots
Official Octo implementation Moving official repository; select a commit for execution Lecture 05: diffusion policy and observation/history masks Versioned APIs and dependencies require inspection; repository examples are not locally validated here
OpenVLA paper First submission June 13, 2024 Lecture 05: seven-billion-parameter model, 970,000 demonstrations, fused visual features and Llama 2 backbone Original reported design; the lecture does not generalize its benchmark comparisons beyond the evaluated tasks
Official OpenVLA implementation Moving official repository; select a commit and checkpoint revision for execution Lecture 05: dataset-specific action unnormalization and artifact dependencies Code, base-model terms, checkpoints, processor and controller are separate artifacts; no universal plug-and-play guarantee

Geometry, encoders, and action codecs

Primary source Date and role Facts used Scope and limitations
ROS REP-103 official source Created October 7, 2010; source contains later history Lecture 06: units and coordinate conventions, including optical frames Reference convention; actual devices and imported datasets may differ and must be adapted
ROS REP-105 official source Created October 27, 2010 Lecture 06: mobile platform frame semantics Does not provide a camera calibration or guarantee localization accuracy
Attention Is All You Need First submission June 12, 2017 Lecture 07: scaled dot-product attention VLA masks, shape calculations, and cache examples are course constructions
SigLIP First submission March 27, 2023 Lecture 07: pairwise sigmoid image-language objective Representation objective, not a robot-success guarantee
DINOv2 First submission April 14, 2023 Lecture 07: visual pretraining without language supervision Generic feature results do not establish precise control performance
Prismatic VLMs First submission February 12, 2024 Lecture 07: systematic VLM design-space investigation Robotics consequences require separate evaluation
FAST paper, v1 January 16, 2025 Lecture 08: frequency-space action encoding, coefficient quantization, BPE and FAST+ distinction No local training-speed, compression-ratio, or robot-success reproduction; the CPU exercise omits BPE
Official FAST tokenizer Official artifact entry point Lecture 08: tokenizer implementation/artifact provenance Tokenizer availability is not availability of an entire policy/controller system
SciPy DCT documentation Versioned library reference; record the actual installed version if used Lecture 08: explicit transform-normalization conventions The required exercise can implement the formula directly; no SciPy installation is claimed

The ROS rendered documentation endpoints returned a bot-challenge page during this research. The official source repository and raw source were inspected instead; no content was inferred from the challenge response.

Checks performed during authoring

An independent Python 3 arithmetic check reproduced the examples for observation-age displacement, independent-step survival probability, pixel-to-millimeter scale, attention matrix element ratio, hidden-state storage, K/V-cache storage, scalar bin width, and the constant-signal orthonormal DCT coefficients. These are analytical/synthetic checks, not runtime benchmarks.

The eight pages were also checked for balanced inline/display math delimiters and code fences, absence of stray control characters, and the presence of prerequisite, hardware, effort, lab, and answer-guidance sections. The full course build and navigation are checked at the course integration level.

Course · Start Lecture 01 · Continue after the foundations