Lecture 06 — Compiler, Runtime ABI, and SoC Integration¶
Question: how does a policy graph become executable accelerator commands inside a Linux system without losing its tensor, timing, or action semantics?
Prerequisites: Lecture 05, compiler intermediate representations, virtual memory, DMA, and producer/consumer queues. Use the LLVM/MLIR lectures and embedded Linux course for the underlying concepts.
By the end: specify a lowering path and command ABI, justify one legal fusion, trace buffer ownership, and reject invalid commands and stale action chunks in a simulator.
Deliverable: a compiler/runtime interface document, a small executable command-stream interpreter, and a negative-test report. This is a prototype interface, not a production Linux driver.
1. The compiler needs three kinds of contract¶
| Contract | Examples |
|---|---|
| Mathematical | Operator definition, axes, masking, epsilon, quantization, rounding, and tolerated error |
| Physical | Shape, stride, layout, alignment, packing, banks, buffer lifetime, and DMA range |
| Robotic | Observation identity, clock domain, units, coordinate frame, action horizon, and validity window |
The tensor compiler primarily transforms the first two. The system runtime must preserve the third across submission and completion. A correctly computed action tensor associated with the wrong observation is still an incorrect system result.
Keep a manifest linking model and preprocessing revisions to compiled artifacts, quantization metadata, supported shapes, ABI version, and action adapter. A checksum can detect accidental mismatch; it is not by itself an authorization or authenticity mechanism.
2. Lower progressively and keep the evidence¶
reference policy + frozen observation/action contract
-> captured graph with explicit shapes and numerical semantics
-> supported accelerator partitions + visible fallback partitions
-> tiled tensor/vector operations with selected layouts
-> allocated buffers, dependency events, and memory transfers
-> versioned command stream + constants + runtime manifest
-> functional interpreter, then RTL/FPGA execution
For each transformation, retain a small example and compare against the previous representation. Do not wait until an entire policy has been lowered before checking whether transpose, masking, or scaling semantics survived.
MLIR's Linalg representation supports transformations such as tiling, fusion, and lowering toward loops or target operations. Bufferization addresses the transition from tensor values to mutable memory while accounting for conflicts. Neither framework automatically supplies your chip's ABI or proves the robot's timing contract. Linalg, Bufferization
The report must list every fallback with transferred bytes and synchronization points. Supporting 95% of operators by count can be unhelpful if the remaining 5% contains the slowest repeated block.
3. Worked fusion: saved traffic versus changed semantics¶
Consider a reference path:
where \(XW\) uses a declared wider accumulator and \(Z\) has shape \(16\times256\). Assume the unfused path writes \(Z\) to external memory and reads it back for the vector stage.
The intermediate occupies \(16\times256\times2=8{,}192\) bytes. Keeping it on-chip can avoid one write and one read:
This is the maximum avoided traffic for that intermediate under the stated baseline. If the baseline already retains it in cache or SRAM, measured DRAM savings can be smaller.
A legal implementation must preserve broadcasting, layout, aliasing, and the chosen numerical contract. Keeping the wider accumulator directly through GELU removes the intermediate cast and changes rounding. That may be an acceptable separately validated numerical transformation, but it is not automatically bitwise equivalent fusion.
A reduction adds another constraint. A normalization requiring the whole feature axis cannot generally be evaluated independently on partial feature tiles without combining the required statistics. Fusion is justified by data dependencies and semantics, not by adjacency in a graph drawing.
4. Define the command ABI before the driver¶
The application binary interface (ABI) specifies how independently built compiler output, runtime software, firmware, and hardware agree. Begin with a small versioned command set such as COPY, MATMUL, VECTOR, WAIT, and SIGNAL.
These are teaching opcodes, not an existing standard. Their specification needs enough detail to make a command interpreter reject malformed input deterministically.
| Field or rule | Required meaning |
|---|---|
| ABI version and command length | Reject unknown versions and truncated records |
| Opcode and flags | Define permitted combinations and reserved values |
| Buffer handle, offset, and length | Resolve through owned, validated allocations |
| Shape, stride, and layout ID | Bound every accessed element; state supported dynamic ranges |
| Quantization descriptor | Match scales, grouping, rounding, and payload encoding |
| Accumulator and output mode | Avoid an implicit precision change |
| Dependency and completion IDs | Prevent consumption before production; distinguish completion from acceptance |
| Request and observation IDs | Associate outputs with their source inputs |
| Deadline and clock domain | Define exactly how expiration is interpreted |
| Error and reset generation | Prevent old completions from being mistaken for new work |
Specify byte order, fixed field widths, alignment, reserved fields, extension rules, and overflow-safe range validation. For example, validate offset <= allocation_size and length <= allocation_size - offset using a suitable unsigned type, instead of trusting an overflowing offset + length expression.
Linux's ioctl guidance is useful when designing a stable userspace interface. A prototype should use explicit-width fields and compatible layouts rather than persisting a compiler-dependent C structure as an undocumented binary format. Linux ioctl interfaces
5. Track DMA ownership through completion¶
For an input buffer, the conceptual lifecycle is:
CPU writes -> map/synchronize for device -> publish descriptor
-> device reads -> completion -> synchronize/unmap as required
-> CPU may reuse or release storage
For an output buffer, reverse the producer and consumer roles. Multiple devices may add additional ownership transitions.
A device DMA address is not necessarily a CPU virtual address or a raw physical address. Use the platform's DMA mapping API and account for mapping failure and addressability. Coherent allocation addresses visibility requirements but does not eliminate ordering requirements around publishing descriptors and observing completion. Linux DMA mapping guide
Define which agent owns each buffer at every transition. A cancellation request does not prove that DMA has stopped. The implementation must quiesce, complete, or reset relevant work before recycling storage that the device may still access.
The functional interpreter should model ownership even though it does not perform real DMA. That allows a test to reject a read of an unpublished input or reuse of an in-flight output before hardware exists.
6. Bounded queues and observable errors¶
Use a bounded submission queue and a separate completion record. Define full-queue behavior: block, reject, or replace only work that can be canceled according to the dependency and ownership rules.
Sequence numbers distinguish a wrapped queue slot from its earlier occupant. Publication and consumption need synchronization appropriate to the CPU/device interface. A single-producer/single-consumer CPU ring is not automatically a correct multi-producer or DMA queue. Linux circular-buffer documentation
Require the following negative outcomes to be distinguishable:
- Unsupported shape or numeric format: reject before launching dependent work.
- Invalid range or missing dependency: reject the command stream.
- Device fault or timeout: report an unsuccessful completion and follow the defined recovery path.
- Expired policy output: discard it at the action boundary even if arithmetic completed correctly.
Do not fabricate success by substituting an old output when a command fails. If the robot's supervisor uses a fallback, it must be a separately defined behavior with an observable reason.
7. Connect accelerator results to the SoC¶
The Linux CPU handles model loading, memory management, sensor services, runtime submission, and application integration. It can also run fallback operators. A local controller or separately scheduled control subsystem consumes validated action proposals at its own rate.
Each action chunk should identify the source observation and policy revision, carry the coordinate frame and units, and state the intended action times or horizon. Check freshness when publishing the chunk and again when consuming actions from it. A chunk can expire after admission.
The course's timestamp and queue mechanisms help build a testable interface. They do not establish a functional-safety certification or a hard real-time bound. Those require the corresponding system analysis and validation.
Wireless carries telemetry, updates, or appropriately bounded higher-level requests in the initial design. Put it outside the time-critical motor/flight-control dependency chain. Use the ESP-Hosted integration course to study host/companion integration, and specify behavior under disconnect or delayed messages.
State whether the prototype uses a module, companion radio, or integrated RF design. A Linux-capable CPU plus an accelerator block does not automatically provide boot firmware, memory-controller IP, sensor interfaces, or wireless capability.
8. Exercise: run one block through the entire contract¶
Lower the worked matrix/vector block into your teaching command stream. Use host memory and a functional interpreter first; hardware access is optional for this stage.
- Emit the reference tensor graph, selected layouts, buffer plan, and commands.
- Check unfused and fused outputs under the same rounding contract.
- Track buffer ownership and dependency completion in an execution log.
- Attach observation and request IDs to the resulting action-like output.
- Implement a bounded consumer that rejects an expired output.
Negative cases: truncate a command; use an unsupported shape; overflow a proposed range; omit a dependency; reuse a buffer before completion; and deliver a completion from an old reset generation. Each must produce an explicit rejected or failed result.
Acceptance: submit the ABI document, interpreter, one reference comparison, and a table of rejected cases. Report what remains untested on real hardware: DMA visibility, interrupts, reset behavior, clock synchronization, and sustained contention.
9. Answered self-checks¶
1. Is eliminating an intermediate cast always legal fusion? No. It changes rounding and can change the result. Preserve the cast for equivalence or evaluate the change as a separate numerical optimization.
2. Does coherent memory make queue ordering unnecessary? No. Visibility, ownership, and publication order are distinct requirements. Follow the platform's synchronization and DMA rules.
3. Does a completed accelerator command imply that its action is usable? No. The output can be stale, belong to an old request or reset generation, use the wrong frame, or violate the action contract.
Sources and next step¶
- MLIR Linalg and Bufferization: inspect transformation and memory semantics.
- Linux DMA API: establish address mapping, visibility, and ownership transitions.
- Linux ioctl interfaces and circular buffers: design explicit, versioned interfaces and correct queue synchronization.
Continue to Lecture 07 — Scheduling and Freshness. The runtime interface now carries the information needed to analyze interference, observation age, and deadline misses.
Inspect an actual lowering boundary¶
Use the real MLIR CPU fixture to observe Linalg become loops, LLVM dialect, and native CPU execution. Its zero initialization, full reduction, output checks, and deallocation are visible. This establishes a working CPU path only. Your accelerator extension must separately implement command encoding, buffer ownership, completion, and the target arithmetic contract; naming a custom dialect operation does not supply those behaviors.