Skip to content

AMD GPU for HPC and AI

AGFH

Deep Dive · Specialization

Specialized course identity for AMD GPU for HPC and AI.

Artifact: specialization case study · Measure: performance, reliability, role fit

Timeline: 6–12 months.

Prerequisites: Phase 1 §4 (C++ and Parallel Computing — CUDA/OpenCL), Phase 4 Track C (ML compiler fundamentals), Nvidia GPU sub-track recommended for context.


Why AMD GPU

AMD Instinct is another major accelerator ecosystem for AI/HPC. Study one named device and a matching ROCm release; CPU/GPU memory organization, supported kernels and numerical formats differ between generations. Source review: 2026-10-18. No AMD hardware execution is claimed in this guide.


1. AMD GPU Architecture

  • CDNA vs RDNA:
    • CDNA (Compute DNA): data-center optimized — MI300X, MI250X. Matrix cores, large HBM, high bandwidth.
    • RDNA (Radeon DNA): consumer/gaming GPUs. Relevant for understanding the architecture family.
  • MI300X architecture:
    • Chiplet design: 8 XCDs (Accelerator Complex Dies) + 4 IODs.
    • 192 GB HBM3 with 5.3 TB/s bandwidth.
    • Matrix cores: FP16, BF16, FP8, INT8.
    • Infinity Fabric for inter-chiplet and inter-GPU communication.
  • Compute units:
    • CDNA wavefronts have 64 threads; RDNA targets can use 32-thread waves. Query the target and avoid a universal AMD=64 assumption.
    • SIMD units, LDS (Local Data Share) vs CUDA shared memory.
    • Occupancy model differences from Nvidia.

Compare generations without mixing memory organizations

MI300X has 192 GB HBM3; MI325X has 256 GB HBM3E; MI350X/MI355X use CDNA4 with 288 GB HBM3E and advertised 8 TB/s local-memory bandwidth. MI300A is an APU with CPU and GPU components sharing a different platform organization. These are named case studies, not a claim that they are the newest available products. Partition modes can change the logical devices visible to software, so record the actual mode. AMD architecture, MI350X specification.

Worked check: 8×288 GB is 2,304 GB nominal across GPUs. A 400 GB allocation does not fit on one 288 GB device merely because the node aggregate is larger. It needs a supported sharding/offload strategy, whose communication and workspace are additional.

2. ROCm Software Stack

  • ROCm (Radeon Open Compute):
    • Open-source GPU compute platform. Use the exact GPU/OS/framework compatibility matrix for a pinned release; an old “ROCm 6+” minimum does not establish current package compatibility.
    • Kernel driver (amdgpu), runtime (hip-runtime), compiler (amd-clang).
  • HIP (Heterogeneous-computing Interface for Portability):
    • CUDA-like API for AMD GPUs. hipMalloc, hipMemcpy, hipLaunchKernelGGL.
    • HIPIFY: Tool to convert CUDA source to HIP (hipify-perl, hipify-clang).
    • Source portability is a starting point; backend support and performance still require separate validation.
  • Libraries:
    • rocBLAS (GEMM), MIOpen (cuDNN equivalent), rocFFT, rocSPARSE.
    • RCCL (ROCm Communication Collectives Library) — AMD's NCCL equivalent for multi-GPU.
    • Composable Kernel (CK) — AMD's equivalent to CUTLASS for custom GEMM/attention kernels.
  • Profiling:
    • ROCprofiler-SDK / rocprofv3 — current tracing/profiling workflow; legacy rocprofiler/roctracer examples need a version boundary.
    • ROCm Compute Profiler (rocprofiler-compute, formerly Omniperf) — roofline, occupancy and memory analysis.
    • ROCm Systems Profiler — system/application timeline profiling; identify older Omnitrace recipes as historical.

3. Porting CUDA to AMD

  • HIPIFY workflow:
    • Automated conversion: hipify-clang for source-to-source translation.
    • Manual work: CUDA-specific intrinsics, inline PTX, warp-level primitives.
  • Key differences:
    • CUDA warps have 32 lanes; AMD wave size depends on the target. Use warpSize and a sufficiently wide lane mask for reduction, ballot, and shuffle code.
    • LDS bank count and access phases depend on architecture and instruction width. MI300/CDNA3 uses 32 banks, so “AMD has 64 banks” is not a portable rule. AMD LDS guidance.
    • Memory coalescing rules differ slightly.
    • AMD Matrix Cores serve a similar matrix-acceleration purpose; instruction shapes, formats and programming interfaces differ. Use supported rocWMMA, CK or compiler/library paths.
  • Framework support:
    • PyTorch: native ROCm support (torch.cuda works on AMD via HIP).
    • TensorFlow, JAX: ROCm backends available.
    • Compiler/runtime support is project- and release-specific. TVM targets and ONNX Runtime execution providers are different interfaces; verify maintained AMD support for the exact version instead of assuming an old ROCm provider remains available.
    • vLLM, TensorRT-LLM alternatives for AMD: vLLM + ROCm, AMD Inference Server.

4. Multi-GPU and Cluster Operations

  • Infinity Fabric:
    • Intra-node GPU-to-GPU interconnect. Bandwidth and topology compared to NVLink.
    • MI300X: 896 GB/s aggregate bidirectional fabric bandwidth per GPU; shared across links/peers, not a per-peer guarantee.
  • RCCL:
    • All-reduce, all-gather, reduce-scatter on AMD GPUs.
    • Tuning for MI300X topology and Infinity Fabric.
  • Multi-node:
    • InfiniBand and RoCE (RDMA over Converged Ethernet) support.
    • GPUDirect RDMA equivalent on AMD.
    • Slurm and Kubernetes with ROCm container support.

5. Kernel Development on AMD

  • HIP kernel writing:
    • Thread hierarchy: grid → block → thread (same as CUDA).
    • Shared memory (__shared__), synchronization (__syncthreads()).
    • Wavefront-aware programming: __ballot, __shfl, warp intrinsics via wavefront equivalents.
  • Composable Kernel (CK):
    • Templated library for high-performance GEMM, attention, and custom ops.
    • Tile-based programming model (similar concept to CuTe/CUTLASS).
    • Writing custom kernels with CK building blocks.
  • Triton on AMD:
    • Triton supports AMD GPUs via ROCm backend.
    • Same Python kernel code, different backend code generation.
    • Performance tuning differences (tile sizes, occupancy targets).

Resources


Projects

  1. HIPIFY a CUDA kernel — Take your CUDA matmul or vector-add kernel from Phase 1 §4. Convert to HIP using hipify-clang. Run on a supported AMD GPU; a ROCm container packages userspace and does not emulate an AMD GPU. Compare output and performance.
  2. Profile with ROCm Compute Profiler — Profile a PyTorch model on ROCm. Generate a roofline plot. Identify compute-bound vs memory-bound layers.
  3. RCCL benchmark — Run RCCL all-reduce across multiple AMD GPUs. Compare bandwidth and latency with NCCL on equivalent Nvidia hardware.
  4. Triton on AMD — Write a Triton fused kernel (e.g., layer norm + residual). Run on both Nvidia (CUDA) and AMD (ROCm). Compare generated code and performance.
  5. CK custom GEMM — Use Composable Kernel to implement a custom GEMM with epilogue fusion (bias + activation). Benchmark against rocBLAS.

Port one reduction before a full model

Bring a CUDA reduction from Phase 1 and test input lengths 31, 32, 33, 63, 64, and 65 plus a large nonmultiple. These cases expose hard-coded warp masks, lane counts, and incomplete tail handling. Verify the same numeric contract on CPU and the selected AMD target before profiling.

Separate “HIPIFY produced source,” “the kernel passed,” and “the port is efficient.” Record architecture, wave size, workgroup size, registers/LDS, achieved bandwidth, and backend/library revisions. Pass that result to platform evaluation; a container image or successful compilation alone does not establish portability.

Practice and assessment

Entry diagnostic (prerequisite check): Draw GPU/CPU/NIC topology and explain bandwidth versus message latency. If this is unfamiliar, use the prerequisite material above or the roadmap diagnostics before starting the lab.

  1. Baseline lab: Establish compute, memory-transfer, and collective baselines on available hardware.
  2. Guided variation: Change placement or message size and test a recoverable worker failure.
  3. Independent transfer: repeat the method on a new input or configuration and explain where your original assumptions stop applying.

Evidence to submit: Topology, achieved bandwidth/latency, scaling efficiency, error logs, and configuration. Keep raw results and reproduction steps; distinguish measured results from estimates or work not run.

Self-check: Which observation would disprove your explanation of the result? Identify a competing explanation and the smallest experiment that separates them. A passing answer names a concrete predicted difference, not only another tool to try.

Exit criteria: reproduce the baseline, demonstrate the negative/variation check, and meet the shared completion rubric. Use the lesson cycle for retrieval practice and spaced review.