QNumerics 2026 3-3 — GPU Programming in Julia: Memory, Kernels & Performance

Описание: QNumerics Summer School 2026 — Session 3-3
GPU Programming in Julia with CUDA.jl: Memory, Kernels, and Performance
Instructor: Katharine Hyatt

This session introduces GPU programming through CUDA.jl using an NVIDIA GPU in Google Colab. It presents the GPU as an asynchronous coprocessor with separate device memory, explaining why highly parallel workloads such as matrix multiplication and independent simulations can achieve large speedups while small launches and frequent host–device transfers do not. At the array level, participants use CuArray objects, map and broadcast operations, reductions, and vendor-optimized linear algebra, while examining CUDA.jl’s memory pool, unified and pinned memory, batched work, and concurrent task submission. The lecture then moves to custom kernels: threads are organized into warps, blocks, and grids; block execution order is unspecified; and global indices identify each thread’s work. Kernel examples illustrate Julia dispatch, type stability, compile-time data-size requirements, output arrays, and restrictions on allocation and exceptions. The final performance section covers warp divergence, launch profiling, global, local, and shared memory, synchronization, coalesced access, block-size trade-offs, tensor cores, and the precision constraints of scientific GPU computing.

Learning objectives:
Identify workloads whose parallel structure and arithmetic intensity suit GPU execution.
Create and manipulate CUDA-backed arrays with Julia’s high-level array and linear-algebra interfaces.
Minimize launch and host–device transfer overhead through batching, asynchronous work, and suitable memory choices.
Map global work onto CUDA threads, warps, blocks, and grids in a custom Julia kernel.
Write type-stable, allocation-free kernels that communicate results through device arrays.
Diagnose divergence, poor utilization, and inefficient memory access with CUDA profiling tools.

Learn more about QNumerics: https://qnumerics.org/2026/
QuantumSavory, the umbrella organization: https://quantumsavory.org/