GPU Programming Fundamentals Training in Astana, Kazakhstan

This course introduces the core principles of GPU programming, including GPU architecture, parallel programming concepts, and hands-on development with CUDA C/C++. Participants will gain foundational knowledge in writing parallel algorithms, managing GPU memory, and optimizing code performance for GPU execution.

INTRODUCTION TO GPU COMPUTING AND CUDA

Introduction to GPU Architecture

  Why GPU computing? CPU vs GPU comparison

  CUDA architecture: cores, warps, blocks, and grids

  Execution model and SIMT (Single Instruction Multiple Thread)

Getting Started with CUDA Programming

  CUDA development tools and environment setup

  Writing and compiling a basic CUDA program

  Anatomy of a CUDA kernel and launch configuration

GPU MEMORY MODEL AND THREADING

Understanding GPU Memory Hierarchy

  Global, shared, constant, and local memory

  Memory coalescing and access patterns

  Performance impact of memory usage

Thread Organization and Synchronization

  Thread indexing in CUDA

  Synchronizing threads with __syncthreads()

  Avoiding race conditions and deadlocks

PERFORMANCE OPTIMIZATION AND APPLICATIONS

Debugging and Profiling CUDA Applications

  Using tools like nvprof, Nsight Compute, and cuda-memcheck

  Identifying and fixing performance bottlenecks

  Measuring speed-up and efficiency

Implementing Parallel Algorithms

  Parallel reduction techniques

  Matrix multiplication and stencil operations

  Real-world use cases: image processing, simulations

Final Hands-On and Wrap-Up

  Review, Q&A, and recommended learning paths