GPGPU Computing in C++ with CUDA

Some algorithms are massively parallel, for instance applying filters to images, convolutions, matrix operations, particle based physics simulations, evaluating neural networks. These algorithms can be massively accelerated by executing them on the GPU instead of the CPU. In this course we use NVIDIA CUDA for writing programs that perform their computations on the GPU.

This course goes close to the metal and as such we focus a fair bit on GPU hardware architecture, since the main goal of GPU programming is to get better performance we need to know which patterns that work best and how to debug and optimize for GPU computing.

NVIDIA CUDA is a library and extension to C++ that allows you to mix code that runs on the CPU with code for the GPU in same source file.

Target audience and prerequisites

Experienced C++ Developers who have developed with multi-threaded programming using threads on CPU. 

What you will learn

• Introduction
• GPGPU History
• Hardware
  • CPU Hardware
  • GPU Hardware
• CUDA Basics
  • Writing a CUDA kernel
  • Grid-stride loops
  • Error Handling
  • Inside a kernel
  • Global Memory
  • CUDA Toolchain
• Debugging
  • CUDA GDB
  • MemCheck
  • Nsight
• Tiling
  • Basic Tiling
  • Coalescing
  • Shared Memory
  • Bank Conflicts
  • GPU Stalls
  • Occupancy

• Algorithms and Datastructures on a GPU
  • Vectors
  • Lists
  • Trees
  • Map
  • Transpose
  • Reduce
  • Parallel Prefix Sum
  • Stream Compaction
  • NVIDIA Thrust
• Synchronization
  • Fences
  • Atomics
  • Volatile
  • Invariants
  • Warp Functions
  • Cooperative Groups
• Profiling
  • Measuring Performance
  • Bottlenecks
  • Events
  • Visual Profiler
  • Texture Memory
  • Constant Memory
• Streams
  • Command Queues
  • Memory Busses
  • Overlapping Transfeers
  • Dynamic Parallelism
  • TDR, How to avoid