Software & Programming of High Perfomance Systems

Course outline

1. Software development issues in HPC environments
   1. Cost of Software Development in HPC Environments
   2. Scalability and transferability of code
   3. Data complexity and parallel algorithms
2. Batch Systems
   1. Usage restrictions of resources in HPC environments
   2. SLURM
3. Nvidia GPU architecture
   1. Streaming Processor (SP)
   2. Streaming Multiprocessor (SM)
   3. SM features per GPU generation
   4. Other architectural features
   5. The concept of Compute Capability
4. The CUDA Programming Model
   1. What is CUDA?
   2. The concept of a Host and a Device
   3. Grid and blocks of threads
   4. Limitations on grid and block sizes
   5. Computational kernels
   6. Embedded variables of CUDA
   7. CUDA Runtime Flow
      8. Workload distribution
5. Memory access optimization
   1. Exploitation of the CUDA memory hierarchy
      2. CUDA shared memory programming strategy
         1. Splitting data into smaller tiles
         2. Data reuse
      3. Performance optimization issues
      4. Barriers
      5. DRAM bursting and exploitation in CUDA applications
6. Program flow control
   1. Warp divergence
      2. Warp divergence avoidance
7. Atomic instructions
   1. Atomic instructions in CUDA
      2. The Compare-And-Swap (CAS) atomic instruction
      3. Implementing other atomic instructions using CAS
8. CUDA Streams
   1. Synchronous & Asynchronous execution
      2. CUDA Streams
      3. Assigning calculations to a stream
      4. Scheduling within a stream
      5. Asynchronous data transfer from/to the GPU
      6. Events
9. Architecture of the Xeon Phi coprocessor
10. Native and offload programming modes

     11. Application of the OpenMP programming model on the Xeon Phi coprocessor