Parallel Processing

Course Outline

1. Introduction to the concepts of Parallel Processing
2. Classification of Parallel Computing Systems
   1. Flynn’s taxonomy
   2. Classification based on the architecture of the memory
   3. Vector architectures
   4. Systolic arrays
3. Performance metrics
   1. Amdahl’s Law
   2. Gustafson’s Law
   3. Speedup
   4. Karp-Flatt metric
4. Mutual exclusion - Semaphores - Atomic instructions
5. Programming models for parallel architectures
   1. Threads - The POSIX Threads programming model
   2. The OpenMP programming model
      1. Execution model
      2. Directives
      3. Functions
      4. Environment variables
      5. Tasks - User-level scheduling
   3. The MPI programming model
      1. Execution model
      2. Point-to-point communication
      3. Collective communication
      4. Topologies
6. Data dependencies - Vectorization
7. Interconnection networks
   1. General concepts
   2. Static interconnection networks
      1. One-dimensional, two-dimensional, Hypercube
   3. Dynamic interconnection networks
      1. Bus
      2. Single-stage
      3. Multi-stage – Shuffles
      4. Crossbar