HIGH PERFORMANCE PARALLEL PROGRAMMING (CS61064)

Spring 2023



Instructor

Soumyajit Dey (GPU, CUDA),
Pralay Mitra (OpenMP/MPI paradigm),

Teaching Assistant

Srijeeta Maity , Danny Jeron Pereira (GPU, CUDA)
Dibya Kanti Halder, Purnima Gautam, Debraj Das (Open-MP/MPI)

Class timing

MON(10:00-10:55) , WED(08:00-08:55) , WED(09:00-09:55) , THURS(10:00-10:55)

Venue

Lectures: CSE room 120
Tutorials: PC Lab, Annex building, CSE

Announcements

First meeting date : 4th January 2023 at scheduled time

Prerequisites: Proficiency in C programming

(Programming environments used in course shall be restricted to OpenMP, MPI, CUDA)

Course Modules

Serial no Topic (GPU/CUDA) Slides
Module 0.1 Basics of Computer Architecture Download
Module 0.2 GPU Architecture Download
Module 1 Introduction to CUDA, Multi-dimensional Mapping Download , Download
Module 2 Warp Scheduling and Divergence Download
Module 3 Memory, Tiled Matrix Multiplication
Module 4 Transpose Operation, Reduction Operations
Module 5 Fusion,Coarsening, Neural Network Training/Inferencing

Serial no Topic (OpenMP/MPI)
Module 0 OpenMP introduction, worksharing constructs
Module 1 OpenMP scheduling, handling array
Module 2 OpenMP synchronization; announcement of term project on OpenMP
Module 3 OpenMP Matrix handling
Module 4 MPI Part I
Module 5 MPI Part II

Syllabus Click here

NVIDIA CUDA Programming Guide Click here

Tutorials

Tutorial No. Date Topic
Tutorial 1 25th January Hands-on basic programming in CUDA
Tutorial 2 To be announced Divergence in CUDA
Tutorial 3 To be announced Reduction in CUDA
Tutorial 4 To be announced Coarsening and fusion in CUDA

CUDA End Term Projects

To be announced

Test Schedule

Test Name Date and Time
Test 1 To be announced

Marking Scheme

To be announced

References

1. “Using OpenMP” by Barbara Chapman, Gabriele Jost and Ruud van der Pas
2. “MPI: The Complete Reference” by Marc Snir, Jack Dongarra, Janusz S. Kowalik, Steven Huss-Lederman, Steve W. Otto, David W. Walker
3. “Parallel Programming with MPI” by Peter Pacheco
4. "Programming Massively Parallel Processors" - David Kirk and Wen-mei Hwu
5. CUDA Reference manual
6. “Computer Architecture -- A Quantitative Approach” - John L. Hen- nessy and David A. Patterson
If you are absent for two consecutive classes, turn up with documented reasons in the next class. Attendance below institute guidelines shall lead to deregistration - Yes, we are following this strictly