HIGH PERFORMANCE PARALLEL PROGRAMMING (CS61064)

Spring 2022


Instructor

Soumyajit Dey (GPU, CUDA),
Pralay Mitra (OpenMP/MPI paradigm),

Teaching Assistant

Srijeeta Maity (GPU, CUDA)
Dibya Kanti Halder, Sourav Garai (Open-MP/MPI)
Prerequisites : Proficiency in C programming
Programming environments used in course shall be restricted to OpenMP, MPI, CUDA

Live session

WED (1400-1800)

Venue - Online in MS Teams

Announcements

First meeting date : 5th January 2022 at scheduled time
In MS Teams, look for course HP3-2022

Course Modules

Serial no Topic (GPU/CUDA) Session Date
Module 0 Basics of Computer Architecture Jan 5th
Module 1 Introduction to CUDA, Multi-dimensional mapping Jan 19th
Module 2 Warp Scheduling and Divergence Feb 9th
Module 3 Memory, Tiled Matrix Multiplication, Term Project Distribution March 2nd
Module 4 Transpose Operation, Reduction Operations March 16th
Module 5 Fusion,Coarsening, Neural Network Training/Inferencing March 30th
Serial no Topic (OpenMP/MPI) Session Date
Module 0 OpenMP introduction, worksharing constructs Jan 12th
Module 1 OpenMP scheduling, handling array Feb 2nd
Module 2 OpenMP synchronization; announcement of term project on OpenMP Feb 16th
Module 3 OpenMP Matrix handling March 9th
Module 4 MPI Part I March 23rd
Module 5 MPI Part II April 6th

Test Schedule

Test Name Date and Time
Test 1 Feb 23, 3pm onwards
Test 2 April 13th, 3pm onwards
OpenMP Term Projects Submission Deadline : March 1st
CUDA Term Projects Submission Deadline : April 10th

Syllabus

Teaching material (CUDA)

Note: material on other topics are available with respective faculty members

Tutorials

CUDA End Term Projects

NVIDIA CUDA Programming Guide

Marking Scheme

TBA

References

1. “Using OpenMP” by Barbara Chapman, Gabriele Jost and Ruud van der Pas
2. “MPI: The Complete Reference” by Marc Snir, Jack Dongarra, Janusz S. Kowalik, Steven Huss-Lederman, Steve W. Otto, David W. Walker
3. “Parallel Programming with MPI” by Peter Pacheco
4. "Programming Massively Parallel Processors" - David Kirk and Wen-mei Hwu
5. CUDA Reference manual
6. “Computer Architecture -- A Quantitative Approach” - John L. Hen- nessy and David A. Patterson
If you are absent for two consecutive classes, turn up with documented reasons in the next class. Attendance below institute guidelines shall lead to deregistration - Yes, we are following this strictly