Scalable Data Mining (CS60021)

Instructors: Sourangshu Bhattacharya

Teaching Assistants: Suman Bera, Saptarshi Mondal, Vaishnovi Arun

Class Schedule: MON(11:00-11:55) , TUE(08:00-08:55) , TUE(09:00-09:55)

Classroom: NC - 121

Last year course website: https://cse.iitkgp.ac.in/~sourangshu/coursefiles/cs60021_2024a.html

Announcements:

First class will be on Monday 28th July.
First class test will be on 12th August in the class. Syllabus till the first two weeks.
First Assignment has been uploaded on moodle. Deadline 17th August.

Course Schedule:

Week	Dates	Topic / Activity	Links / Material
Week 1	28/7, 29/7	Introduction to ML DL, Stochastic Gradient Descent	Slides - Intro
Week 2	4/8, 5/8	SGD convergence, Accelarated SGD, Convergence rate SGD, Linear-rate SGD methods	Slides - SGD Convergence, Accelerated SGD, Slides - SGD Convergence rate
Week 3	11/8, 12/8	Convergence rate SGD, Linear-rate SGD methods, Batch-normalization	Slides - Batch-normalization
Week 4	18/8, 19/8	ADMM for distributed loss minimization	Slides - ADMM
Week 5	25/8, 26/8,	Hadoop + Spark	Slides - Hadoop, Spark
Week 6 + 7	2/9, 8/9	DL frameworks	Slides - Pytorch
Week 7 + 8	9/9, 15/9, 16/9	Nearest Neighbor Search	Slides - LSH, HNSW
Week 9	6/10, 7/10	Subset Selection	Slides - Submodular Functions, Sparse Approximation, Convex Online
Week 10	13/10, 14/10	Streaming - Reservoir sampling, Bloom Filters, Cuckoo filters	Slides - Reservoir sampling, Bloom Filters, Cuckoo filters
Week 11	20/10, 21/10	Streaming - Distinct count	Slides - Flajolet Martin, K-min value sketeches, Median of Means estimation
Week 12 + 13	27/10, 28/10, 3/11, 4/11	Streaming - Frequent count	Slides - Misra-Gries, Space-saving, Count-min, Count sketeches.

Syllabus:

Optimization and Machine learning algorithms:

Optimization algorithms: Stochastic gradient descent, Variance reduction, Momentum algorithms, ADAM.
Algorithms for distributed optimization: Distributed stochastic gradient descent and related methods. ADMM and decomposition methods.
(New) Federated Learning.

Software paradigms:

Big Data Processing: Motivation and Fundamentals, Map-reduce framework, Functional programming, and Scala Programming using map-reduce paradigm, Example programs.
Deep Learning Frameworks (Pytorch): Motivation, Computation graphs, Tensors, Autograd, Modules, Example programs.

Algorithmic techniques:

Finding similar items: Shingles, Minhashing, Locality Sensitive Hashing families, FAISS.
Stream processing: Motivation, Sampling, Bloom filtering, Count-based sketches: FM sketch, AMS sketch, Hash-based sketches: count sketch.
Subset Selection Methods: Submodular Optimization, Sparse Approximation, Convex Optimisation.

References:

Mining of Massive Datasets. 2nd edition. - Jure Leskovec, Anand Rajaraman, Jeff Ullman. Cambridge University Press. http://www.mmds.org/
Tensorflow for Machine Intelligence: A hands-on introduction to learning algorithms. Sam Abrahams et al. Bleeding edge press.
Hadoop: The Definitive Guide. Tom White. O'Reilly Press.
Recent literature.