Machine Learning Spring 2024,   L-T-P: 3-0-0

Schedule

Instructor     Aritra Hazra
Timing (Slot: A3)     Monday (08:00 – 10:00)
Tuesday (12:00 – 13:00)
Venue     NC-324   (Nalanda Complex)
Teaching Assistants     Akash Kundu   |   Ayan Maity   |   Chappidi Yoga Satwik   |   Kajori Ghosh   |   Sumanta Dey   |   Suvadeep Hajra

Notices and Announcements

April 22, 2024
The graded answers scripts for End-semester examination will be shown on Thursday (02-May-2024) from 10:00 AM - 10:30 AM in CSE-120 (Ground Floor of CSE Dept.).

April 05, 2024
End-semester examination is scheduled on 18-Apr-2024 (AN) and will be conducted centrally by the institute. The syllabus is everything that is covered in the course.

March 23, 2024
Project (Phase-3) has been released and you have already received an email with the necessary details. Please note that the submission deadline is 13-Apr-2024 (STRICT)! The submissions will be accepted (only) via CSE-Moodle.

February 25, 2024
Project (Phase-2) has been released and you have already received an email with the necessary details. Please note that the submission deadline is 16-Mar-2024 (STRICT)! The submissions will be accepted (only) via CSE-Moodle.

February 19, 2024
The evaluated answers scripts for Mid-semester examination will be shown on Tuesday (27-Feb-2024) from 6:00 PM - 6:30 PM in CSE-120 (Ground Floor of CSE Dept.).

February 01, 2024
Mid-semester examination is scheduled on 15-Feb-2024 (AN) and will be conducted centrally by the institute. The syllabus is whatever covered upto Support Vetor Machines (SVM).

January 21, 2024
Project (Phase-1) has been released and you have already received an email with the necessary details. Please note that the submission deadline is 10-Feb-2024 (STRICT)! The submissions will be accepted (only) via CSE-Moodle.

January 19, 2024
As per Govt. of India mandate (conveyed by Ministry of Education), the institute shall remain closed for half-day (till 2:30pm) on 22-Jan-2024 (Monday). So, the class on 22-Jan-2024 (Mon, 8:00am-10:00am) is cancelled.

January 01, 2024
Hello everyone! Wish you all a very Happy New Year!
It is time for you all to understand and learn about how machines learn, which is the primary subject matter of this course. So, here we go ...
  • The first class for this course will be held on 02-Jan-2024 (Tue, 12pm) at NC-324 (Nalanda).
  • You need to enrol into our course from CSE-Moodle too. If you are new to CSE-Moodle (this is not the Institute-Moodle!), please register there first. Then you can login/enrol. An email has been sent already detailing the same with the enrolment-key.
  • To initiate our proceedings, the registered students are requested to go through the materials/slides posted under PROLOGUE part, to get an overview of the course and its logistics.

December 31, 2023
Shortlisting has been finalized based on all the requests available until now via ERP portal. No further requests/emails will be processed/addressed. Due to operational constraints, we are unable to consider all requests! (Only) the approved/registered students will receive an introductory mail soon (preferably by 01-Jan-2024) with all the required details.

Selection Criteria Followed for Shortlisting Students:
  • Research Students (MS and PhD) : Everyone
  • CS Students (UG and PG) : CGPA >= 7.00   (Exception for UG-21 Batch : CGPA >= 7.50)
  • Non-CS Students (UG and PG) : CGPA >= 8.75   (Exception for BTech-20 and Dual-19 Batch : CGPA >= 8.00)
  • Declined: All students who do not satisfy the above approval criteria.

December 18, 2023
Interested students are requested to apply via the ERP portal. Please keep in mind the following requirements.

  • The timings for this course as mentioned above are to be treated as frozen and final. These timings are chosen so as to reduce clashes with other courses. However, it is impossible to find any slot that is free for everybody. So any further change may solve a few cases but will introduce new cases of clash. An applicant must not have any clash with these timings. If so, the students are supposed to make attempts to shift the other courses/labs clashing with these.

  • I shall consider requests via ERP (only) from all students till 08:00pm, 30-Dec-2023 and finalize the approvals among those requested students based on their CGPA (only) immediately after 8:00pm (if possible). Considering the first class to be held on 02-Jan-2024, the declined students are requested to switch over to other courses before the subject registration deadline expires. We shall NOT consider any further request for approval beyond that. All approvals by me in ERP will wait until that time. I may not respond to individual e-mails about the application and the application-approval processes. Please note that, I cannot take all requested students due to seat limitations. However, this selection will primarily be based/prioritized on – (i) preferring the final year students and research scholars, and (ii) the CGPA of the students.

  • Pre-requisites for this course are:   Probability and Statistics, Linear Algebra (basics), Algorithms, Programming Knowledge (in C/C++/Java/Python).

Coverage

TopicDetailsDurationScheduleReferences
PROLOGUE (Metadata)

Course Information and Logistics.

–

Information

–
Introduction

Overview of Machine Learning – The Landscape.
The Learning Problem.
Feasibility of Learning.

2-hours

02-Jan-2024
08-Jan-2024

Mitchell [1] (Chapter-1)
Mostafa-Ismail-Lin [2] (Chapter-1)
Alpaydin [3] (Chapter-1+2)
Duda-Hart-Stork [7] (Chapter-1)
Concept Learning

General-to-Specific Hypotheses Ordering.
Find-S and Candidate Elimination Algorithm.
Version Space and Inductive Bias.

1.5-hour

08-Jan-2024
09-Jan-2024

Mitchell [1] (Chapter-2)
Decision Tree Learning

Decision Tree Representation and Learning Algorithm (ID3).
Attribute Selection using Entropy Measures and Gains.
Hypotheses Space and Inductive bias.
Overfitting, Generalization and Occam's Razor.
Extensions: Continuous-valued Attributes, Alternative Attribute Selection Measures.

1.5-hour

09-Jan-2024
15-Jan-2024

Mitchell [1] (Chapter-3)
Alpaydin [3] (Chapter-9)
Tan-Steinbach-Kumar [4] (Chapter-4)
Bayesian Learning

Probability Overview.
MLE and MAP Estimates.
Naive Bayes Classifier.
Gaussian Naive Bayes Classifier.
Bayesian Networks.

4-hours

15-Jan-2024
16-Jan-2024
23-Jan-2024

Mitchell [1] (Chapter-6 + Additional Chapters)
Bishop [5] (Chapter-8)
Tan-Steinbach-Kumar [4] (Chapter-5)
Alpaydin [3] (Chapter-14)
Instance-based Learning

k-Nearest Neighbour (kNN) Classifier.
Voronoi Diagram and Distance-Weighted kNN.
Distance Metrics and Curse of Dimensionality.
Computational Complexity: Condensing and High Dimensional Search (kd-tree).

2-hours

29-Jan-2024

Mitchell [1] (Chapter-8)
Mostafa-Ismail-Lin [2] (Chapter-6)
Tan-Steinbach-Kumar [4] (Chapter-5)
Duda-Hart-Stork [7] (Chapter-4)
Linear Models and Regression

Linear Classification.
Linear Regression.
Non-linear Transformation.
Logistic Regression.

2-hours

30-Jan-2024
05-Feb-2024

Mostafa-Ismail-Lin [2] (Chapter-3)
Alpaydin [3] (Chapter-10)
Tan-Steinbach-Kumar [4] (Chapter-5)
Hastie-Tibshirani-Friedman [6] (Chapter-2+3)
Artificial Neural Network

Perceptron Learning Algorithm: Delta Rule and Gradient Descent.
Multi-layer Perceptron Learning: Backpropagation and Stochastic Gradient Descent.
Hypotheses Space, Inductive Bias and Convergence.
Variants of Neural Network Structures.

2-hours

05-Feb-2024
06-Feb-2024

Mitchell [1] (Chapter-4)
Mostafa-Ismail-Lin [2] (Chapter-7)
Alpaydin [3] (Chapter-11)
Tan-Steinbach-Kumar [4] (Chapter-5)
Hastie-Tibshirani-Friedman [6] (Chapter-11)
Support Vector Machines

Decision Boundary and Support Vector: Optimization and Primal-Dual Problem.
Extension to SVM: Soft Margin and Non-linear Decision Boundary.
Kernel Functions and Radial Basis Functions (detailed later).

2-hours

06-Feb-2024
12-Feb-2024

Mostafa-Ismail-Lin [2] (Chapter-8)
Hastie-Tibshirani-Friedman [6] (Chapter-12)
Alpaydin [3] (Chapter-10)
Tan-Steinbach-Kumar [4] (Chapter-5)
Classifier/Hypothesis Evaluation

Accuracy, Precision, Recall and F-Measures.
Scores, Sampling, Bootstraping and ROC.
Hypotheses Testing and Cross-validation.

1-hour

12-Feb-2024

Mitchell [1] (Chapter-5)
Tan-Steinbach-Kumar [4] (Chapter-4+5)
Wikipedia Articles [Link-A, Link-B, Link-C]
Computational Learning Theory

Error and Noise Formalisms.
Training vs. Testing.
Theory of Generalization.
PAC Learnability and VC Dimensions.
Bias-Variance Tradeoffs.
Overfitting, Regularization and Validation.

6-hours

26-Feb-2024
27-Feb-2024
04-Mar-2024
05-Mar-2024

Mostafa-Ismail-Lin [2] (Chapter-2+4)
Shwartz-David [9]
Mitchell [1] (Chapter-7)
Unsupervised Learning: Clustering

Partitional Clustering and Hierarchical Clustering.
Cluster Types, Attributes and Salient Features.
k-Means, Hierarchical and Density-based Clustering Algorithms.
Inter and Intra Clustering Similarity, Cohesion and Separation.
MST and DBSCAN Clustering Algorithms.

2-hours

11-Mar-2024

Tan-Steinbach-Kumar [4] (Chapter-8+9)
Hastie-Tibshirani-Friedman [6] (Chapter-14)
Duda-Hart-Stork [7] (Chapter-10)
Ensemble Learning

Bagging and Boosting.
Adaboost and Random Forest.

2-hours

12-Mar-2024

Tan-Steinbach-Kumar [4] (Chapter-5)
Hastie-Tibshirani-Friedman [6] (Chapter-15)
Some Leftover Topics
(Additional Details)

Dimensionality Reduction and Principal Component Analysis (PCA).
Kernel Machines and Kernel Trick.
Radial Basis Functions (more details).
Core Learning Principles.
E-M Algorithm and Gaussian Mixture Model (GMM).
Hidden Markov Model (HMM).

5-hours

18-Mar-2024
19-Mar-2024
01-Apr-2024

Mostafa-Ismail-Lin [2] (Chapter-5+9+C)
Mitchell [1] (Chapter-6)
EM-GMM Slides by Dr. Tom Mitchell [Link]
HMM Slides by Dr. Pascal Poupart [Link]
Alpaydin [3] (Chapter-13+15)
Duda-Hart-Stork [7] (Chapter-3)
Murphy [8]
Other Learning Paradigms

Deep Learning.
Reinforcement Learning.

Transfer Learning.
Semi-supervised Learning.
Active Learning.

3.5-hours

02-Apr-2024
08-Apr-2024
09-Apr-2024

Sutton-Barto [10]
Mitchell [1] (Chapter-13)
Alpaydin [3] (Chapter-18)
Goodfellow-Bengio-Courville [11]

(+ Research Survey Articles)
Conclusion

Explainability in Learning.

ML in Summary and Path Ahead.

0.5-hour

09-Apr-2024

Slides by DARPA (2017) [Link]
Molnar [12]

(+ Research Survey Articles)

** For Reference Slides/Materials, Visit the following Course Pages:

  • Course by Dr. Pabitra Mitra (CSE, IIT Kharagpur)
  • Course by Dr. Yaser S. Abu-Mostafa (CalTech, USA)
  • Course by Dr. Tom Mitchell (CMU, USA)
  • Course by Dr. Maria-Florina Balcan (CMU, USA)
  • Short Term Course by MIT

    • Books and References

    1. Tom Mitchell; Machine Learning, First Edition, McGraw Hill, 1997.   [ TEXTBOOK ]
    2. Yaser S. Abu-Mostafa, Malik Magdon-Ismail, Hsuan-Tien Lin; Learning From Data, First Edition, AML Book, 2012.   [ TEXTBOOK ]

    3. Ethem Alpaydin; Introduction to Machine Learning, Fourth Edition, The MIT Press, March 2020.
    4. Pang-Ning Tan, Michael Steinbach, Vipin Kumar; Introduction to Data Mining, Second Edition, Pearson Addison-Wesley, 2019.
    5. Christopher Bishop; Pattern Recognition and Machine Learning, First Edition, Springer-Verlag New York, 2006.    [ Open-Access ]
    6. Trevor Hastie, Robert Tibshirani, Jerome Friedman; The Elements of Statistical Learning, Second Edition, Springer, 2009.
    7. Richard O. Duda, Peter E. Hart, David G. Stork; Pattern Classification, Second Edition, John Wiley & Sons, November 2000.

    8. Kevin P. Murphy; Machine Learning: A Probabilistic Perspective, MIT Press, 2012.
    9. Shai Shalev-Shwartz, Shai Ben-David; Understanding Machine Learning: From Theory to Algorithms, First Edition, Cambridge University Press, 2014.
    10. Richard S. Sutton and Andrew G. Barto; Reinforcement Learning: An Introduction, 2nd Edition, MIT Press, 2020.
    11. Ian Goodfellow, Yoshua Bengio and Aaron Courville; Deep Learning, MIT Press, 2016.
    12. Christoph Molnar; Interpretable Machine Learning, Leanpub Publisher, 2019.

    Tests

    Mini-Projects

    Project Submission Page : CSE-Moodle

    Previous Course Offerings:   Spring-2023   |   Spring-2021

    Machine Learning Spring 2024,   L-T-P: 3-0-0