CS60050 : Machine Learning Spring 2023, L-T-P: 3-0-0

Schedule

Instructor     Aritra Hazra
Timing (Slot: A3)     Monday (08:00 – 10:00)   and   Tuesday (12:00 – 13:00)
Venue     NC443   (Nalanda Complex)
Teaching Assistants     Abhinav Bohra   |   Naincy Vimal   |   Rijoy Mukherjee   |   Somnath Hazra   |   Suryansh Kumar   |   Suvadeep Hajra

Notices and Announcements

April 20, 2023
The graded answers scripts for End-semester examination will be shown on Tuesday (02-May-2023) from 11:00 AM - 11:30 AM in CSE-120 (Ground Floor of CSE Dept.).

March 10, 2023
End-semester examination is scheduled on 18-Apr-2023 (AN) and will be conducted centrally by the institute. The syllabus is everything that is covered in the course.

March 26, 2023
Project (Phase-3) has been released and you have already received an email with the necessary details. Please note that the submission deadline is 15-Apr-2023 (STRICT)! The submissions will be accepted (only) via CSE-Moodle.

February 26, 2023
Project (Phase-2) has been released and you have already received an email with the necessary details. Please note that the submission deadline is 18-Mar-2023 (STRICT)! The submissions will be accepted (only) via CSE-Moodle.

February 24, 2023
The graded answers scripts for Mid-semester examination will be shown on Tuesday (28-Feb-2023) from 6:00 PM - 6:30 PM in CSE-120 (Ground Floor of CSE Dept.).

February 07, 2023
Mid-semester examination is scheduled on 15-Feb-2023 (AN) and will be conducted centrally by the institute. The syllabus is whatever covered till today's class (i.e. upto SVM).

January 22, 2023
Project (Phase-1) has been released and you have already received an email with the necessary details. Please note that the submission deadline is 11-Feb-2023 (STRICT)! The submissions will be accepted (only) via CSE-Moodle.

January 01, 2023
Hello everyone! Wish you all a very Happy New Year!
It is time for you all to understand and learn about how machines learn, which is the primary subject matter of this course. So, here we go ...
  • The first class for this course will be held on 03-Jan-2023 (Tue, 12pm) at NC443 (Nalanda).
  • You need to enrol into our course from CSE-Moodle too. If you are new to CSE-Moodle (this is not the Institute-Moodle!), please register there first. Then you can login/enrol. An email has been sent already detailing the same with the enrolment-key.
  • To initiate our proceedings, the registered students are requested to go through the materials/slides posted under PROLOGUE part, to get an informal overview (metadata) of the course.

December 31, 2022 – 08:00pm
Shortlisting has been finalized based on all the requests available until now via ERP portal. No further requests / emails will be processed / addressed. Due to operational constraints, we are unable to consider all requests! (Only) the approved / registered students will receive an introductory mail soon (preferably by tomorrow) with all the required details.

Selection Criteria Followed for Shortlisting Students:
  • Research Scholar Students (MS and PhD) : Everyone
  • CS Students (UG and PG) : CGPA >= 7.00   (Exception for UG-20 Batch : CGPA >= 8.25)
  • Non-CS Students (UG and PG) : CGPA >= 9.00   (Exception for Dual-18 Batch : CGPA >= 8.00)
  • Declined: All students who do not satisfy the above approval criteria.

December 22, 2022
Interested students are requested to apply via the ERP portal. Please keep in mind the following requirements.

  • The timings for this course as mentioned above are to be treated as frozen and final. These timings are chosen so as to reduce clashes with other courses. However, it is impossible to find any slot that is free for everybody. So any further change may solve a few cases but will introduce new cases of clash. An applicant must not have any clash with these timings. If so, the students are supposed to make attempts to shift the other courses/labs clashing with these.

  • I shall consider requests via ERP (only) from all students till 8:00pm, 31-Dec-2022 and finalize the approvals among those requested students based on their CGPA (only) immediately after 8:00pm (if possible). Considering the first class to be held on 03-Jan-2023, the declined students are requested to switch over to other courses before the subject registration deadline expires. We shall NOT consider any further request for approval beyond that. All approvals by me in ERP will wait until that time. I may not respond to individual e-mails about the application and the application-approval processes. Please note that, I cannot take all requested students due to seat limitations. However, this selection will primarily be based/prioritized on -- (i) preferring the final year students and research scholars, and (ii) the CGPA of the students.

  • Pre-requisites for this course are:   Probability and Statistics, Linear Algebra (basics), Algorithms, Programming Knowledge (in C/C++/Java/Python).

Coverage

TopicDetailsDurationScheduleReferences
PROLOGUE (Metadata)

Course Information and Logistics.
Overview of Machine Learning – The Landscape.

–

Information

–
Introduction

The Learning Problem.
Feasibility of Learning.

2-hours

03-Jan-2023
09-Jan-2023

Mitchell [1] (Chapter-1)
Mostafa-Ismail-Lin [2] (Chapter-1)
Alpaydin [3] (Chapter-1+2)
Duda-Hart-Stork [7] (Chapter-1)
Concept Learning

General-to-Specific Hypotheses Ordering.
Find-S and Candidate Elimination Algorithm.
Version Space and Inductive Bias.

1.5-hour

09-Jan-2023

Mitchell [1] (Chapter-2)
Decision Tree Learning

Decision Tree Representation and Learning Algorithm (ID3).
Attribute Selection using Entropy Measures and Gains.
Hypotheses Space and Inductive bias.
Overfitting, Generalization and Occam's Razor.
Extensions: Continuous-valued Attributes, Alternative Attribute Selection Measures.

1.5-hour

10-Jan-2023
16-Jan-2023

Mitchell [1] (Chapter-3)
Alpaydin [3] (Chapter-9)
Tan-Steinbach-Kumar [4] (Chapter-4)
Bayesian Learning

Probability Overview.
MLE and MAP Estimates.
Naive Bayes Classifier.
Gaussian Naive Bayes Classifier.
Bayesian Networks.

4-hours

16-Jan-2023
17-Jan-2023
23-Jan-2023

Mitchell [1] (Chapter-6)
Tan-Steinbach-Kumar [4] (Chapter-5)
Bishop [5] (Chapter-8)
Alpaydin [3] (Chapter-14)
Instance-based Learning

k-Nearest Neighbour (kNN) Classifier.
Voronoi Diagram and Distance-Weighted kNN.
Distance Metrics and Curse of Dimensionality.
Computational Complexity: Condensing and High Dimensional Search (kd-tree).

2-hours

23-Jan-2023
24-Jan-2023

Mitchell [1] (Chapter-8)
Mostafa-Ismail-Lin [2] (Chapter-6)
Tan-Steinbach-Kumar [4] (Chapter-5)
Duda-Hart-Stork [7] (Chapter-4)
Linear Models and Regression

Linear Classification.
Linear Regression.
Non-linear Transformation.
Logistic Regression.

2-hours

30-Jan-2023

Mostafa-Ismail-Lin [2] (Chapter-3)
Alpaydin [3] (Chapter-10)
Tan-Steinbach-Kumar [4] (Chapter-5)
Hastie-Tibshirani-Friedman [6] (Chapter-2+3)
Artificial Neural Network

Perceptron Learning Algorithm: Delta Rule and Gradient Descent.
Multi-layer Perceptron Learning: Backpropagation and Stochastic Gradient Descent.
Hypotheses Space, Inductive Bias and Convergence.
Variants of Neural Network Structures.

2-hours

31-Jan-2023
06-Feb-2023

Mitchell [1] (Chapter-4)
Mostafa-Ismail-Lin [2] (Chapter-7)
Alpaydin [3] (Chapter-11)
Tan-Steinbach-Kumar [4] (Chapter-5)
Hastie-Tibshirani-Friedman [6] (Chapter-11)
Support Vector Machines

Decision Boundary and Support Vector: Optimization and Primal-Dual Problem.
Extension to SVM: Soft Margin and Non-linear Decision Boundary.
Kernel Functions and Radial Basis Functions (detailed later).

2-hours

06-Feb-2023
07-Feb-2023

Mostafa-Ismail-Lin [2] (Chapter-8)
Hastie-Tibshirani-Friedman [6] (Chapter-12)
Alpaydin [3] (Chapter-10)
Tan-Steinbach-Kumar [4] (Chapter-5)
Classifier / Hypothesis Evaluation

Accuracy, Precision, Recall and F-Measures.
Scores, Sampling, Bootstraping and ROC.
Hypotheses Testing and Cross-validation.

1-hour

13-Feb-2023

Mitchell [1] (Chapter-5)
Tan-Steinbach-Kumar [4] (Chapter-4+5)
Computational Learning Theory

Error and Noise Formalisms.
Training vs. Testing.
Theory of Generalization.
PAC Learnability and VC Dimensions.
Bias-Variance Tradeoffs.
Overfitting, Regularization and Validation.

6-hours

27-Feb-2023
28-Feb-2023
06-Mar-2023
07-Mar-2023

Mostafa-Ismail-Lin [2] (Chapter-2+4)
Shwartz-David [9]
Mitchell [1] (Chapter-7)
Unsupervised Learning: Clustering

Partitional Clustering and Hierarchical Clustering.
Cluster Types, Attributes and Salient Features.
k-Means, Hierarchical and Density-based Clustering Algorithms.
Inter and Intra Clustering Similarity, Cohesion and Separation.
MST and DBSCAN Clustering Algorithms.

2-hours

13-Mar-2023

Tan-Steinbach-Kumar [4] (Chapter-8+9)
Hastie-Tibshirani-Friedman [6] (Chapter-14)
Duda-Hart-Stork [7] (Chapter-10)
Ensemble Learning

Bagging and Boosting.
Adaboost and Random Forest.

2-hours

14-Mar-2023

Tan-Steinbach-Kumar [4] (Chapter-5)
Hastie-Tibshirani-Friedman [6] (Chapter-15)
Some Leftover Topics
(Additional Details)

Dimensionality Reduction and Principal Component Analysis (PCA).
Kernel Machines and Kernel Trick.
Radial Basis Functions (more details).
Core Learning Principles.
E-M Algorithm and Gaussian Mixture Model.
Hidden Markov Model.

4-hours

20-Mar-2023
21-Mar-2023
27-Mar-2023

Mostafa-Ismail-Lin [2] (Chapter-5+9+C)
Alpaydin [3] (Chapter-13+15)
Duda-Hart-Stork [7] (Chapter-3)
Murphy [8]
Other Learning Paradigms

Reinforcement Learning.
Deep Learning.

Transfer Learning.
Semi-supervised Learning.
Active Learning.

5-hours

27-Mar-2023
28-Mar-2023
03-Apr-2023
10-Apr-2023

Sutton-Barto [10]
Mitchell [1] (Chapter-13)
Alpaydin [3] (Chapter-18)
Goodfellow-Bengio-Courville [11]

(+ Research Survey Articles)
Conclusion

Explainability in Learning.

ML in Summary and Path Ahead.

1-hour

10-Apr-2023

Molnar [12]

(+ Research Survey Articles)

** For Reference Slides/Materials, Visit the following Course Pages:

  • Course by Dr. Pabitra Mitra (CSE, IIT Kharagpur)
  • Course by Dr. Yaser S. Abu-Mostafa (CalTech, USA)
  • Course by Dr. Tom Mitchell (CMU, USA)
  • Short Term Course by MIT

    • Books and References

    1. Tom Mitchell; Machine Learning, First Edition, McGraw Hill, 1997.   [ TEXTBOOK ]
    2. Yaser S. Abu-Mostafa, Malik Magdon-Ismail, Hsuan-Tien Lin; Learning From Data, First Edition, AML Book, 2012.   [ TEXTBOOK ]

    3. Ethem Alpaydin; Introduction to Machine Learning, Third Edition, The MIT Press, September 2014.
    4. Pang-Ning Tan, Michael Steinbach, Vipin Kumar; Introduction to Data Mining, Second Edition, Pearson Addison-Wesley, 2019.
    5. Christopher Bishop; Pattern Recognition and Machine Learning, First Edition, Springer-Verlag New York, 2006.    [ Open-Access ]
    6. Trevor Hastie, Robert Tibshirani, Jerome Friedman; The Elements of Statistical Learning, Second Edition, Springer, 2009.
    7. Richard O. Duda, Peter E. Hart, David G. Stork; Pattern Classification, Second Edition, John Wiley & Sons, November 2000.

    8. Kevin P. Murphy; Machine Learning: A Probabilistic Perspective, MIT Press, 2012.
    9. Shai Shalev-Shwartz, Shai Ben-David; Understanding Machine Learning: From Theory to Algorithms, First Edition, Cambridge University Press, 2014.
    10. Richard S. Sutton and Andrew G. Barto; Reinforcement Learning: An Introduction, 2nd Edition, MIT Press, 2020.
    11. Ian Goodfellow, Yoshua Bengio and Aaron Courville; Deep Learning, MIT Press, 2016.
    12. Christoph Molnar; Interpretable Machine Learning, Leanpub Publisher, 2019.

    Tests

    Projects

    Project Submission Page : CSE-Moodle
     CS60050 : Machine Learning Spring 2023, L-T-P: 3-0-0