Schedule

Instructor		Prof. Aritra Hazra
Timing		Wednesday (12:00–12:55), Thursday (11:00–11:55), Friday (09:00-09:55)
Venue		[ Online Class Sessions ] MS-Teams Meeting-Link :   Wednesday (12:00pm-01:00pm)   |   Thursday (11:00am-12:00pm)   |   Friday (09:00am-10:00am) [ Online Examinations ] CSE-Moodle Page-Link :   Course Page
Teaching Assistants		Aditya Rastogi, Jaspreet Singh Suddel, Jay Gopilal Bhutada, Milap Bhupendrabhai Radia, Prashant Tiwari, Priya Sharma, Ronit Samaddar, Rupak Kumar Thakur, Saurav Roy, Shubham Gautam, Vindhyansh Mall

Notices and Announcements

January 05, 2021

Hello everyone! It is time for you all to understand and learn about how machines learn, which is the primary subject matter of this course. So, here we go ...

• The first class for this course in Spring-2021 will be on 06-Jan-2021 (Wed, 12pm).

• We shall be conducting all the classes/sessions Live (online) through MS-Teams on regular dates/times (details are given above and emailed to you too).

• To initiate our proceedings, the registered students are requested to go through the materials/slides posted under PROLOGUE part, to get an informal overview (metadata) of the course.

January 04, 2021 – 08:00pm

Shortlisting has been finalized based on all the requests available until now via ERP portal. No further requests will be processed. Due to operational constraints, we are unable to consider all requests!
(Only) the registered students will receive a course welcome mail soon (preferably by tomorrow morning) with all the logistics and operational details.

Rules for Shortlisting in ML course:

• Seats with Automatic ERP Approvals (first come first served basis) : 75
          [   UG : 30   |   PG : 30   |   RS : 15   ]

• Additional Manual Approvals made: 40+   (requests taken upto 8:00pm, 04-Jan-2021)
  Selection Criteria followed:
      1.   For UG-16-Batch : CGPA >= 8.10
      2.   For UG-17-Barch : CGPA >= 8.60
      3.   For UG-18-Batch : CGPA >= 9.25
      4.   For UG-19-Batch : CGPA >= 9.25
      5.   For UG-19CS-Batch : CGPA >= 9.20
      6.   Declined: All students who do not satisfy the approval criteria.

December 29, 2020

Interested students are requested to apply via the ERP portal. Please keep in mind the following requirements.

• The timings for this course as mentioned above are to be treated as frozen and final. These timings are chosen so as to reduce clashes with other courses. However, it is impossible to find any slot that is free for everybody. So any further change may solve a few cases but will introduce new cases of clash. An applicant must not have any clash with these timings. If so, the students are supposed to make attempts to shift the other courses/labs clashing with these.

• The maximum capacity of the course is set as 70 (30 UG + 25 PG + 15 RS). Initial shortlisting will be made automatically by ERP on a first-come and first-served basis (and on no other criteria) until the capacity limit has not been exceeded.

• If the applicant counts are much more than the set capacity, we may (shall positively try to) select FEW extra students from the applicants pool. However, this selection will primarily be based/prioritized on - (i) preferring the final year students and research scholars, and (ii) the CGPA of the students.

• In order that the applicants who are not shortlisted can register for other courses, and keeping in view that the last date for course registration is 07-Jan-2021 (and our first class is on 06-Jan-2021), we consider only those ERP requests that are recorded by 04-Jan-2021 08:00pm. We shall finalize the shortlisting (if needed) immediately after that and inform (throgh email) all shortlisted students. All approvals by us in ERP will wait until that time. We may not respond to individual e-mails about the application and the application-approval processes.

• Pre-requisites for this course are:   Probability and Statistics, Linear Algebra (basics), Algorithms, Programming Knowledge (in C/C++/Java/Python).

Coverage

Topic	Details	Date	Handouts	Student-Scribes (Unedited)
PROLOGUE (Metadata)	Course Information and Logistics. Overview of Machine Learning – The Landscape.	–	Information	Schedule
Introduction	The Learning Problem. Feasibility of Learning.	06-Jan-2021 07-Jan-2021	Handout-01a Handout-01b	Scribe-01a Scribe-01b
Concept Learning	General-to-Specific Hypotheses Ordering. Find-S and Candidate Elimination Algorithm. Version Space and Inductive Bias.	08-Jan-2021 13-Jan-2021	Handout-02a Handout-02b	Scribe-02
Decision Tree Learning	Decision Tree Representation and Learning Algorithm (ID3). Attribute Selection using Entropy Measures and Gains. Hypotheses Space and Inductive bias. Overfitting, Generalization and Occam's Razor. Extensions: Continuous-valued Attributes, Alternative Attribute Selection Measures.	14-Jan-2021 15-Jan-2021	Handout-03a Handout-03b	Scribe-03a Scribe-03b
Bayesian Learning	Probability Overview. MLE and MAP Estimates. Naive Bayes Classifier. Gaussian Naive Bayes Classifier. Bayesian Networks.	20-Jan-2021 21-Jan-2021 22-Jan-2021 27-Jan-2021	Handout-04a Handout-04b Handout-04c Handout-04d	Scribe-04a Scribe-04b Scribe-04c Scribe-04d
Instance-based Learning	k-Nearest Neighbour (kNN) Classifier. Voronoi Diagram and Distance-Weighted kNN. Distance Metrics and Curse of Dimensionality. Computational Complexity: Condensing and High Dimensional Search (kd-tree).	28-Jan-2021 29-Jan-2021	Handout-05a Handout-05b	Scribe-05a Scribe-05b
Linear Models and Regression	Linear Classification. Linear Regression. Non-linear Transformation. Logistic Regression.	03-Feb-2021 04-Feb-2021	Handout-06a Handout-06b	Scribe-06a Scribe-06b
Artificial Neural Network	Perceptron Learning Algorithm: Delta Rule and Gradient Descent. Multi-layer Perceptron Learning: Backpropagation and Stochastic Gradient Descent. Hypotheses Space, Inductive Bias and Convergence. Variants of Neural Network Structures.	05-Feb-2021 10-Feb-2021	Handout-07a Handout-07b	Scribe-07a Scribe-07b
Support Vector Machines	Decision Boundary and Support Vector: Optimization and Primal-Dual Problem. Extension to SVM: Soft Margin and Non-linear Decision Boundary. Kernel Functions and Radial Basis Functions (detailed later).	11-Feb-2021 12-Feb-2021	Handout-08a Handout-08b	Scribe-08a Scribe-08b
Classifier / Hypothesis Evaluation	Accuracy, Precision, Recall and F-Measures. Scores, Sampling, Bootstraping and ROC. Hypotheses Testing and Cross-validation.	17-Feb-2021	Handout-09	Scribe-09
Computational Learning Theory	Error and Noise Formalisms. Training vs. Testing. Theory of Generalization. PAC Learnability and VC Dimensions. Bias-Variance Tradeoffs. Overfitting, Regularization and Validation.	18-Feb-2021 19-Feb-2021 24-Feb-2021 25-Feb-2021 26-Feb-2021 04-Mar-2021 05-Mar-2021	Handout-10a Handout-10b Handout-10c Handout-10d Handout-10e Handout-10f Handout-10g	Scribe-10a Scribe-10b Scribe-10c Scribe-10d Scribe-10e Scribe-10f Scribe-10g
Unsupervised Learning: Clustering	Partitional Clustering and Hierarchical Clustering. Cluster Types, Attributes and Salient Features. k-Means, Hierarchical and Density-based Clustering Algorithms. Inter and Intra Clustering Similarity, Cohesion and Separation. MST and DBSCAN Clustering Algorithms.	10-Mar-2021 11-Mar-2021	Handout-11a Handout-11b	Scribe-11a Scribe-11b
Ensemble Learning	Bagging and Boosting. Adaboost and Random Forest.	12-Mar-2021 17-Mar-2021	Handout-12a Handout-12b	Scribe-12a Scribe-12b
Some Leftover Topics (Additional Details)	Dimensionality Reduction and Principal Component Analysis (PCA). Kernel Machines and Radial Basis Function (more details). E-M Algorithm, Gaussian Mixture Model and Hidden Markov Model. Core Learning Principles.	18-Mar-2021 19-Mar-2021 24-Mar-2021 25-Mar-2021 26-Mar-2021	Handout-13a Handout-13b Handout-13c Handout-13d Handout-13e	Scribe-13a Scribe-13b Scribe-13c Scribe-13d Scribe-13e
Other Learning Paradigms	Reinforcement Learning. Deep Learning. Transfer Learning. Semi-supervised Learning. Active Learning.	31-Mar-2021 01-Apr-2021 07-Apr-2021 08-Apr-2021 09-Apr-2021	Handout-14a Handout-14b Handout-15a Handout-15b Handout-16+17	Scribe-14a Scribe-14b Scribe-15a Scribe-15b Scribe-16+17
Conclusion	Explainability in Learning. ML in Summary.	14-Apr-2021	Handout-18	Scribe-18

** For Reference Slides/Materials, Visit the following Course Pages:

Course by Dr. Pabitra Mitra (CSE, IIT Kharagpur)

Course by Dr. Yaser S. Abu-Mostafa (CalTech, USA)

Course by Dr. Tom Mitchell (CMU, USA)

Short Term Course by MIT

Books and References

1. Tom Mitchell, Machine Learning, First Edition, McGraw Hill, 1997.
2. Richard O. Duda, Peter E. Hart, David G. Stork, Pattern Classification, Second Edition, John Wiley & Sons, November 2000.
3. Christopher Bishop, Pattern Recognition and Machine Learning, First Edition, Springer-Verlag New York, 2006.    [ Open-Access ]
4. Ethem Alpaydin, Introduction to Machine Learning, Third Edition, The MIT Press, September 2014.
5. Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning, Second Edition, Springer, 2009.
6. Shai Shalev-Shwartz, Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms, First Edition, Cambridge University Press, 2014.
7. Yaser S. Abu-Mostafa, Malik Magdon-Ismail, Hsuan-Tien Lin, Learning From Data, First Edition, AML Book, 2012.

Tech Reports by Students (Unedited)

• Tech Report 0 : ML in Summary
• Tech Report 1 : Semi-Supervised Learning
• Tech Report 2 : Reinforcement Learning
• Tech Report 3 : Deep Learning
• Tech Report 4 : Transfer Learning
• Tech Report 5 : Explainability in ML

Tests

• Short Test 1  (01-Feb-2021, Monday, 8:30pm)   [ Syllabus: Topics from 06-Jan-2021 to 22-Jan-2021 classes ]
• Long Test 1  (22-Feb-2021, Monday, 8:30pm)   [ Syllabus: Topics from 27-Jan-2021 to 12-Feb-2021 classes ]
• Short Test 2  (15-Mar-2021, Monday, 8:30pm)   [ Syllabus: Topics from 17-Feb-2021 to 05-Mar-2021 classes ]
• Long Test 2  (05-Apr-2021, Monday, 8:30pm)   [ Syllabus: Topics from 10-Mar-2021 to 26-Mar-2021 classes ]

Projects

• Mini-Project 1  (Duration: 25-Jan-2021 – 11-Feb-2021)
          Group-Details  [ Check Your Group Members and Assigned Project Information here ]
          Project-Details  [ Browse appropriate Directory with your assigned Project Code here ]
          Evaluation-Details  [ For Queries, Please Contact Concerned TAs ]
  
  
• Mini-Project 2  (Duration: 15-Feb-2021 – 03-Mar-2021)
          Group-Details  [ Check Your Group Members and Assigned Project Information here ]
          Project-Details  [ Browse appropriate Directory with your assigned Project Code here ]
          Evaluation-Details  [ For Queries, Please Contact Concerned TAs ]
  
  
• Mini-Project 3  (Tentative Duration: 08-Mar-2021 – 25-Mar-2021)
          Group-Details  [ Check Your Assigned Project Information here ]
          Project-Details  [ Browse appropriate Directory with your assigned Project Code here ]
          Evaluation-Details  [ For Queries, Please Contact Concerned TAs ]
  
  
• Mini-Project 4  (Tentative Duration: 29-Mar-2021 – 12-Apr-2021)   [ If required ]