Statistical Learning Theory
(CS60018)

Instructor: Pabitra Mitra CSE, pabitra@cse.iitkgp.ac.in, Phone: 03222-282356


Course Outline:
Statistical learning Theory will include topics on theoretical foundations of machone learning. Topics to be discussed consist the following:

    Learnability, PAC Learning, Uniform Learnability
    Vapnik-Chervonenkis dimension, Growth Functions
    Structural Risk Minimization
    Weak Learnability and Boosting
    Online Learning Models

Books:

1. Understanding Machine Learning: From Theory to Algorithms, by Shai Shalev-Shwartz and Shai Ben-David, Cambridge University Press (referred as UML)
Understanding Machine Learning: From Theory to Algorithms, Lecture Videos
2. Foundations of Machine Learning, by Mehryar Mohri, Afshin Rostamizadeh and Ameet Talwalkar, MIT Press
3. An Introduction to Computational Learning Theory, by Michael J. Kearns and Umesh Vazirani, MIT Press (referred as COLT)

Lecture Schedule:

Topic and Slides
Notes
Additional Readings
Introduction, Sample complexity bound for
learning axis parallel rectangles.

Chapter 1 of Kearnes Vazirani book (COLT book)

Probability Concepts
Appendix B of UML book

Definition of PAC Learning
Chapter 1-2 of Shai Shalev-Shwartz and Shai Ben-David book (UML book)
A Theory of the Learnable - Valiant
PAC Learnability of Finite Hypothesis Classes, Empirical Risk Minimization
Chapter 3 of UML book

Agnostic PAC Learnability of Finite Hypothesis Class, Uniform Convergence
Chapter 4 of UML book

VC dimension
Chapter 6 of UML book

Shattering
Chapter 6 of UML book

Growth Function
Chapter 6 of UML book

Fundamental Theorem of Statistical Learning Theory
Chapter 6 of UML book

Rademacher Complexity
Chapter 26 of UML book

Nonuniform Learnability, Structural Risk Minimization
Chapter 7 of UML book

Weak Learnability, Adaboost
Chapter 10 of UML book
Adaboost by Prof. Robert Schapire
Online Learning
Chapter 21 of UML book
Talk by Prof. Avrim Blum
Perceptron Learning
Chapter 21,of UML book

Convex learning Problems
Chapter 12 of UML book


Homework Assignments 2025:

Homework 1
Homework 2
Homework 3


Homework Assignments 2019:

Homework 1
Homework 2
Homework 3
Homework 4

Homework Assignments 2018:

Homework 1
Homework 2
Homework 3
Homework 4

Exams:
Midsem 2025 Endsem 2025    Midsem 2019 Endsem 2019    Midsem 2018 Endsem 2018

Term Project