CS446/ECE449: Machine Learning (Spring 2022)

Course Information

The goal of Machine Learning is to find structure in data. In this course we will cover three main areas, (1) discriminative models, (2) generative models, and (3) reinforcement learning models. In particular we will cover the following: linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.

Pre-requisites: Probability, linear algebra, and proficiency in Python.

Recommended Text: (1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (3) Pattern Recognition and Machine Learning by Christopher Bishop, (4) Graphical Models by Nir Friedman and Daphne Koller, (5) Reinforcement Learning by Richard Sutton and Andrew Barto, (6) Understanding Machine Learning by Shai Shalev-Shwartz and Shai Ben-David

Course Deliverables:
(1) Homework due at noon, see below for dates (no late submission accepted)
(2) Midterm
(3) Midterm 2

Grading:

3 credit: 60% homework (drop 1 homework), 20% midterm, 20% midterm 2
4 credit: 60% homework (drop 0 homework), 20% midterm, 20% midterm 2


Grading policy is subject to change.

TA/Office Hours:
Please see calendar on this [link]

Midterm 2: May 03 2022 during regular class time (attendance mandatory)

Instructor & TAs

Matus Telgarsky

Instructor
Email: mjt[at]illinois.edu
Website: [link]

Alexander Schwing

Instructor
Email: aschwing[at]illinois.edu
Website: [link]

Jing Liu

Senior Teaching Assistant
Email: jil292[at]illinois.edu
Website: [link]

Justin Li

Teaching Assistant
Email: jdli3[at]illinois.edu
Website: [link]

Jinglin Chen

Teaching Assistant
Email: jinglinc[at]illinois.edu
Website: [link]

Huichen Li

Teaching Assistant
Email: huichenli[at]gmail.com
Website: [link]

Yuan-Ting Hu

Teaching Assistant
Email: ythu2[at]illinois.edu
Website: [link]

Zhongzheng Ren

Teaching Assistant
Email: zr5[at]illinois.edu
Website: [link]

Xiaoming Zhao

Teaching Assistant
Email: xz23[at]illinois.edu
Website: [link]

Class Time & Location

Class Time: Tuesday, Thursday 12:30PM-1:45PM (hybrid/online; see campuswire for link)

Discussion & Homework submission

Campuswire: [link] (code in class email)
Gradescope: [link] (code: RWP5E2)

Material & Info

Material: [link]



Lectures

The syllabus is subject to change.

EventDateDescriptionSlidesRecordingMaterialFormat
Lecture 1 01/18/2022 Overview; start of linear regression [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 2 01/20/2022 Linear Regression [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 3 01/25/2022 Logistic Regression [Slides] [Slides Split] [Slides Annot] [Rec]   hybrid
Lecture 4 01/27/2022 Linear prediction: features, overfitting, and losses [Slides] [Slides Split] [Slides Annot] [Rec]   hybrid
Lecture 5 02/01/2022 Convex optimization [Slides] [Slides Split] [Slides Annot] [Rec]   hybrid
Lecture 6 02/03/2022 Support Vector Machines 1 [Slides] [Slides Split] [Slides Annot] [Rec]   hybrid
Lecture 7 02/08/2022 Support Vector Machines 2 [Slides] [Slides Split] [Slides Annot] [Rec]   hybrid
HW 02/08/2022 Homework 1 (due at noon)     [HW] online
Lecture 8 02/10/2022 Deep Nets 1 [Slides] [Slides Split] [Slides Annot] [Rec]   hybrid
Lecture 9 02/15/2022 Pytorch Tutorial [Slides] [Slides Split] [Slides Annot] [Rec]   hybrid
Lecture 10 02/17/2022 Deep Nets 2 [Slides] [Slides Split] [Slides Annot] [Rec]   hybrid
Lecture 11 02/22/2022 Nearest Neighbors and decision trees [Slides] [Slides Split] [Slides Annot] [Rec]   hybrid
HW 02/22/2022 Homework 2 (due at noon)     [HW] online
Lecture 12 02/24/2022 Ensemble methods [Slides] [Slides Split] [Slides Annot] [Rec]   hybrid
Lecture 13 03/01/2022 Learning theory [Slides] [Slides Split] [Slides Annot] [Rec]   hybrid
Lecture 14 03/03/2022 Review [Slides] [Slides Split] [Slides Annot] [Rec]   hybrid
HW 03/03/2022 Homework 3 (due at noon)     [HW] online
Midterm 03/08/2022 Midterm       online
Lecture 15 03/10/2022 PCA [Slides] [Slides Split] [Slides Annot] [Rec]   online
Break 03/15/2022 Break       online
Break 03/17/2022 Break       online
Lecture 16 03/22/2022 k-Means [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 17 03/24/2022 Gaussian Mixture Models [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 18 03/29/2022 Expectation Maximization [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 19 03/31/2022 Variational Auto-Encoders [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 20 04/05/2022 Generative Adversarial Nets [Slides] [Slides Split] [Slides Annot] [Rec]   online
HW 04/05/2022 Homework 4 (due at noon)     [HW] online
Lecture 21 04/07/2022 Autoregressive Methods [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 22 04/12/2022 Transformers [Slides] [Slides Split] [Slides Annot] [Rec]   online
HW 04/14/2022 Homework 5 (due at noon)     [HW] online
Lecture 23 04/14/2022 Graph Neural Nets [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 24 04/19/2022 MDP [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 25 04/21/2022 Q-Learning [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 26 04/26/2022 Actor-Critic & Policy Gradient [Slides] [Slides Split] [Slides Annot] [Rec]   online
HW 04/26/2022 Homework 6 (due at noon)     [HW] online
Lecture 27 04/28/2022 Review [Slides] [Slides Split] [Slides Annot] [Rec]   online
Midterm 2 05/03/2022 Midterm 2       online