CS446/ECE449: Machine Learning (Spring 2023)

Course Information

The goal of Machine Learning is to find structure in data. In this course we will cover three main areas, (1) discriminative models, (2) generative models, and (3) reinforcement learning models. In particular we will cover the following: linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.

Pre-requisites: Probability, linear algebra, and proficiency in Python.

Recommended Text: (1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (3) Pattern Recognition and Machine Learning by Christopher Bishop, (4) Graphical Models by Nir Friedman and Daphne Koller, (5) Reinforcement Learning by Richard Sutton and Andrew Barto, (6) Understanding Machine Learning by Shai Shalev-Shwartz and Shai Ben-David

Course Deliverables:
(1) Homework due at noon, see below for dates
(2) Quiz 1
(3) Quiz 2

Grading:

3 credit: 60% homework (drop 1 homework), 20% midterm, 20% final
4 credit: 60% homework (drop 0 homework), 20% midterm, 20% final


Late Policy 3 late days in total.

Grading policy is subject to change.

TA/Office Hours (see campuswire for link):
Please see calendar on this [link]


Instructor & TAs

Matus Telgarsky

Instructor
Email: mjt[at]illinois.edu
Website: [link]

Zhizhen Zhao

Instructor
Email: zhizhenz[at]illinois.edu
Website: [link]

Hongyu Shen

Teaching Assistant
Email: hongyu2[at]illinois.edu
Website: [link]

Yichi Zhang

Teaching Assistant
Email: yichi5[at]illinois.edu
Website: [link]

Ali Ebrahimpour

Teaching Assistant
Email: ae20[at]illinois.edu
Website: [link]

Huichen Li

Teaching Assistant
Email: huichenli[at]gmail.com
Website: [link]

Justin Li

Teaching Assistant
Email: jdli3[at]illinois.edu
Website: [link]

Jason Ren

Teaching Assistant
Email: zr5[at]illinois.edu
Website: [link]

Class Time & Location

Class Time: Tuesday, Thursday 12:30PM-1:45PM (hybrid/online; see campuswire for link)

Discussion & Homework submission

Campuswire: [link] (code in class email)
Gradescope: [link] (code: 4V5ER6)

Material & Info

Material: [link]



Lectures

The syllabus is subject to change.

EventDateDescriptionSlidesRecordingMaterialFormat
Lecture 1 01/17/2023 Overview [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 2 01/19/2023 Probability and Estimation [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 3 01/24/2023 Introduction to Optimization [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 4 01/26/2023 Principal Component Analysis (PCA) [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 5 01/31/2023 k-Means [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 6 02/02/2023 Gaussian Mixture Models [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 7 02/07/2023 Expectation Maximization [Slides] [Slides Split] [Slides Annot] [Rec]   online
HW 02/07/2023 Homework 1 (due at noon)     [HW] online
Lecture 8 02/09/2023 Deep Nets 1 [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 9 02/14/2023 Deep Nets 2 [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 10 02/16/2023 Variational Auto-Encoders [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 11 02/21/2023 Generative Adversarial Nets [Slides] [Slides Split] [Slides Annot] [Rec]   online
HW 02/21/2023 Homework 2 (due at noon)     [HW] online
Lecture 12 02/23/2023 Diffusion Models [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 13 02/28/2023 Autoregressive Methods [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 14 03/02/2023 Review [Slides] [Slides Split] [Slides Annot] [Rec]   online
HW 03/02/2023 Homework 3 (due at noon)     [HW] online
Midterm 03/07/2023 Quiz 1       online
Lecture 15 03/09/2023 Attention and Transformers [Slides] [Slides Split] [Slides Annot] [Rec]   online
Break 03/14/2023 Break       online
Break 03/16/2023 Break       online
Lecture 16 03/21/2023 Linear Regression [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 17 03/23/2023 Logistic Regression [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 18 03/28/2023 Linear Prediction: features, overfitting and losses [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 19 03/30/2023 Convex Optimization [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 20 04/04/2023 Support Vector Machine 1 [Slides] [Slides Split] [Slides Annot] [Rec]   online
HW 04/04/2023 Homework 4 (due at noon)     [HW] online
Lecture 21 04/06/2023 Support Vector Machine 2 [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 22 04/11/2023 Ensemble Methods [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 23 04/13/2023 Reinforcement Learning 1 [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 24 04/18/2023 Reinforcement Learning 2 [Slides] [Slides Split] [Slides Annot] [Rec]   online
HW 04/18/2023 Homework 5 (due at noon)     [HW] online
Lecture 25 04/20/2023 Learning Theory 1 [Slides] [Slides Split] [Slides Annot] [Rec]   online
Lecture 26 04/25/2023 Learning Theory 2 [Slides] [Slides Split] [Slides Annot] [Rec]   online
HW 04/25/2023 Homework 6 (due at noon)     [HW] online
Lecture 27 04/27/2023 Review [Slides] [Slides Split] [Slides Annot] [Rec]   online
Final 05/02/2023 Quiz 2       online