CS446/ECE449: Machine Learning (Spring 2021)

Course Information

The goal of Machine Learning is to find structure in data. In this course we will cover three main areas, (1) discriminative models, (2) generative models, and (3) reinforcement learning models. In particular we will cover the following: linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.

Pre-requisites: Probability, linear algebra, and proficiency in Python.

Recommended Text: (1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (3) Pattern Recognition and Machine Learning by Christopher Bishop, (4) Graphical Models by Nir Friedman and Daphne Koller, (5) Reinforcement Learning by Richard Sutton and Andrew Barto, (6) Understanding Machine Learning by Shai Shalev-Shwartz and Shai Ben-David

Course Deliverables:
(1) Homework due at noon, see below for dates (no late submission accepted)
(2) Midterm
(3) Final

Grading:

3 credit: 60% homework (drop 1 homework), 20% midterm, 20% final
4 credit: 60% homework (drop 0 homework), 20% midterm, 20% final


Grading policy is subject to change.

TA Hours:
Time: Monday 5pm, Monday 8pm, Wednesday 4:30pm, Friday 10am, Friday 11am on campuswire.

Final Exam: TBD.

Instructor & TAs

Matus Telgarsky

Instructor
Email: mjt[at]illinois.edu
Office Hour: Tuesday 4:45-5:30pm (after class)
Website: [link]

Alexander Schwing

Instructor
Email: aschwing[at]illinois.edu
Office Hour: Tuesday 4:45-5:30pm (after class)
Website: [link]

Jing Liu

Senior Teaching Assistant
Email: jil292[at]illinois.edu
Office Hour: Wednesday from 4:30pm-5:30pm
Website: [link]

Priyank Agrawal

Teaching Assistant
Email: priyank4[at]illinois.edu
Office Hour: Friday from 11am-noon
Website: [link]

Ansel Blume

Teaching Assistant
Email: blume5[at]illinois.edu
Office Hour: Monday from 8pm-9pm
Website: [link]

Safa Messaoud

Teaching Assistant
Email: messaou2[at]illinois.edu
Office Hour: Friday from 10am-11am
Website: [link]

Efthymios Tzinis

Teaching Assistant
Email: etzinis2@illinois.edu
Office Hour: Monday from 1pm-2pm
Website: [link]

Xiaoming Zhao

Teaching Assistant
Email: xz23[at]illinois.edu
Office Hour: Monday from 5pm-6pm
Website: [link]

Class Time & Location

Class Time: Tuesday, Thursday 3:30PM-4:45PM (online; see campuswire for link)

Discussion & Homework submission

Campuswire: [link] (code in class email)
Gradescope: [link] (code: 3YPDNK)

Material & Info

Material: [link]



Lectures

The syllabus is subject to change.

EventDateDescriptionSlidesRecordingMaterial
Lecture 1 01/26/2021 Overview; start of linear regression [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 2 01/28/2021 Linear Regression [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 3 02/02/2021 Logistic Regression [Slides] [Slides Split] [Slides Annot] [Rec]  
HW 02/02/2021 Homework 0 (math & pytorch) (due at noon)     [HW]
Lecture 4 02/04/2021 Linear prediction: features, overfitting, and losses [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 5 02/09/2021 Convex optimization [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 6 02/11/2021 Support Vector Machines 1 [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 7 02/16/2021 Support Vector Machines 2 [Slides] [Slides Split] [Slides Annot] [Rec]  
HW 02/16/2021 Homework 1 (due at noon)     [HW]
Lecture 8 02/18/2021 Deep Nets 1 [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 9 02/23/2021 pytorch tutorial [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 10 02/25/2021 Deep Nets 2 [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 11 03/02/2021 Nearest Neighbors and decision trees [Slides] [Slides Split] [Slides Annot] [Rec]  
HW 03/02/2021 Homework 2 (due at noon)     [HW]
Lecture 12 03/04/2021 Ensemble methods [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 13 03/09/2021 Learning theory [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 14 03/11/2021 Review [Slides] [Slides Split] [Slides Annot] [Rec]  
HW 03/11/2021 Homework 3 (due at noon)     [HW]
Midterm 03/16/2021 Midterm      
Lecture 15 03/18/2021 PCA [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 16 03/23/2021 k-Means [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 17 03/25/2021 Gaussian Mixture Models [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 18 03/30/2021 Expectation Maximization [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 19 04/01/2021 Variational Auto-Encoders [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 20 04/06/2021 Generative Adversarial Nets [Slides] [Slides Split] [Slides Annot] [Rec]  
HW 04/06/2021 Homework 4 (due at noon)     [HW]
Lecture 21 04/08/2021 Autoregressive Methods [Slides] [Slides Split] [Slides Annot] [Rec]  
Break 04/13/2021 Break      
HW 04/15/2021 Homework 5 (due at noon)     [HW]
Lecture 22 04/15/2021 MDP [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 23 04/20/2021 Q-Learning [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 24 04/22/2021 Actor-Critic & Policy Gradient [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 25 04/27/2021 Graph Neural Nets [Slides] [Slides Split] [Slides Annot] [Rec]  
HW 04/27/2021 Homework 6 (due at noon)     [HW]
Lecture 26 04/29/2021 Transformers [Slides] [Slides Split] [Slides Annot] [Rec]  
Lecture 27 05/04/2021 Review [Slides] [Slides Split] [Slides Annot] [Rec]  
Final TBD Final