CS446/ECE449: Machine Learning (Fall 2022)

Course Information

The goal of Machine Learning is to find structure in data. In this course we will cover three main areas, (1) discriminative models, (2) generative models, and (3) reinforcement learning models. In particular we will cover the following: decision trees, Naive Bayes, Gaussian Bayes, linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.

Pre-requisites: Probability, linear algebra, and proficiency in Python.

Recommended Text: (1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Machine Learning, Tom Mitchell, (3) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (4) Pattern Recognition and Machine Learning by Christopher Bishop, (5) Graphical Models by Nir Friedman and Daphne Koller, and (6) Reinforcement Learning by Richard Sutton and Andrew Barto, (7) Understanding Machine Learning by Shai Shalev-Shwartz and Shai Ben-David.

Course Deliverables:
(1) Homework due at noon, see below for dates
(2) Midterm
(3) Final

3 credit: Homework 60% (drop 1 homework), Midterm 20%, Final 20%
4 credit: Homework 60% (drop 0 homework), Midterm 20%, Final 20%

Grading policy is subject to change.

Office Hours: [Zoom link],
Monday 3-4pm; Tuesday 8-9pm; Wednesday 9-10am; Thursday 6-7pm; Friday 2-3pm.

Late Policy: 3 late days in total.

Instructor & TAs

Liangyan Gui

Email: lgui[at]illinois.edu

Shenlong Wang

Email: shenlong[at]illinois.edu

Junkun Chen

Teaching Assistant
Email: junkun3[at]illinois.edu

Sirui Xu

Teaching Assistant
Email: siruixu2[at]illinois.edu

Vlas Zyrianov

Teaching Assistant
Email: vlasz2[at]illinois.edu

Min Jin Chong

Teaching Assistant
Email: mchong6[at]illinois.edu


Class Time: Wednesday, Friday 12:30-1:45PM
Location: [Zoom link].
Campuswire for discussions: [link] (code in class email)
GradeScope for assignments : [link] (code in class email)


The syllabus is subject to change.

Lecture 1 08/24/2022 Introduction [Slides]  
Lecture 2 08/26/2022 Decision Tree Learning [Slides]  
Lecture 3 08/31/2022 Probability and Estimation [Slides]  
Assignment Due 08/31/2022 Assignment 0 Due (11:59AM Central Time)    
Lecture 4 09/02/2022 Naive Bayes [Slides]  
Lecture 5 09/07/2022 Gaussian Naive Bayes [Slides]  
Lecture 6 09/09/2022 Logistic Regression [Slides]  
Lecture 7 09/14/2022 Linear Regression [Slides]  
Assignment Due 09/14/2022 Assignment 1 Due (11:59AM Central Time)    
Lecture 8 09/16/2022 Learning theory [Slides]  
Lecture 9 09/21/2022 Kernel methods and SVM [Slides]  
Lecture 10 09/23/2022 SVM [Slides]  
Lecture 11 09/28/2022 SVM II, Ensemble Methods [Slides]  
Assignment Due 09/28/2022 Assignment 2 Due (11:59AM Central Time)    
Lecture 12 09/30/2022 PyTorch Tutorial    
Lecture 13 10/05/2022 Deep Nets 1 [Slides]  
Lecture 14 10/07/2022 Deep Nets 2 & Midterm Review [Slides][Midterm Review]  
10/12/2022 Midterm Exam    
Lecture 15 10/14/2022 PCA, SVD [Slides]  
Assignment Due 10/16/2022 Assignment 3 Due (11:59AM Central Time)    
Lecture 16 10/19/2022 K-Means [Slides]    
Lecture 17 10/21/2022 Gaussian Mixture Models [Slides]    
Lecture 18 10/26/2022 Expectation Maximization [Slides]      
Lecture 19 10/28/2022 Variational Auto-Encoders [Slides]      
Lecture 20 11/02/2022 Generative Adversarial Nets [Slides]      
Assignment Due 11/02/2022 Assignment 4 Due (11:59AM Central Time)    
Lecture 21 11/04/2022 Autoregressive Methods [Slides]      
Lecture 22 11/09/2022 Energy-based Models [Slides]      
Lecture 23 11/11/2022 Markov Decision Process [Slides]      
Lecture 24 11/16/2022 Q Learning [Slides]      
Assignment Due 11/16/2022 Assignment 5 Due (11:59AM Central Time)    
Lecture 25 11/18/2022 Policy Gradient [Slides]      
11/23/2022 Fall Break    
11/25/2022 Fall Break    
Lecture 26 11/30/2022 Attention, Transformers [Slides]      
Assignment Due 11/30/2022 Assignment 6 Due (11:59AM Central Time)    
Lecture 27 12/02/2022 Self-Supervised Learning [Slides]      
Lecture 28 12/07/2022 Review [Slides]      
Exam 12/16/2022 Final Exam