CS446/ECE449: Machine Learning (Fall 2022)

Course Information

The goal of Machine Learning is to find structure in data. In this course we will cover three main areas, (1) discriminative models, (2) generative models, and (3) reinforcement learning models. In particular we will cover the following: decision trees, Naive Bayes, Gaussian Bayes, linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.

Pre-requisites: Probability, linear algebra, and proficiency in Python.

Recommended Text: (1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Machine Learning, Tom Mitchell, (3) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (4) Pattern Recognition and Machine Learning by Christopher Bishop, (5) Graphical Models by Nir Friedman and Daphne Koller, and (6) Reinforcement Learning by Richard Sutton and Andrew Barto, (7) Understanding Machine Learning by Shai Shalev-Shwartz and Shai Ben-David.

Course Deliverables:
(1) Homework due at noon, see below for dates
(2) Midterm
(3) Final

3 credit: Homework 60% (drop 1 homework), Midterm 20%, Final 20%
4 credit: Homework 60% (drop 0 homework), Midterm 20%, Final 20%

Grading policy is subject to change.

Office Hours: [Zoom link],
Monday 3-4pm; Tuesday 8-9pm; Wednesday 9-10am; Thursday 6-7pm; Friday 2-3pm.

Late Policy: 3 late days in total.

Instructor & TAs

Liangyan Gui

Email: lgui[at]illinois.edu

Shenlong Wang

Email: shenlong[at]illinois.edu

Junkun Chen

Teaching Assistant
Email: junkun3[at]illinois.edu

Sirui Xu

Teaching Assistant
Email: siruixu2[at]illinois.edu

Vlas Zyrianov

Teaching Assistant
Email: vlasz2[at]illinois.edu

Min Jin Chong

Teaching Assistant
Email: mchong6[at]illinois.edu


Class Time: Wednesday, Friday 12:30-1:45PM
Location: [Zoom link].
Campuswire for discussions: [link] (code in class email)
GradeScope for assignments : [link] (code in class email)


The syllabus is subject to change.

Lecture 1 08/24/2022 Introduction [Slides]  
Lecture 2 08/26/2022 Decision Tree Learning [Slides]  
Lecture 3 08/31/2022 Probability and Estimation [Slides]  
Assignment Due 08/31/2022 Assignment 0 Due (11:59AM Central Time)    
Lecture 4 09/02/2022 Naive Bayes [Slides]  
Lecture 5 09/07/2022 Gaussian Naive Bayes [Slides]  
Lecture 6 09/09/2022 Logistic Regression [Slides]  
Lecture 7 09/14/2022 Linear Regression [Slides]  
Assignment Due 09/14/2022 Assignment 1 Due (11:59AM Central Time)    
Lecture 8 09/16/2022 Learning theory [Slides]  
Lecture 9 09/21/2022 Kernel methods and SVM [Slides]  
Lecture 10 09/23/2022 SVM [Slides]  
Lecture 11 09/28/2022 SVM II, Ensemble Methods [Slides]  
Assignment Due 09/28/2022 Assignment 2 Due (11:59AM Central Time)    
Lecture 12 09/30/2022 PyTorch Tutorial    
Lecture 13 10/05/2022 Deep Nets 1    
Lecture 14 10/07/2022 Deep Nets 2    
10/12/2022 Midterm Exam    
Assignment Due 10/16/2022 Assignment 3 Due (11:59AM Central Time)    
Lecture 15 10/14/2022 PCA, SVD, Auto-Encoder      
Lecture 16 10/19/2022 K-Means      
Lecture 17 10/21/2022 Gaussian Mixture Models      
Lecture 18 10/26/2022 Expectation Maximization        
Assignment Due 10/26/2022 Assignment 4 Due (11:59AM Central Time)    
Lecture 19 10/28/2022 Variational Auto-Encoders      
Lecture 20 11/02/2022 Generative Adversarial Nets      
Lecture 21 11/04/2022 Autoregressive Methods      
Lecture 22 11/09/2022 Energy-based Models      
Assignment Due 11/09/2022 Assignment 5 Due (11:59AM Central Time)    
Lecture 23 11/11/2022 Markov Decision Process      
Lecture 24 11/16/2022 Q Learning      
Lecture 25 11/18/2022 Policy Gradient      
11/23/2022 Fall Break    
11/25/2022 Fall Break    
Lecture 26 11/30/2022 Attention, Transformers      
Assignment Due 11/30/2022 Assignment 6 Due (11:59AM Central Time)    
Lecture 27 12/02/2022 Self-Supervised Learning      
Lecture 28 12/07/2022 Review      
Exam Final Exam