CS446/ECE449: Machine Learning (Fall 2023)

Course Information

The goal of Machine Learning is to find structure in data. In this course we will cover three main areas, (1) supervised learning, (2) unsupervised learning, and (3) reinforcement learning models. In particular we will cover the following: perceptron, decision trees, Naive Bayes, Gaussian Bayes, linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.

Pre-requisites: Probability, linear algebra, and proficiency in Python.

Recommended Text: (1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Machine Learning, Tom Mitchell, (3) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (4) Pattern Recognition and Machine Learning by Christopher Bishop, (5) Graphical Models by Nir Friedman and Daphne Koller, and (6) Reinforcement Learning by Richard Sutton and Andrew Barto, (7) Understanding Machine Learning by Shai Shalev-Shwartz and Shai Ben-David.

Course Deliverables:
(1) Homework due at noon, see below for dates
(2) Midterm
(3) Final

Grading:
3 credit: Homework 60% (drop 1 homework), Midterm 20%, Final 20%
4 credit: Homework 60% (drop 0 homework), Midterm 20%, Final 20%

Grading policy is subject to change.

Late Policy: 3 late days in total.


Instructor & TAs

Liangyan Gui

Instructor
Email: lgui[at]illinois.edu

Shenlong Wang

Instructor
Email: shenlong[at]illinois.edu





Zhengyuan Li

Teaching Assistant
Email: zli138[at]illinois.edu

Hang Yu

Teaching Assistant
Email: hangy6[at]illinois.edu

Qiaobo Li

Teaching Assistant
Email: qiaobol2[at]illinois.edu

Chulin Xie

Teaching Assistant
Email: chulinx2[at]illinois.edu

Logistics

Class Time: Wednesday, Friday 12:30-1:45PM
Location: [Zoom link], 1404 Siebel Center for Comp Sci.
Office Hours: Monday 2-3pm, [Zoom link]; Tuesday 6-7pm, Siebel 4407; Wednesday 9-10am, [Zoom link]; Thursday 6-7pm, Siebel 4407; Friday 3-4pm, [Zoom link]
Campuswire for discussions: [link] (code in class email)
GradeScope for assignments : [link] (code in class email)



Lectures

The syllabus is subject to change.

EventDateDescriptionSlidesAssignments
Lecture 1 08/23/2023 Introduction, KNN [Slides]  
Lecture 2 08/25/2023 Perceptron [Slides]  
Lecture 3 08/30/2023 Probability and Estimation [Slides]  
Assignment Due 08/30/2023 Assignment 0 Due (11:59AM Central Time)    
Lecture 4 09/01/2023 Naive Bayes [Slides]  
Lecture 5 09/06/2023 Gaussian Naive Bayes [Slides]  
Lecture 6 09/08/2023 Logistic Regression [Slides]  
Lecture 7 09/13/2023 Optimization, Linear Regression [Slides]  
Assignment Due 09/13/2023 Assignment 1 Due (11:59AM Central Time)    
Lecture 8 09/15/2023 SVM [Slides]  
Lecture 9 09/20/2023 SVM [Slides]  
Lecture 10 09/22/2023 Kernel Methods [Slides]  
Lecture 11 09/27/2023 Theory [Slides]  
Assignment Due 09/28/2023 Assignment 2 Due (11:59AM Central Time)    
Lecture 12 09/29/2023 Decision Tree Learning, Ensemble Methods [Slides]  
Lecture 13 10/04/2023 AdaBoost, Deep Nets 1 [Slides]  
Lecture 14 10/06/2023 Deep Nets 2 & Midterm Review [Slides]  
10/11/2023 Midterm Exam    
Lecture 15 10/13/2023 PCA, SVD [Slides]  
Assignment Due 10/15/2023 Assignment 3 Due (11:59AM Central Time)    
Lecture 16 10/18/2023 K-Means [Slides]    
Lecture 17 10/20/2023 Gaussian Mixture Models [Slides]    
Lecture 18 10/25/2023 Variational Auto-Encoders [Slides]      
Lecture 19 10/27/2023 Generative Adversarial Nets [Slides]      
Lecture 20 11/01/2023 Diffusion Models [Slides]      
Assignment Due 11/03/2023 Assignment 4 Due (11:59AM Central Time)    
Lecture 21 11/03/2023 Markov Decision Process [Slides]      
Lecture 22 11/08/2023 Q Learning [Slides]      
No class 11/10/2023 No class      
Lecture 23 11/15/2023 Policy Gradient [Slides]      
Assignment Due 11/17/2023 Assignment 5 Due (11:59AM Central Time)    
Lecture 24 11/17/2023 Sequential Models [Slides]      
11/22/2023 Fall Break    
11/24/2023 Fall Break    
Lecture 25 11/29/2023 Attention, Transformers [Slides]      
Assignment Due 12/06/2023 Assignment 6 Due (11:59AM Central Time)    
Lecture 26 12/01/2023 Self-Supervised Learning, Foundation Models [Slides]      
Lecture 27 12/06/2023 Review [Slides]      
Exam 12/08/2023, 1:30pm Final Exam