CS446/ECE449: Machine Learning (Fall 2024)

Course Information

The goal of Machine Learning is to find structure in data. In this course we will cover three main areas, (1) supervised learning, (2) unsupervised learning, and (3) reinforcement learning models. In particular we will cover the following: perceptron, decision trees, Naive Bayes, Gaussian Bayes, linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.

Pre-requisites: Probability, linear algebra, and proficiency in Python.

Recommended Text: (1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Machine Learning, Tom Mitchell, (3) Pattern Recognition and Machine Learning by Christopher Bishop, (4) The Elements of Statistical Learning: Data Mining, Inference, and Prediction by Jerome H. Friedman, Robert Tibshirani, and Trevor Hastie.

Course Deliverables:
(1) Homework due at noon, see below for dates
(2) Midterm
(3) Final

Grading:
3 credit: Homework 60% (drop 1 homework), Midterm 20%, Final 20%
4 credit: Homework 60% (drop 0 homework), Midterm 20%, Final 20%

Grading policy is subject to change.

Late Policy: 3 late days in total.


Instructor & TAs

Liangyan Gui

Instructor
Email: lgui[at]illinois.edu





Yen-Chi Cheng

Teaching Assistant
Email: yenchic3[at]illinois.edu

Ali Ebrahimpour Boroojeny

Teaching Assistant
Email: ae20[at]illinois.edu

Xiyan Xu

Teaching Assistant
Email: xiyanxu2[at]illinois.edu

Logistics

Class Time: Wednesday, Friday 12:30-1:45PM
Location: 1404 Siebel Center for Comp Sci.
Office Hours: 2102 Siebel Center for Comp Sci. Monday 10-11am, Tuesday 2-3pm, Wednesday 9:30-10:30am, Thursday 2-3pm, Friday 2-3pm.
Campuswire for discussions: [link] (code in class email)
GradeScope for assignments : [link] (code in class email)



Lectures

The syllabus is subject to change.

EventDateDescriptionSlides References
Lecture 1 08/28/2024 Introduction [Slides]  
Lecture 2 08/30/2024 kNN [Slides] Bishop: Sec 2.5; Murphy: Sec 1.4
Lecture 3 09/04/2024 Perceptron [Slides] The Perceptron
Assignment Due 09/04/2024 Assignment 0 Due (11:59AM Central Time)    
Assignment Released 09/04/2024 Assignment 1 [PDF]  
Lecture 4 09/06/2024 PyTorch Tutorial  
Lecture 5 09/11/2024 Probability and Estimation [Slides] Mitchell: Chapter 2, Goodfellow et al.: Chapter 3
Lecture 6 09/13/2024 Naive Bayes [Slides] Mitchell: Chapter 3
Lecture 7 09/18/2024 Gaussian Naive Bayes [Slides] Mitchell: Chapter 3
Assignment Due 09/18/2024 Assignment 1 Due (11:59AM Central Time)    
Assignment Due 09/19/2024 Assignment 2 Released    
Lecture 8 09/20/2024 Logistic Regression [Slides] Mitchell: Chapter 3
Lecture 9 09/25/2024 Optimization [Slides] Murphy: Sec 8.1,8.2,8.3
Lecture 10 09/27/2024 Linear Regression [Slides] Murphy: Sec 7, 8.3; Bishop: Sec 9.2
Lecture 11 10/02/2024 SVM [Slides] Murphy: Sec 14.5; Bishop: Sec 7.1
Assignment Due 10/03/2024 Assignment 2 Due (11:59AM Central Time)    
Lecture 12 10/04/2024 SVM II [Slides] Murphy: Sec 14.5; Bishop: Sec 7.1
Lecture 13 10/09/2024 Empirical Risk Minimization [Slides]  
Lecture 14 10/11/2024 Midterm Review [Slides]  
10/16/2024 Midterm Exam    
Lecture 15 10/18/2024 Bias-Variance Tradeoff [Slides]  
Assignment Due 10/18/2024 Assignment 3 Due (11:59AM Central Time)    
Lecture 16 10/23/2024 Model Selection [Slides]    
Lecture 17 10/25/2024 Kernels [Slides] Bishop: Sec 6.1, 6.2
Lecture 18 10/30/2024 Kernels II [Slides] Bishop: Sec 6.1, 6.2
Lecture 19 11/01/2024 Decision Tree Learning [Slides] Mitchell: 3; Bishop: Sec 14.4
Lecture 20 11/06/2024 Ensemble Methods, AdaBoost [Slides] Bishop: Sec 14.3, 14.4
Assignment Due 11/06/2024 Assignment 4 Due (11:59AM Central Time)    
Lecture 21 11/08/2024 Hierarchical Clustering, K-Means [Slides] Murphy, 21.3; Hastie et.al.: Sec 14.3.6, 14.3.7
Lecture 22 11/13/2024 PCA, SVD [Slides] Murphy, 12.2; Hastie et.al.: Sec 14.5.1, 14.5.2
Lecture 23 11/15/2024 Neural Networks [Slides] Goodfellow et al.: Chapter 6.1-6.4
Lecture 24 11/20/2024 Deep Learning [Slides] Goodfellow et al.: Chapter 6.1-6.5
Lecture 25 11/22/2024 Generative Modelling [Slides-CNNs][Slides-Generative Modelling] Goodfellow et al.: Chapter 6-9
Assignment Due 11/22/2024 Assignment 5 Due (11:59AM Central Time)    
11/27/2024 Fall Break    
11/29/2024 Fall Break    
Lecture 26 12/04/2024 Generative Modelling II [Slides]      
Assignment Due 12/04/2024 Assignment 6 Due (11:59AM Central Time)    
Lecture 27 12/06/2024 Sequential Models [Slides]      
Lecture 28 12/11/2024 Review [Slides]      
Exam Dec 19 Final Exam