CS446/ECE449: Machine Learning (Fall 2024)
Course Information
The goal of Machine Learning is to find structure in data. In this course we will cover three main areas, (1) supervised learning, (2) unsupervised learning, and (3) reinforcement learning models. In particular we will cover the following: perceptron, decision trees, Naive Bayes, Gaussian Bayes, linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.Pre-requisites: Probability, linear algebra, and proficiency in Python.
Recommended Text: (1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Machine Learning, Tom Mitchell, (3) Pattern Recognition and Machine Learning by Christopher Bishop, (4) The Elements of Statistical Learning: Data Mining, Inference, and Prediction by Jerome H. Friedman, Robert Tibshirani, and Trevor Hastie.
Course Deliverables:
(1) Homework due at noon, see below for dates
(2) Midterm
(3) Final
Grading:
3 credit: Homework 60% (drop 1 homework), Midterm 20%, Final 20%
4 credit: Homework 60% (drop 0 homework), Midterm 20%, Final 20%
Grading policy is subject to change.
Late Policy: 3 late days in total.
Instructor & TAs
Yen-Chi Cheng
Teaching AssistantEmail: yenchic3[at]illinois.edu
Ali Ebrahimpour Boroojeny
Teaching AssistantEmail: ae20[at]illinois.edu
Xiyan Xu
Teaching AssistantEmail: xiyanxu2[at]illinois.edu
Logistics
Class Time: Wednesday, Friday 12:30-1:45PMLocation: 1404 Siebel Center for Comp Sci.
Office Hours: 2102 Siebel Center for Comp Sci. Monday 10-11am, Tuesday 2-3pm, Wednesday 9:30-10:30am, Thursday 2-3pm, Friday 2-3pm.
Campuswire for discussions: [link] (code in class email)
GradeScope for assignments : [link] (code in class email)
Lectures
The syllabus is subject to change.
Event | Date | Description | Slides | References | ||
---|---|---|---|---|---|---|
Lecture 1 | 08/28/2024 | Introduction | [Slides] | |||
Lecture 2 | 08/30/2024 | kNN | [Slides] | Bishop: Sec 2.5; Murphy: Sec 1.4 | ||
Lecture 3 | 09/04/2024 | Perceptron | [Slides] | The Perceptron | ||
Assignment Due | 09/04/2024 | Assignment 0 Due (11:59AM Central Time) | ||||
Assignment Released | 09/04/2024 | Assignment 1 | [PDF] | |||
Lecture 4 | 09/06/2024 | PyTorch Tutorial | ||||
Lecture 5 | 09/11/2024 | Probability and Estimation | [Slides] | Mitchell: Chapter 2, Goodfellow et al.: Chapter 3 | ||
Lecture 6 | 09/13/2024 | Naive Bayes | [Slides] | Mitchell: Chapter 3 | ||
Lecture 7 | 09/18/2024 | Gaussian Naive Bayes | [Slides] | Mitchell: Chapter 3 | ||
Assignment Due | 09/18/2024 | Assignment 1 Due (11:59AM Central Time) | ||||
Assignment Due | 09/19/2024 | Assignment 2 Released | ||||
Lecture 8 | 09/20/2024 | Logistic Regression | [Slides] | Mitchell: Chapter 3 | ||
Lecture 9 | 09/25/2024 | Optimization | [Slides] | Murphy: Sec 8.1,8.2,8.3 | ||
Lecture 10 | 09/27/2024 | Linear Regression | [Slides] | Murphy: Sec 7, 8.3; Bishop: Sec 9.2 | ||
Lecture 11 | 10/02/2024 | SVM | [Slides] | Murphy: Sec 14.5; Bishop: Sec 7.1 | ||
Assignment Due | 10/03/2024 | Assignment 2 Due (11:59AM Central Time) | ||||
Lecture 12 | 10/04/2024 | SVM II | [Slides] | Murphy: Sec 14.5; Bishop: Sec 7.1 | ||
Lecture 13 | 10/09/2024 | Empirical Risk Minimization | [Slides] | |||
Lecture 14 | 10/11/2024 | Midterm Review | [Slides] | |||
10/16/2024 | Midterm Exam | |||||
Lecture 15 | 10/18/2024 | Bias-Variance Tradeoff | [Slides] | |||
Assignment Due | 10/18/2024 | Assignment 3 Due (11:59AM Central Time) | ||||
Lecture 16 | 10/23/2024 | Model Selection | [Slides] | |||
Lecture 17 | 10/25/2024 | Kernels | [Slides] | Bishop: Sec 6.1, 6.2 | ||
Lecture 18 | 10/30/2024 | Kernels II | [Slides] | Bishop: Sec 6.1, 6.2 | ||
Lecture 19 | 11/01/2024 | Decision Tree Learning | [Slides] | Mitchell: 3; Bishop: Sec 14.4 | ||
Lecture 20 | 11/06/2024 | Ensemble Methods, AdaBoost | [Slides] | Bishop: Sec 14.3, 14.4 | ||
Assignment Due | 11/06/2024 | Assignment 4 Due (11:59AM Central Time) | ||||
Lecture 21 | 11/08/2024 | Hierarchical Clustering, K-Means | [Slides] | Murphy, 21.3; Hastie et.al.: Sec 14.3.6, 14.3.7 | ||
Lecture 22 | 11/13/2024 | PCA, SVD | [Slides] | Murphy, 12.2; Hastie et.al.: Sec 14.5.1, 14.5.2 | ||
Lecture 23 | 11/15/2024 | Neural Networks | [Slides] | Goodfellow et al.: Chapter 6.1-6.4 | ||
Lecture 24 | 11/20/2024 | Deep Learning | [Slides] | Goodfellow et al.: Chapter 6.1-6.5 | ||
Lecture 25 | 11/22/2024 | Generative Modelling | [Slides-CNNs][Slides-Generative Modelling] | Goodfellow et al.: Chapter 6-9 | ||
Assignment Due | 11/22/2024 | Assignment 5 Due (11:59AM Central Time) | ||||
11/27/2024 | Fall Break | |||||
11/29/2024 | Fall Break | |||||
Lecture 26 | 12/04/2024 | Generative Modelling II | [Slides] | |||
Assignment Due | 12/04/2024 | Assignment 6 Due (11:59AM Central Time) | ||||
Lecture 27 | 12/06/2024 | Sequential Models | [Slides] | |||
Lecture 28 | 12/11/2024 | Review | [Slides] | |||
Exam | Dec 19 | Final Exam |