CS446/ECE449: Machine Learning (Spring 2024)

Course Information

The goal of machine learning is to develop algorithms and models that enable computers to learn from data and make predictions or decisions without being explicitly programmed for a particular task. In this course we will cover three main areas, (1) supervised learning, (2) unsupervised learning, and (3) reinforcement learning. In particular we will cover the following: perceptron, decision trees, Naive Bayes, Gaussian Bayes, linear regression, logistic regression, support vector machines, learning theory, deep learning, structured methods, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.

Pre-requisites: Probability, linear algebra, and proficiency in Python.

Recommended Text:
(1) Probabilistic Machine Learning: An Introduction [KM]
(2) Deep Learning [GBC]
(3) Understanding Machine Learning: From Theory to Algorithms [SB]
(4) The Elements of Statistical Learning: Data Mining, Inference, and Prediction [HTF]
(5) Deep Learning: Foundations and Concepts [BB]

Course Deliverables:
(1) Homework due at 23:59pm, see below for dates
(2) Midterm
(3) Final

Grading:
3 credit: Homework 60% (drop 1 homework), Midterm 20%, Final 20%
4 credit: Homework 60% (drop 0 homework), Midterm 20%, Final 20%

Grading policy is subject to change.

Late Policy: Each student will have a 3-day grace period in total for the homework assignments. You may use this grace period for any homework assignment throughout the semester. After the grace period, late homework will be penalized 20% per day. Students with legitimate reasons (special accommodations, see below) who contact the professor before the deadline may apply for an extension.


Instructor & TAs

Han Zhao

Instructor
Email: hanzhao[at]illinois.edu

Shenlong Wang

Instructor
Email: shenlong[at]illinois.edu





Yubin Ge

Teaching Assistant
Email: yubinge2[at]illinois.edu

Amnon Attali

Teaching Assistant
Email: aattali2[at]illinois.edu

Jing Wen

Teaching Assistant
Email: jw116[at]illinois.edu

Jane Du

Teaching Assistant
Email: zd16[at]illinois.edu

Logistics

Class Time: Tuesday, Thursday 12:30pm-1:45pm
Location: 1320 DCL.
Syllabus: [link]
Campuswire for discussions: [link] (code in class email)
GradeScope for assignments : [link] (code in class email)


Lectures

EventDateDescriptionSlidesInstructorReadings
Lecture 1 01/16/2024 Introduction, KNN [Slides] Han Zhao
Lecture 2 01/18/2024 Naive Bayes [Slides] Han Zhao [KM] Chapter 9.3; [BB] Chapter 11.2.4
Lecture 3 01/23/2024 Linear Regression [Slides] Shenlong Wang [KM] Chapter 11.1, 11.2
Lecture 4 01/25/2024 Logistic Regression [Slides] Shenlong Wang [KM] Chapter 10.1, 10.2
Lecture 5 01/30/2024 Support Vector Machine [Slides] Han Zhao [HTF] Chapter 12.2; [SB] Chapter 15.1-15.2
Lecture 6 02/01/2024 Kernel Methods [Slides] Han Zhao [SB] Chapter 16.1-16.2; [KM] Chapter 17.3.1-17.3.4
Lecture 7 02/06/2024 Decision Trees, Random Forests [Slides] Shenlong Wang [SB] [KM] Chapter 18.1
Assignment Due 02/06/2024 Assignment 1 Due (23:59 Central Time)    
Lecture 8 02/08/2024 Boosting [Slides] Shenlong Wang [SB] [KM] Chapter 18.3, 18.4, 18.5
Lecture 9 02/13/2024 PAC Learning Theory (I) [Slides] Han Zhao [SB] Chapter 2.1-2.2
Lecture 10 02/15/2024 PAC Learning Theory (II) [Slides] Han Zhao [SB] Chapter 2.3, Chapter 3, 4
Lecture 11 02/20/2024 Perceptron Algorithm [Slides] Shenlong Wang
Assignment Due 02/20/2024 Assignment 2 Due (23:59 Central Time)    
Lecture 12 02/22/2024 Deep Learning: MLPs, Backpropagation [Slides] Shenlong Wang
Lecture 13 02/27/2024 Deep Learning: Convolutional Neural Networks [Slides] Shenlong Wang
Lecture 14 02/29/2024 Deep Learning: Sequential Models, RNNs, LSTMs [Slides] Shenlong Wang
Lecture 15 03/05/2024 Deep Learning: Attention, Transformers [Slides] Shenlong Wang
Mid-term Exam 03/07/2024 Mid-term Exam    
Assignment Due 03/10/2024 Assignment 3 Due (23:59 Central Time)    
No Class 03/12/2024 Spring break    
No Class 03/14/2024 Spring break    
Lecture 16 03/19/2024 Principal Component Analysis / SVD [Slides] Shenlong Wang
Lecture 17 03/21/2024 K-means [Slides] Shenlong Wang
Lecture 18 03/26/2024 Gaussian Mixture Models [Slides] Shenlong Wang
Lecture 19 03/28/2024 Information Theory 101 [Slides] Han Zhao [KM] Chapter 6
Lecture 20 04/02/2024 Auto-encoders / Variational Autoencoders [Slides] Shenlong Wang
Assignment Due 04/02/2024 Assignment 4 Due (23:59 Central Time)    
Lecture 21 04/04/2024 Generative Adversarial Networks [Slides] Han Zhao
Lecture 22 04/09/2024 Diffusion Models / Score Matching [Slides] Han Zhao
Lecture 23 04/11/2024 Self-supervised Learning / Contrastive Learning [Slides] Han Zhao
Assignment Due 04/11/2024 Assignment 5 Due (23:59 Central Time)    
Lecture 24 04/16/2024 Language Modeling / Foundation Models [Slides] Han Zhao
Lecture 25 04/18/2024 Markov Decision Process [Slides] Han Zhao
Lecture 26 04/23/2024 Q-Learning [Slides] Han Zhao
Lecture 27 04/25/2024 Policy Gradient [Slides] Han Zhao
Lecture 28 04/30/2024 Course Review [Slides] Han Zhao
Assignment Due 04/30/2024 Assignment 6 Due (23:59 Central Time)    
Exam TBD Final Exam