# CS446/ECE449: Machine Learning (Fall 2023)

## Course Information

The goal of Machine Learning is to find structure in data. In this course we will cover three main areas, (1) supervised learning, (2) unsupervised learning, and (3) reinforcement learning models. In particular we will cover the following: perceptron, decision trees, Naive Bayes, Gaussian Bayes, linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.**Pre-requisites:**Probability, linear algebra, and proficiency in Python.

**Recommended Text:**(1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Machine Learning, Tom Mitchell, (3) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (4) Pattern Recognition and Machine Learning by Christopher Bishop, (5) Graphical Models by Nir Friedman and Daphne Koller, and (6) Reinforcement Learning by Richard Sutton and Andrew Barto, (7) Understanding Machine Learning by Shai Shalev-Shwartz and Shai Ben-David.

**Course Deliverables:**

(1) Homework due at noon, see below for dates

(2) Midterm

(3) Final

**Grading:**

3 credit: Homework 60% (drop 1 homework), Midterm 20%, Final 20%

4 credit: Homework 60% (drop 0 homework), Midterm 20%, Final 20%

Grading policy is subject to change.

**Late Policy:**3 late days in total.

## Instructor & TAs

### Liangyan Gui

**Instructor**

Email: lgui[at]illinois.edu

### Shenlong Wang

**Instructor**

Email: shenlong[at]illinois.edu

### Zhengyuan Li

**Teaching Assistant**

Email: zli138[at]illinois.edu

### Hang Yu

**Teaching Assistant**

Email: hangy6[at]illinois.edu

### Qiaobo Li

**Teaching Assistant**

Email: qiaobol2[at]illinois.edu

### Chulin Xie

**Teaching Assistant**

Email: chulinx2[at]illinois.edu

## Logistics

**Class Time**: Wednesday, Friday 12:30-1:45PM

**Location**: [Zoom link], 1404 Siebel Center for Comp Sci.

**Office Hours**: Monday 2-3pm, [Zoom link]; Tuesday 6-7pm, Siebel 4407; Wednesday 9-10am, [Zoom link]; Thursday 6-7pm, Siebel 4407; Friday 3-4pm, [Zoom link]

**Campuswire for discussions**: [link] (code in class email)

**GradeScope for assignments**: [link] (code in class email)

## Lectures

The syllabus is subject to change.

Event | Date | Description | Slides | Assignments | ||
---|---|---|---|---|---|---|

Lecture 1 | 08/23/2023 | Introduction, KNN | [Slides] | |||

Lecture 2 | 08/25/2023 | Perceptron | [Slides] | |||

Lecture 3 | 08/30/2023 | Probability and Estimation | [Slides] | |||

Assignment Due | 08/30/2023 | Assignment 0 Due (11:59AM Central Time) | ||||

Lecture 4 | 09/01/2023 | Naive Bayes | [Slides] | |||

Lecture 5 | 09/06/2023 | Gaussian Naive Bayes | [Slides] | |||

Lecture 6 | 09/08/2023 | Logistic Regression | [Slides] | |||

Lecture 7 | 09/13/2023 | Optimization, Linear Regression | [Slides] | |||

Assignment Due | 09/13/2023 | Assignment 1 Due (11:59AM Central Time) | ||||

Lecture 8 | 09/15/2023 | SVM | [Slides] | |||

Lecture 9 | 09/20/2023 | SVM | [Slides] | |||

Lecture 10 | 09/22/2023 | Kernel Methods | [Slides] | |||

Lecture 11 | 09/27/2023 | Theory | [Slides] | |||

Assignment Due | 09/28/2023 | Assignment 2 Due (11:59AM Central Time) | ||||

Lecture 12 | 09/29/2023 | Decision Tree Learning, Ensemble Methods | [Slides] | |||

Lecture 13 | 10/04/2023 | AdaBoost, Deep Nets 1 | [Slides] | |||

Lecture 14 | 10/06/2023 | Deep Nets 2 & Midterm Review | [Slides] | |||

10/11/2023 | Midterm Exam | |||||

Lecture 15 | 10/13/2023 | PCA, SVD | [Slides] | |||

Assignment Due | 10/15/2023 | Assignment 3 Due (11:59AM Central Time) | ||||

Lecture 16 | 10/18/2023 | K-Means | [Slides] | |||

Lecture 17 | 10/20/2023 | Gaussian Mixture Models | [Slides] | |||

Lecture 18 | 10/25/2023 | Variational Auto-Encoders | [Slides] | |||

Lecture 19 | 10/27/2023 | Generative Adversarial Nets | [Slides] | |||

Lecture 20 | 11/01/2023 | Diffusion Models | [Slides] | |||

Assignment Due | 11/03/2023 | Assignment 4 Due (11:59AM Central Time) | ||||

Lecture 21 | 11/03/2023 | Markov Decision Process | [Slides] | |||

Lecture 22 | 11/08/2023 | Q Learning | [Slides] | |||

No class | 11/10/2023 | No class | ||||

Lecture 23 | 11/15/2023 | Policy Gradient | [Slides] | |||

Assignment Due | 11/17/2023 | Assignment 5 Due (11:59AM Central Time) | ||||

Lecture 24 | 11/17/2023 | Sequential Models | [Slides] | |||

11/22/2023 | Fall Break | |||||

11/24/2023 | Fall Break | |||||

Lecture 25 | 11/29/2023 | Attention, Transformers | [Slides] | |||

Assignment Due | 12/06/2023 | Assignment 6 Due (11:59AM Central Time) | ||||

Lecture 26 | 12/01/2023 | Self-Supervised Learning, Foundation Models | [Slides] | |||

Lecture 27 | 12/06/2023 | Review | [Slides] | |||

Exam | 12/08/2023, 1:30pm | Final Exam |