# CS446/ECE449: Machine Learning (Fall 2023)

## Course Information

The goal of Machine Learning is to find structure in data. In this course we will cover three main areas, (1) supervised learning, (2) unsupervised learning, and (3) reinforcement learning models. In particular we will cover the following: perceptron, decision trees, Naive Bayes, Gaussian Bayes, linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.**Pre-requisites:**Probability, linear algebra, and proficiency in Python.

**Recommended Text:**(1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Machine Learning, Tom Mitchell, (3) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (4) Pattern Recognition and Machine Learning by Christopher Bishop, (5) Graphical Models by Nir Friedman and Daphne Koller, and (6) Reinforcement Learning by Richard Sutton and Andrew Barto, (7) Understanding Machine Learning by Shai Shalev-Shwartz and Shai Ben-David.

**Course Deliverables:**

(1) Homework due at noon, see below for dates

(2) Midterm

(3) Final

**Grading:**

3 credit: Homework 60% (drop 1 homework), Midterm 20%, Final 20%

4 credit: Homework 60% (drop 0 homework), Midterm 20%, Final 20%

Grading policy is subject to change.

**Late Policy:**3 late days in total.

## Instructor & TAs

### Liangyan Gui

**Instructor**

Email: lgui[at]illinois.edu

### Shenlong Wang

**Instructor**

Email: shenlong[at]illinois.edu

### Zhengyuan Li

**Teaching Assistant**

Email: zli138[at]illinois.edu

### Hang Yu

**Teaching Assistant**

Email: hangy6[at]illinois.edu

### Qiaobo Li

**Teaching Assistant**

Email: qiaobol2[at]illinois.edu

### Chulin Xie

**Teaching Assistant**

Email: chulinx2[at]illinois.edu

## Logistics

**Class Time**: Wednesday, Friday 12:30-1:45PM

**Location**: [Zoom link], 1404 Siebel Center for Comp Sci.

**Office Hours**: Monday 2-3pm, [Zoom link]; Tuesday 6-7pm, Siebel 4407; Wednesday 9-10am, [Zoom link]; Thursday 6-7pm, Siebel 4407; Friday 3-4pm, [Zoom link]

**Campuswire for discussions**: [link] (code in class email)

**GradeScope for assignments**: [link] (code in class email)

## Lectures

The syllabus is subject to change.

Event | Date | Description | Slides | Assignments | ||
---|---|---|---|---|---|---|

Lecture 1 | 08/23/2023 | Introduction, KNN | [Slides] | |||

Lecture 2 | 08/25/2023 | Perceptron | [Slides] | |||

Lecture 3 | 08/30/2023 | Probability and Estimation | [Slides] | |||

Assignment Due | 08/30/2023 | Assignment 0 Due (11:59AM Central Time) | ||||

Lecture 4 | 09/01/2023 | Naive Bayes | [Slides] | |||

Lecture 5 | 09/06/2023 | Gaussian Naive Bayes | [Slides] | |||

Lecture 6 | 09/08/2023 | Logistic Regression | [Slides] | |||

Lecture 7 | 09/13/2023 | Optimization, Linear Regression | [Slides] | |||

Assignment Due | 09/13/2023 | Assignment 1 Due (11:59AM Central Time) | ||||

Lecture 8 | 09/15/2023 | SVM | [Slides] | |||

Lecture 9 | 09/20/2023 | SVM | [Slides] | |||

Lecture 10 | 09/22/2023 | Kernel Methods | [Slides] | |||

Lecture 11 | 09/27/2023 | Theory | [Slides] | |||

Assignment Due | 09/28/2023 | Assignment 2 Due (11:59AM Central Time) | ||||

Lecture 12 | 09/29/2023 | Decision Tree Learning, Ensemble Methods | [Slides] | |||

Lecture 13 | 10/04/2023 | Deep Nets 1 | ||||

Lecture 14 | 10/06/2023 | Deep Nets 2 & Midterm Review | ||||

10/11/2023 | Midterm Exam | |||||

Lecture 15 | 10/13/2023 | PCA, SVD | ||||

Assignment Due | 10/15/2023 | Assignment 3 Due (11:59AM Central Time) | ||||

Lecture 16 | 10/18/2023 | K-Means | ||||

Lecture 17 | 10/20/2023 | Gaussian Mixture Models | ||||

Lecture 18 | 10/25/2023 | Expectation Maximization | ||||

Lecture 19 | 10/27/2023 | Variational Auto-Encoders | ||||

Lecture 20 | 11/01/2023 | Generative Adversarial Nets | ||||

Assignment Due | 11/01/2023 | Assignment 4 Due (11:59AM Central Time) | ||||

Lecture 21 | 11/03/2023 | Autoregressive Methods | ||||

Lecture 22 | 11/08/2023 | Diffusion Models | ||||

Lecture 23 | 11/10/2023 | Markov Decision Process | ||||

Lecture 24 | 11/15/2023 | Q Learning | ||||

Assignment Due | 11/15/2023 | Assignment 5 Due (11:59AM Central Time) | ||||

Lecture 25 | 11/17/2023 | Policy Gradient | ||||

11/22/2023 | Fall Break | |||||

11/24/2023 | Fall Break | |||||

Lecture 26 | 11/29/2023 | Attention, Transformers | ||||

Assignment Due | 11/29/2023 | Assignment 6 Due (11:59AM Central Time) | ||||

Lecture 27 | 12/01/2023 | Self-Supervised Learning | ||||

Lecture 28 | 12/06/2023 | Review | ||||

Exam | TBD | Final Exam |