# CS446/ECE449: Machine Learning (Fall 2022)

## Course Information

The goal of Machine Learning is to find structure in data. In this course we will cover three main areas, (1) discriminative models, (2) generative models, and (3) reinforcement learning models. In particular we will cover the following: decision trees, Naive Bayes, Gaussian Bayes, linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.**Pre-requisites:**Probability, linear algebra, and proficiency in Python.

**Recommended Text:**(1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Machine Learning, Tom Mitchell, (3) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (4) Pattern Recognition and Machine Learning by Christopher Bishop, (5) Graphical Models by Nir Friedman and Daphne Koller, and (6) Reinforcement Learning by Richard Sutton and Andrew Barto, (7) Understanding Machine Learning by Shai Shalev-Shwartz and Shai Ben-David.

**Course Deliverables:**

(1) Homework due at noon, see below for dates

(2) Midterm

(3) Final

**Grading:**

3 credit: Homework 60% (drop 1 homework), Midterm 20%, Final 20%

4 credit: Homework 60% (drop 0 homework), Midterm 20%, Final 20%

Grading policy is subject to change.

**Office Hours: [Zoom link],**

Monday 3-4pm; Tuesday 8-9pm; Wednesday 9-10am; Thursday 6-7pm; Friday 2-3pm.

**Late Policy:**3 late days in total.

## Instructor & TAs

### Liangyan Gui

**Instructor**

Email: lgui[at]illinois.edu

### Shenlong Wang

**Instructor**

Email: shenlong[at]illinois.edu

### Junkun Chen

**Teaching Assistant**

Email: junkun3[at]illinois.edu

### Sirui Xu

**Teaching Assistant**

Email: siruixu2[at]illinois.edu

### Vlas Zyrianov

**Teaching Assistant**

Email: vlasz2[at]illinois.edu

### Min Jin Chong

**Teaching Assistant**

Email: mchong6[at]illinois.edu

## Logistics

**Class Time**: Wednesday, Friday 12:30-1:45PM

**Location**: [Zoom link].

**Campuswire for discussions**: [link] (code in class email)

**GradeScope for assignments**: [link] (code in class email)

## Lectures

The syllabus is subject to change.

Event | Date | Description | Slides | Assignments | ||
---|---|---|---|---|---|---|

Lecture 1 | 08/24/2022 | Introduction | [Slides] | |||

Lecture 2 | 08/26/2022 | Decision Tree Learning | [Slides] | |||

Lecture 3 | 08/31/2022 | Probability and Estimation | [Slides] | |||

Assignment Due | 08/31/2022 | Assignment 0 Due (11:59AM Central Time) | ||||

Lecture 4 | 09/02/2022 | Naive Bayes | [Slides] | |||

Lecture 5 | 09/07/2022 | Gaussian Naive Bayes | [Slides] | |||

Lecture 6 | 09/09/2022 | Logistic Regression | [Slides] | |||

Lecture 7 | 09/14/2022 | Linear Regression | [Slides] | |||

Assignment Due | 09/14/2022 | Assignment 1 Due (11:59AM Central Time) | ||||

Lecture 8 | 09/16/2022 | Learning theory | [Slides] | |||

Lecture 9 | 09/21/2022 | Kernel methods and SVM | [Slides] | |||

Lecture 10 | 09/23/2022 | SVM | [Slides] | |||

Lecture 11 | 09/28/2022 | SVM II, Ensemble Methods | [Slides] | |||

Assignment Due | 09/28/2022 | Assignment 2 Due (11:59AM Central Time) | ||||

Lecture 12 | 09/30/2022 | PyTorch Tutorial | ||||

Lecture 13 | 10/05/2022 | Deep Nets 1 | [Slides] | |||

Lecture 14 | 10/07/2022 | Deep Nets 2 & Midterm Review | [Slides][Midterm Review] | |||

10/12/2022 | Midterm Exam | |||||

Lecture 15 | 10/14/2022 | PCA, SVD | [Slides] | |||

Assignment Due | 10/16/2022 | Assignment 3 Due (11:59AM Central Time) | ||||

Lecture 16 | 10/19/2022 | K-Means | [Slides] | |||

Lecture 17 | 10/21/2022 | Gaussian Mixture Models | [Slides] | |||

Lecture 18 | 10/26/2022 | Expectation Maximization | [Slides] | |||

Lecture 19 | 10/28/2022 | Variational Auto-Encoders | [Slides] | |||

Lecture 20 | 11/02/2022 | Generative Adversarial Nets | [Slides] | |||

Assignment Due | 11/02/2022 | Assignment 4 Due (11:59AM Central Time) | ||||

Lecture 21 | 11/04/2022 | Autoregressive Methods | [Slides] | |||

Lecture 22 | 11/09/2022 | Energy-based Models | [Slides] | |||

Lecture 23 | 11/11/2022 | Markov Decision Process | [Slides] | |||

Lecture 24 | 11/16/2022 | Q Learning | [Slides] | |||

Assignment Due | 11/16/2022 | Assignment 5 Due (11:59AM Central Time) | ||||

Lecture 25 | 11/18/2022 | Policy Gradient | [Slides] | |||

11/23/2022 | Fall Break | |||||

11/25/2022 | Fall Break | |||||

Lecture 26 | 11/30/2022 | Attention, Transformers | [Slides] | |||

Assignment Due | 11/30/2022 | Assignment 6 Due (11:59AM Central Time) | ||||

Lecture 27 | 12/02/2022 | Self-Supervised Learning | [Slides] | |||

Lecture 28 | 12/07/2022 | Review | [Slides] | |||

Exam | 12/16/2022 | Final Exam |