# CS446/ECE449: Machine Learning (Spring 2021)

## Course Information

The goal of Machine Learning is to find structure in data. In this course we will cover three main areas, (1) discriminative models, (2) generative models, and (3) reinforcement learning models. In particular we will cover the following: linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.**Pre-requisites:**Probability, linear algebra, and proficiency in Python.

**Recommended Text:**(1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (3) Pattern Recognition and Machine Learning by Christopher Bishop, (4) Graphical Models by Nir Friedman and Daphne Koller, (5) Reinforcement Learning by Richard Sutton and Andrew Barto, (6) Understanding Machine Learning by Shai Shalev-Shwartz and Shai Ben-David

**Course Deliverables:**

(1) Homework

**due at noon, see below for dates (no late submission accepted)**

(2) Midterm

(3) Final

**Grading:**

3 credit: 60% homework (drop 1 homework), 20% midterm, 20% final

4 credit: 60% homework (drop 0 homework), 20% midterm, 20% final

Grading policy is subject to change.

**TA Hours:**

Time: Monday 5pm, Monday 8pm, Wednesday 4:30pm, Friday 10am, Friday 11am on campuswire.

**Final Exam:**TBD.

## Instructor & TAs

### Matus Telgarsky

**Instructor**

Email: mjt[at]illinois.edu

Office Hour: Tuesday 4:45-5:30pm (after class)

Website: [link]

### Alexander Schwing

**Instructor**

Email: aschwing[at]illinois.edu

Office Hour: Tuesday 4:45-5:30pm (after class)

Website: [link]

### Jing Liu

**Senior Teaching Assistant**

Email: jil292[at]illinois.edu

Office Hour: Wednesday from 4:30pm-5:30pm

Website: [link]

### Priyank Agrawal

**Teaching Assistant**

Email: priyank4[at]illinois.edu

Office Hour: Friday from 11am-noon

Website: [link]

### Ansel Blume

**Teaching Assistant**

Email: blume5[at]illinois.edu

Office Hour: Monday from 8pm-9pm

Website: [link]

### Safa Messaoud

**Teaching Assistant**

Email: messaou2[at]illinois.edu

Office Hour: Friday from 10am-11am

Website: [link]

### Efthymios Tzinis

**Teaching Assistant**

Email: etzinis2@illinois.edu

Office Hour: Monday from 1pm-2pm

Website: [link]

### Xiaoming Zhao

**Teaching Assistant**

Email: xz23[at]illinois.edu

Office Hour: Monday from 5pm-6pm

Website: [link]

### Class Time & Location

Class Time: Tuesday, Thursday 3:30PM-4:45PM (online; see campuswire for link)## Lectures

The syllabus is subject to change.

Event | Date | Description | Slides | Recording | Material |
---|---|---|---|---|---|

Lecture 1 | 01/26/2021 | Overview; start of linear regression | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 2 | 01/28/2021 | Linear Regression | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 3 | 02/02/2021 | Logistic Regression | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

HW | 02/02/2021 | Homework 0 (math & pytorch) (due at noon) | [HW] | ||

Lecture 4 | 02/04/2021 | Linear prediction: features, overfitting, and losses | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 5 | 02/09/2021 | Convex optimization | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 6 | 02/11/2021 | Support Vector Machines 1 | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 7 | 02/16/2021 | Support Vector Machines 2 | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

HW | 02/16/2021 | Homework 1 (due at noon) | [HW] | ||

Lecture 8 | 02/18/2021 | Deep Nets 1 | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 9 | 02/23/2021 | pytorch tutorial | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 10 | 02/25/2021 | Deep Nets 2 | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 11 | 03/02/2021 | Nearest Neighbors and decision trees | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

HW | 03/02/2021 | Homework 2 (due at noon) | [HW] | ||

Lecture 12 | 03/04/2021 | Ensemble methods | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 13 | 03/09/2021 | Learning theory | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 14 | 03/11/2021 | Review | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

HW | 03/11/2021 | Homework 3 (due at noon) | [HW] | ||

Midterm | 03/16/2021 | Midterm | |||

Lecture 15 | 03/18/2021 | PCA | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 16 | 03/23/2021 | k-Means | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 17 | 03/25/2021 | Gaussian Mixture Models | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 18 | 03/30/2021 | Expectation Maximization | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 19 | 04/01/2021 | Variational Auto-Encoders | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 20 | 04/06/2021 | Generative Adversarial Nets | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

HW | 04/06/2021 | Homework 4 (due at noon) | [HW] | ||

Lecture 21 | 04/08/2021 | Autoregressive Methods | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Break | 04/13/2021 | Break | |||

HW | 04/15/2021 | Homework 5 (due at noon) | [HW] | ||

Lecture 22 | 04/15/2021 | MDP | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 23 | 04/20/2021 | Q-Learning | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 24 | 04/22/2021 | Actor-Critic & Policy Gradient | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 25 | 04/27/2021 | Graph Neural Nets | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

HW | 04/27/2021 | Homework 6 (due at noon) | [HW] | ||

Lecture 26 | 04/29/2021 | Transformers | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Lecture 27 | 05/04/2021 | Review | [Slides] [Slides Split] [Slides Annot] | [Rec] | |

Final | TBD | Final |