CS 446 AGS: Machine Learning (Spring 2019)
Course Information
The goal of Machine Learning is to build computer systems that can adapt and learn from data. In this course we will cover three main areas, (1) discriminative models, (2) generative models, and (3) reinforcement learning models. In particular we will cover the following: linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.Pre-requisites: Probability, linear algebra, and proficiency in Python.
Recommended Text: (1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (3) Pattern Recognition and Machine Learning by Christopher Bishop, (4) Graphical Models by Nir Friedman and Daphne Koller, and (5) Reinforcement Learning by Richard Sutton and Andrew Barto.
Grading:
3 credit: Homework 33%, Midterm 33%, Final 34%
4 credit: Homework+Piazza answers 25%, Scribe 8.3%, Midterm 33%, Final 34%
For homework, the lowest homework grade will be dropped (the scribe cannot be dropped) and we will compute the average score of the rest of the 9 assignments.Grading policy is subject to change.
TA Hours:
Time: Wednesday 5:00pm to 6:00pm starting from January 23.
Room: ECEB 2015.
Late Policy: No late submission will be accepted after the due date.
Midterm: March 12, 18:00 - 19:20. Room: ECEB 1002. [Solution]
Final Exam: May 6, starting from 13:30. Room: ECEB 1002. The exam is two and a half hours long.
Instructor & TAs
Alexander Schwing
InstructorEmail: aschwing[at]illinois.edu
Office Hour: Tues. 7:30-8:30PM
Room: CSL 103
Website: [link]
Lectures
The syllabus is subject to change.
Event | Date | Description | Materials | Scribes |
---|---|---|---|---|
Lecture 1 | Jan. 15 | Introduction (Nearest Neighbor) | [Link] | |
Lecture 2 | Jan. 17 | Linear Regression | [Link] | |
Lecture 3 | Jan. 22 | Logistic Regression | [Link] | [1] [2] |
Lecture 4 | Jan. 24 | Optimization Primal | [Link] | [1] [2] [3] |
Lecture 5 | Jan. 29 | Optimization Dual | [Link] | [1] [2] |
Lecture 6 | Jan. 31 | Support Vector Machine | [Link] | [1] |
Lecture 7 | Feb. 5 | Multiclass Classification and Kernel Methods | [Link] | [1] [2] [3] [4] [5] |
Assignment Due | Feb. 7 | Assignment 1 Due (11:59AM Central Time) | ||
Lecture 8 | Feb. 7 | Deep Nets 1 (Layers) | [Link] | [1] [2] |
Lecture 9 | Feb. 12 | Deep Nets 2 (Backpropagation) | [Link] | [1] [2] [3] |
Assignment Due | Feb. 14 | Assignment 2 Due (11:59AM Central Time) | ||
Lecture 10 | Feb. 14 | Ensemble Methods (Boosting/Random Forest/Deep Nets) & Regularization/Cross-Val | [Link] | |
Lecture 11 | Feb. 19 | Structured Prediction (exhaustive search, dynamic programming) | [Link] | [1] |
Assignment Due | Feb. 21 | Assignment 3 Due (11:59AM Central Time) | ||
Lecture 12 | Feb. 21 | Structured Prediction (ILP, LP relaxation, message passing, graph cut) | [Link] | [1] [2] [3] |
Lecture 13 | Feb. 26 | Conditional Random Fields, Structured SVM, Deep Structured Methods | [Link] | [1] [2] [3] |
Assignment Due | Feb. 28 | Assignment 4 Due (11:59AM Central Time) | ||
Lecture 14 | Feb. 28 | Learning Theory | [Link] | |
Lecture 15 | Mar. 5 | Learning Theory | [Link] | [1] |
Assignment Due | Mar. 7 | Assignment 5 Due (11:59AM Central Time) | ||
Review | Mar. 7 | Review Session | ||
Exam | Mar. 12 | Midterm | ||
Assignment Due | Mar. 14 | Assignment 6 Due (11:59AM Central Time) | ||
Lecture 16 | Mar. 14 | PCA, SVD | [Link] | [1] [2] [3] [4] |
No Lecture | Mar. 19 | Spring Break | ||
No Lecture | Mar. 21 | Spring Break | ||
Lecture 17 | Mar. 26 | k-Means | [Link] | [1] [2] [3] [4] |
Lecture 18 | Mar. 28 | Gaussian Mixture Models | [Link] | [1] |
Lecture 19 | Apr. 2 | Expectation Maximization/Majorize-Minimize/Concave-Convex Procedure | [Link] | [1] |
Assignment Due | Apr. 4 | Assignment 7 Due (11:59AM Central Time) | ||
Lecture 20 | Apr. 4 | Hidden Markov Models | [Link] | [1] [2] [3] |
Lecture 21 | Apr. 19 | Variational Auto-Encoders | [Link] | |
Lecture 22 | Apr. 11 | Generative Adversarial Nets | [Link] | [1] |
Lecture 23 | Apr. 16 | Autoregressive Methods | [Link] | |
Lecture 24 | Apr. 18 | MDP | [Link] | [1] [2] [3] |
Assignment Due | Apr. 18 | Assignment 8 Due (11:59AM Central Time) | ||
Lecture 25 | Apr. 23 | Q-Learning | [Link] | |
Lecture 26 | Apr. 25 | Policy Gradient, Actor-Critic | [Link] | [1] |
Assignment Due | Apr. 25 | Assignment 9 Due (11:59AM Central Time) | ||
Review | Apr. 30 | Review Session | ||
Assignment Due | May. 6 | Assignment 10 Due (11:59AM Central Time) | ||
Exam | May. 6 | Final Exam |