CS 446 AGS: Machine Learning (Spring 2019)

Course Information

The goal of Machine Learning is to build computer systems that can adapt and learn from data. In this course we will cover three main areas, (1) discriminative models, (2) generative models, and (3) reinforcement learning models. In particular we will cover the following: linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.

Pre-requisites: Probability, linear algebra, and proficiency in Python.

Recommended Text: (1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (3) Pattern Recognition and Machine Learning by Christopher Bishop, (4) Graphical Models by Nir Friedman and Daphne Koller, and (5) Reinforcement Learning by Richard Sutton and Andrew Barto.

Grading:

3 credit: Homework 33%, Midterm 33%, Final 34%

4 credit: Homework+Piazza answers 25%, Scribe 8.3%, Midterm 33%, Final 34%

For homework, the lowest homework grade will be dropped (the scribe cannot be dropped) and we will compute the average score of the rest of the 9 assignments.
Grading policy is subject to change.

TA Hours:
Time: Wednesday 5:00pm to 6:00pm starting from January 23.
Room: ECEB 2015.

Late Policy: No late submission will be accepted after the due date.

Midterm: March 12, 18:00 - 19:20. Room: ECEB 1002. [Solution]
Final Exam: May 6, starting from 13:30. Room: ECEB 1002. The exam is two and a half hours long.

Instructor & TAs

Alexander Schwing

Instructor
Email: aschwing[at]illinois.edu
Office Hour: Tues. 7:30-8:30PM
Room: CSL 103
Website: [link]

Jyoti Aneja

Teaching Assistant
Email: janeja2[at]illinois.edu
Website: [link]

Yuan-Ting Hu

Teaching Assistant
Email: ythu2[at]illinois.edu
Website: [link]

Safa Messaoud

Teaching Assistant
Email: messaou2[at]illinois.edu
Website: [link]

Class Time & Location

Class Time: Tuesday, Thursday 18:00-19:15PM
Location: ECEB 1002 (map)

Course Discussions

Piazza for discussions: [link]
GradeScope for assignments (self-enrollment code M5J3WN): [link]

Homework & Scribe

Homework: [link]
Scribe: [link]



Lectures

The syllabus is subject to change.

EventDateDescriptionMaterialsScribes
Lecture 1 Jan. 15 Introduction (Nearest Neighbor) [Link]
Lecture 2 Jan. 17 Linear Regression [Link]
Lecture 3 Jan. 22 Logistic Regression [Link] [1] [2]
Lecture 4 Jan. 24 Optimization Primal [Link] [1] [2] [3]
Lecture 5 Jan. 29 Optimization Dual [Link] [1] [2]
Lecture 6 Jan. 31 Support Vector Machine [Link] [1]
Lecture 7 Feb. 5 Multiclass Classification and Kernel Methods [Link] [1] [2] [3] [4] [5]
Assignment Due Feb. 7 Assignment 1 Due (11:59AM Central Time)
Lecture 8 Feb. 7 Deep Nets 1 (Layers) [Link] [1] [2]
Lecture 9 Feb. 12 Deep Nets 2 (Backpropagation) [Link] [1] [2] [3]
Assignment Due Feb. 14 Assignment 2 Due (11:59AM Central Time)
Lecture 10 Feb. 14 Ensemble Methods (Boosting/Random Forest/Deep Nets) & Regularization/Cross-Val [Link]
Lecture 11 Feb. 19 Structured Prediction (exhaustive search, dynamic programming) [Link] [1]
Assignment Due Feb. 21 Assignment 3 Due (11:59AM Central Time)
Lecture 12 Feb. 21 Structured Prediction (ILP, LP relaxation, message passing, graph cut) [Link] [1] [2] [3]
Lecture 13 Feb. 26 Conditional Random Fields, Structured SVM, Deep Structured Methods [Link] [1] [2] [3]
Assignment Due Feb. 28 Assignment 4 Due (11:59AM Central Time)
Lecture 14 Feb. 28 Learning Theory [Link]
Lecture 15 Mar. 5 Learning Theory [Link] [1]
Assignment Due Mar. 7 Assignment 5 Due (11:59AM Central Time)
Review Mar. 7 Review Session
Exam Mar. 12 Midterm
Assignment Due Mar. 14 Assignment 6 Due (11:59AM Central Time)
Lecture 16 Mar. 14 PCA, SVD [Link] [1] [2] [3] [4]
No Lecture Mar. 19 Spring Break
No Lecture Mar. 21 Spring Break
Lecture 17 Mar. 26 k-Means [Link] [1] [2] [3] [4]
Lecture 18 Mar. 28 Gaussian Mixture Models [Link] [1]
Lecture 19 Apr. 2 Expectation Maximization/Majorize-Minimize/Concave-Convex Procedure [Link] [1]
Assignment Due Apr. 4 Assignment 7 Due (11:59AM Central Time)
Lecture 20 Apr. 4 Hidden Markov Models [Link] [1] [2] [3]
Lecture 21 Apr. 19 Variational Auto-Encoders [Link]
Lecture 22 Apr. 11 Generative Adversarial Nets [Link] [1]
Lecture 23 Apr. 16 Autoregressive Methods [Link]
Lecture 24 Apr. 18 MDP [Link] [1] [2] [3]
Assignment Due Apr. 18 Assignment 8 Due (11:59AM Central Time)
Lecture 25 Apr. 23 Q-Learning [Link]
Lecture 26 Apr. 25 Policy Gradient, Actor-Critic [Link] [1]
Assignment Due Apr. 25 Assignment 9 Due (11:59AM Central Time)
Review Apr. 30 Review Session
Assignment Due May. 6 Assignment 10 Due (11:59AM Central Time)
Exam May. 6 Final Exam