ECE544: Pattern Recognition (Fall 2021)
Course Information
The goal of Pattern Recognition is to find structure in data. In this course we will cover three main areas, (1) discriminative models, (2) generative models, and (3) reinforcement learning models. In particular we will cover the following: linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.Pre-requisites: Probability, linear algebra, and proficiency in Python.
Recommended Text: (1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (3) Pattern Recognition and Machine Learning by Christopher Bishop, (4) Graphical Models by Nir Friedman and Daphne Koller, and (5) Reinforcement Learning by Richard Sutton and Andrew Barto.
Course Deliverables:
(1) Final Exam
(2) Presentations (multiple)
(3) Scribe
To access your deliverable schedule please browse to this file.
Grading:
1/3 class presentations, 1/3 scribe, 1/3 final;
Grading policy is subject to change.
Final Exam: December 7, 11am-12:30pm.
Instructor & TAs
Alexander Schwing
InstructorEmail: aschwing[at]illinois.edu
Office Hour: Friday at 6pm
Website: [link]
Class Time & Location
Class Time: Tuesday, Thursday 11:00AM-12:15PMLocation: Zoom at first (see class email or Canvas for link)
Lectures
The syllabus is subject to change.
Event | Date | Description | Materials | Pre-Recording | Recording |
---|---|---|---|---|---|
Lecture 1 | 08/24/2021 | Introduction | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 2 | 08/26/2021 | Nearest Neighbor | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 3 | 08/31/2021 | Linear Regression | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 4 | 09/02/2021 | Logistic Regression | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 5 | 09/07/2021 | Optimization Primal | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 6 | 09/09/2021 | Optimization Dual | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 7 | 09/14/2021 | Support Vector Machine | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 8 | 09/16/2021 | Multiclass Classification and Kernel Methods | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 9 | 09/21/2021 | Deep Nets 1 (Layers) | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 10 | 09/23/2021 | Deep Nets 2 (Backpropagation + PyTorch) | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 11 | 09/28/2021 | Ensemble Methods (Boosting/Random Forest/Deep Nets) & Regularization/Cross-Val | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 12 | 09/30/2021 | Structured Prediction (exhaustive search, dynamic programming) | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 13 | 10/05/2021 | Learning Theory | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 14 | 10/07/2021 | Review |   | [PreRec] | [Rec] |
Lecture 15 | 10/12/2021 | PCA, SVD | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 16 | 10/14/2021 | k-Means | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 17 | 10/19/2021 | Gaussian Mixture Models | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 18 | 10/21/2021 | Expectation Maximization | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 19 | 10/26/2021 | Hidden Markov Models | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 20 | 10/28/2021 | Variational Auto-Encoders | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 21 | 11/02/2021 | Generative Adversarial Nets | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 22 | 11/04/2021 | Autoregressive Methods | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 23 | 11/09/2021 | Transformers | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 24 | 11/11/2021 | Graph Neural Nets | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 25 | 11/16/2021 | MDP | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 26 | 11/18/2021 | Q-Learning | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Break | 11/23/2021 | Thanksgiving |   |   |   |
Break | 11/25/2021 | Thanksgiving |   |   |   |
Lecture 27 | 11/30/2021 | Policy Gradient/Actor-Critic | [Slides] [Slides Split] [Lecture Slides] | [PreRec] | [Rec] |
Lecture 28 | 12/02/2021 | Review |   | [PreRec] | [Rec] |
Lecture 29 | 12/07/2021 | Final Exam |   |   |   |