ECE544: Pattern Recognition (Fall 2021)

Course Information

The goal of Pattern Recognition is to find structure in data. In this course we will cover three main areas, (1) discriminative models, (2) generative models, and (3) reinforcement learning models. In particular we will cover the following: linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.

Pre-requisites: Probability, linear algebra, and proficiency in Python.

Recommended Text: (1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (3) Pattern Recognition and Machine Learning by Christopher Bishop, (4) Graphical Models by Nir Friedman and Daphne Koller, and (5) Reinforcement Learning by Richard Sutton and Andrew Barto.

Course Deliverables:
(1) Final Exam
(2) Presentations (multiple)
(3) Scribe

To access your deliverable schedule please browse to this file.

Grading:

1/3 class presentations, 1/3 scribe, 1/3 final;


Grading policy is subject to change.

Final Exam: December 7, 11am-12:30pm.

Instructor & TAs

Alexander Schwing

Instructor
Email: aschwing[at]illinois.edu
Office Hour: Friday at 6pm
Website: [link]

Class Time & Location

Class Time: Tuesday, Thursday 11:00AM-12:15PM
Location: Zoom at first (see class email or Canvas for link)

Course Discussions

Canvas: [link]

Practice Material

Practice Material: [link]



Lectures

The syllabus is subject to change.

EventDateDescriptionMaterialsPre-RecordingRecording
Lecture 1 08/24/2021 Introduction [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 2 08/26/2021 Nearest Neighbor [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 3 08/31/2021 Linear Regression [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 4 09/02/2021 Logistic Regression [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 5 09/07/2021 Optimization Primal [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 6 09/09/2021 Optimization Dual [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 7 09/14/2021 Support Vector Machine [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 8 09/16/2021 Multiclass Classification and Kernel Methods [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 9 09/21/2021 Deep Nets 1 (Layers) [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 10 09/23/2021 Deep Nets 2 (Backpropagation + PyTorch) [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 11 09/28/2021 Ensemble Methods (Boosting/Random Forest/Deep Nets) & Regularization/Cross-Val [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 12 09/30/2021 Structured Prediction (exhaustive search, dynamic programming) [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 13 10/05/2021 Learning Theory [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 14 10/07/2021 Review   [PreRec] [Rec]
Lecture 15 10/12/2021 PCA, SVD [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 16 10/14/2021 k-Means [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 17 10/19/2021 Gaussian Mixture Models [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 18 10/21/2021 Expectation Maximization [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 19 10/26/2021 Hidden Markov Models [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 20 10/28/2021 Variational Auto-Encoders [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 21 11/02/2021 Generative Adversarial Nets [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 22 11/04/2021 Autoregressive Methods [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 23 11/09/2021 Transformers [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 24 11/11/2021 Graph Neural Nets [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 25 11/16/2021 MDP [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 26 11/18/2021 Q-Learning [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Break 11/23/2021 Thanksgiving      
Break 11/25/2021 Thanksgiving      
Lecture 27 11/30/2021 Policy Gradient/Actor-Critic [Slides] [Slides Split]  [Lecture Slides] [PreRec] [Rec]
Lecture 28 12/02/2021 Review   [PreRec] [Rec]
Lecture 29 12/07/2021 Final Exam