ECE544: Pattern Recognition (Fall 2022)

Course Information

The goal of Pattern Recognition is to find structure in data. In this course we will cover three main areas, (1) discriminative models, (2) generative models, and (3) reinforcement learning models. In particular we will cover the following: linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.

Pre-requisites: Probability, linear algebra, and proficiency in Python.

Recommended Text: (1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (3) Pattern Recognition and Machine Learning by Christopher Bishop, (4) Graphical Models by Nir Friedman and Daphne Koller, and (5) Reinforcement Learning by Richard Sutton and Andrew Barto.

Course Deliverables:
(1) Quiz
(2) Literature review on a lecture of your choice (about 5 pages using NeurIPS latex template)
(3) Project (either based on your interest or using one of the two datasets we'll provide; maximum group size is 3 students; provide a writeup and a presentation describing your project, the approach, the data, and the results)

Grading:

40% Quiz; 40% Project; 20% Literature review


Grading policy is subject to change.

Quiz: December 6, 11am-12:30pm.

Instructor & TAs

Alexander Schwing

Instructor
Email: aschwing[at]illinois.edu
Office Hour: Friday 5pm (starting 9/2 using class zoom link)
Website: [link]

Zhizhen Zhao

Instructor
Email: zhizhenz[at]illinois.edu
Office Hour: Friday 4pm (starting 10/14 using class zoom link)
Website: [link]

Class Time & Location

Class Time: Tuesday, Thursday 11:00AM-12:15PM
Location: Zoom (see class email or Canvas for link)

Course Discussions

Canvas: [link]

Practice Material

Practice Material: [link]
 

 

Lectures

The syllabus is subject to change.

Event Date Description Materials Recording
Lecture 1 08/23/2022 Introduction [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 2 08/25/2022 Nearest Neighbor [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 3 08/30/2022 Linear Regression [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 4 09/01/2022 Logistic Regression [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 5 09/06/2022 Optimization Primal [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 6 09/08/2022 Optimization Dual [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 7 09/13/2022 Support Vector Machine [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 8 09/15/2022 Multiclass Classification and Kernel Methods [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 9 09/20/2022 Deep Nets 1 (Layers) [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 10 09/22/2022 Deep Nets 2 (Backpropagation + PyTorch) [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 11 09/27/2022 Ensemble Methods (Boosting/Random Forest/Deep Nets) & Regularization/Cross-Val [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 12 09/29/2022 Structured Prediction (exhaustive search, dynamic programming) [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 13 10/04/2022 Learning Theory [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 14 10/06/2022 PCA, SVD [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 15 10/11/2022 k-Means [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 16 10/13/2022 Gaussian Mixture Models [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 17 10/18/2022 Expectation Maximization [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 18 10/20/2022 Hidden Markov Models [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 19 10/25/2022 Variational Auto-Encoders [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 20 10/27/2022 Generative Adversarial Nets [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 21 11/01/2022 Autoregressive Methods [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 22 11/03/2022 Diffusion Models [Slides] [Slides Split]  [Lecture Slides] [Rec]
Break 11/08/2022 General Election Day    
Lecture 23 11/10/2022 Transformers/Graph Neural Nets [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 24 11/15/2022 MDP [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 25 11/17/2022 Q-Learning [Slides] [Slides Split]  [Lecture Slides] [Rec]
Break 11/22/2022 Thanksgiving    
Break 11/24/2022 Thanksgiving    
Lecture 26 11/29/2022 Policy Gradient/Actor-Critic [Slides] [Slides Split]  [Lecture Slides] [Rec]
Lecture 27 12/01/2022 Review   [Rec]
Lecture 28 12/06/2022 Quiz