# ECE544: Pattern Recognition (Fall 2022)

## Course Information

The goal of Pattern Recognition is to find structure in data. In this course we will cover three main areas, (1) discriminative models, (2) generative models, and (3) reinforcement learning models. In particular we will cover the following: linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.**Pre-requisites:**Probability, linear algebra, and proficiency in Python.

**Recommended Text:**(1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (3) Pattern Recognition and Machine Learning by Christopher Bishop, (4) Graphical Models by Nir Friedman and Daphne Koller, and (5) Reinforcement Learning by Richard Sutton and Andrew Barto.

**Course Deliverables:**

(1) Quiz

(2) Literature review on a lecture of your choice (about 5 pages using NeurIPS latex template)

(3) Project (either based on your interest or using one of the two datasets we'll provide; maximum group size is 3 students; provide a writeup and a presentation describing your project, the approach, the data, and the results)

**Grading:**

40% Quiz; 40% Project; 20% Literature review

Grading policy is subject to change.

**Quiz:**December 6, 11am-12:30pm.

## Instructor & TAs

### Alexander Schwing

**Instructor**

Email: aschwing[at]illinois.edu

Office Hour: Friday 5pm (starting 9/2 using class zoom link)

Website: [link]

### Zhizhen Zhao

**Instructor**

Email: zhizhenz[at]illinois.edu

Office Hour: Friday 4pm (starting 10/14 using class zoom link)

Website: [link]

### Class Time & Location

Class Time: Tuesday, Thursday 11:00AM-12:15PMLocation: Zoom (see class email or Canvas for link)

### Course Discussions

Canvas: [link]### Practice Material

Practice Material: [link]## Lectures

The syllabus is subject to change.

Event | Date | Description | Materials | Recording |
---|---|---|---|---|

Lecture 1 | 08/23/2022 | Introduction | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 2 | 08/25/2022 | Nearest Neighbor | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 3 | 08/30/2022 | Linear Regression | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 4 | 09/01/2022 | Logistic Regression | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 5 | 09/06/2022 | Optimization Primal | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 6 | 09/08/2022 | Optimization Dual | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 7 | 09/13/2022 | Support Vector Machine | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 8 | 09/15/2022 | Multiclass Classification and Kernel Methods | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 9 | 09/20/2022 | Deep Nets 1 (Layers) | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 10 | 09/22/2022 | Deep Nets 2 (Backpropagation + PyTorch) | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 11 | 09/27/2022 | Ensemble Methods (Boosting/Random Forest/Deep Nets) & Regularization/Cross-Val | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 12 | 09/29/2022 | Structured Prediction (exhaustive search, dynamic programming) | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 13 | 10/04/2022 | Learning Theory | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 14 | 10/06/2022 | PCA, SVD | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 15 | 10/11/2022 | k-Means | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 16 | 10/13/2022 | Gaussian Mixture Models | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 17 | 10/18/2022 | Expectation Maximization | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 18 | 10/20/2022 | Hidden Markov Models | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 19 | 10/25/2022 | Variational Auto-Encoders | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 20 | 10/27/2022 | Generative Adversarial Nets | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 21 | 11/01/2022 | Autoregressive Methods | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 22 | 11/03/2022 | Diffusion Models | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Break | 11/08/2022 | General Election Day | ||

Lecture 23 | 11/10/2022 | Transformers/Graph Neural Nets | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 24 | 11/15/2022 | MDP | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 25 | 11/17/2022 | Q-Learning | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Break | 11/22/2022 | Thanksgiving | ||

Break | 11/24/2022 | Thanksgiving | ||

Lecture 26 | 11/29/2022 | Policy Gradient/Actor-Critic | [Slides] [Slides Split] [Lecture Slides] | [Rec] |

Lecture 27 | 12/01/2022 | Review | [Rec] | |

Lecture 28 | 12/06/2022 | Quiz |