ECE 566: COMPUTATIONAL INFERENCE AND LEARNING, FALL 2025

• Course Description:

  Computational inference and machine learning have seen a surge of interest in the last 20 years, motivated by applications as diverse as computer vision, speech recognition, analysis of networks and distributed systems, big-data analytics, large-scale computer simulations, and indexing and searching of very large databases. This course introduces mathematical and computational methods that enable such applications. Topics include computational methods for statistical inference, information theory, sparsity analysis, approximate inference and search, and fast optimization.

  The course complements ECE561 (Statistical Inference for Engineers and Data Scientists), ECE544NA (Pattern Recognition and Machine Learning), and ECE543 (Statistical Learning Theory) which introduce core theory for statistical inference and machine learning respectively, but do not focus on computational methods. Teaching materials include notes from the instructor and articles from scientific journals.

  Prerequisites: ECE490 and ECE534.

  Class time and place: 2:00-3:20 PM TR. 3020 ECE building.
  Instructor: Prof. Pierre Moulin. Room 310 CSL. Email: pmoulin at illinois dot edu
  Office Hours: 10:00-11:30 AM Wednesdays. Room 310 CSL.
  TA: Pragati Meshram, Room 448 CSL,  Email: psm12 at illinois dot edu
  Office Hours: 4:00 PM-6:00 PM Mondays. Room 4036 ECE building. (Please email TA before coming)
  Grading: Homeworks (20%), Quiz (10%), Midterm exam (30%), and a Final project (40%).
  Homework submissions: Gradescope. Entry code: will be posted
  Project: List of Topics

Course Material

• ECE561 book by P. Moulin and V. Veeravalli:
  Chapter 1 and 2: Introduction, Hypothesis Testing

Class Notes

• Chapter 1
  Introduction: Review of random variables, optimization, convexity, convergence

Homeworks

• Homework-1. Due on Friday, Sept 12 on Gradescope.

Announcements

Syllabus

• Chapter 1: Introduction
  Review of random variables, optimization, convexity, convergence
• Chapter 2: Inference and Learning
  Maximum likelihood principle, Bayes inference, MMSE and MAP estimation, Empirical Risk Minimization
• Chapter 3: Stochastic Sampling
  Stochastic Approximation, Stochastic Gradient Descent, Monte Carlo methods, importance sampling, Bootstrap, Particle Filtering
• Chapter 4: Expectation Maximization (EM)
  Parameter estimation using the EM algorithm
• Chapter 5: Dynamical Systems
  Hidden Markov Models, Viterbi algorithm, Baum-Welch learning, Linear dynamical systems, RTS smoother
• Chapter 6: Graphical Models
  Exact and approximate inference, Gaussian graphs, Gibbs sampling
• Chapter 7: Variational Inference
  Variational methods, mean-field techniques, variational autoencoders
• Chapter 8: Nonsmooth Optimization
  L1-penalized least squares minimization, proximal minimization
• Chapter 9: Compressive Sensing
  Sparsity analysis, reconstruction of sparse signals
• Chapter 10: Matrix Sensing
  Matrix sensing and matrix completion
• Chapter 11: Similarity Estimation
  Approximate nearest-neighbor search, dimensionality reduction using random projections, hashing