ECE 561: Statistical Inference for Engineers and Data Scientists
SPRING 2023
Course Information
- Class time and place: 11:00 - 12:20 TR, 2013 ECEB
- Instructor: Prof. Pierre Moulin 310 CSL (pmoulin@illinois.edu)
Office Hours: Wednesdays 10:15 - 11:45 am in 310 CSL
- Teaching Assistant: Weichao Mao (weichao2@illinois.edu)
Office hours: Tuesdays 4 - 6 pm in 3034 ECEB
- Syllabus
- Gradescope course entrycode: 3J27N4
- Required Text: P. Moulin and V. V. Veeravalli, Statistical Inference for Engineers and Data Scientists, 2019
- Grading: 10% Homework, 25% Exam 1, 25% Exam 2, 40% Final exam
- References: A number of references are on reserve in the Engineering Library, including books by T. S. Ferguson, J. O. Berger, E. L. Lehmann (2), S. M. Kay (2), B. C. Levy, H. V. Poor, and H. L. Van Trees
Outline
- Elements of Statistical Decision Theory (Ch. 1): a general framework for discussing the Bayes and minimax approaches to detection and estimation; M-ary hypothesis testing and Bayesian parameter estimation obtained as special cases.
- Hypothesis Testing (Ch. 2, 3, 4): Bayesian decision rules; Minimax decision rules; Neyman-Pearson decision rules (the radar problem); composite hypothesis testing.
- Signal Detection in Discrete Time (Ch. 5): models and detector structures; performance evaluation.
- Convex Statistical Distances (Ch. 6): f-divergences and Ali-Silvey distances; bounds on classification error.
- Chernoff Bounds and Large Deviations (Ch. 7,8): Chernoff divergences, Bhattacharyya distances, Kullback-Leibler divergence, large-deviations analysis, application to hypothesis testing.
- Parameter Estimation (Ch. 11-14): Bayesian estimation; nonrandom parameter estimation; Cramer-Rao bounds; maximum likelihood estimation; asymptotic optimality; M-estimators.
- Additional Topics: Robust detection and estimation.
Announcements
- Midterm 1 will be held on Tuesday, March 7, 7-9 pm.
- Midterm 2 will be held on Tuesday, April 4, 7-9 pm.
Homework
- Homework 1: Problems 1.3, 1.5, 1.7, and 2.2 from course textbook. Due Thursday Feb. 2, 5pm on Gradescope. Solution.
- Homework 2: Problems 2.4, 2.6, 2.8, 2.15, 2.18, 3.1, and 3.5 from course textbook. Due Thursday Feb. 23, 5pm on Gradescope. Solution.
- Homework 3: Problems 4.14, 4.15, 5.4, 5.6, 5.7, 5.15, 6.1, 6.9, 7.2, and 7.5 from course textbook. Due Thursday Mar. 23, 5pm on Gradescope. Solution.
- Homework 4: Problems 8.2, 8.3 (a) - (c), and 8.6 from course textbook. Due Saturday Apr. 1, 5pm on Gradescope. Solution.
- Homework 5: Problems 11.4, 11.7, 11.10, 12.1, 12.3, 12.4, 12.6, 13.4, 13.7, and 13.9 from course textbook. Due Saturday Apr. 22, 5pm on Gradescope. Solution.
- Homework 6: Problems 14.1, 14.3, 14.6, and 14.7 from course textbook. Due Thursday May 4, 5pm on Gradescope. Solution.
Past Exams