ECE 561: Statistical Inference for Engineers and Data Scientists

ECE 561: Statistical Inference for Engineers and Data Scientists
SPRING 2023

Course Information

Class time and place: 11:00 - 12:20 TR, 2013 ECEB
Instructor: Prof. Pierre Moulin 310 CSL (pmoulin@illinois.edu)
Office Hours: Wednesdays 10:15 - 11:45 am in 310 CSL
Teaching Assistant: Weichao Mao (weichao2@illinois.edu)
Office hours: Tuesdays 4 - 6 pm in 3034 ECEB
Syllabus
Gradescope course entrycode: 3J27N4
Required Text: P. Moulin and V. V. Veeravalli, Statistical Inference for Engineers and Data Scientists, 2019
Grading: 10% Homework, 25% Exam 1, 25% Exam 2, 40% Final exam
References: A number of references are on reserve in the Engineering Library, including books by T. S. Ferguson, J. O. Berger, E. L. Lehmann (2), S. M. Kay (2), B. C. Levy, H. V. Poor, and H. L. Van Trees

Elements of Statistical Decision Theory (Ch. 1): a general framework for discussing the Bayes and minimax approaches to detection and estimation; M-ary hypothesis testing and Bayesian parameter estimation obtained as special cases.
Hypothesis Testing (Ch. 2, 3, 4): Bayesian decision rules; Minimax decision rules; Neyman-Pearson decision rules (the radar problem); composite hypothesis testing.
Signal Detection in Discrete Time (Ch. 5): models and detector structures; performance evaluation.
Convex Statistical Distances (Ch. 6): f-divergences and Ali-Silvey distances; bounds on classification error.
Chernoff Bounds and Large Deviations (Ch. 7,8): Chernoff divergences, Bhattacharyya distances, Kullback-Leibler divergence, large-deviations analysis, application to hypothesis testing.
Parameter Estimation (Ch. 11-14): Bayesian estimation; nonrandom parameter estimation; Cramer-Rao bounds; maximum likelihood estimation; asymptotic optimality; M-estimators.
Additional Topics: Robust detection and estimation.

Homework 1: Problems 1.3, 1.5, 1.7, and 2.2 from course textbook. Due Thursday Feb. 2, 5pm on Gradescope. Solution.
Homework 2: Problems 2.4, 2.6, 2.8, 2.15, 2.18, 3.1, and 3.5 from course textbook. Due Thursday Feb. 23, 5pm on Gradescope. Solution.
Homework 3: Problems 4.14, 4.15, 5.4, 5.6, 5.7, 5.15, 6.1, 6.9, 7.2, and 7.5 from course textbook. Due Thursday Mar. 23, 5pm on Gradescope. Solution.
Homework 4: Problems 8.2, 8.3 (a) - (c), and 8.6 from course textbook. Due Saturday Apr. 1, 5pm on Gradescope. Solution.
Homework 5: Problems 11.4, 11.7, 11.10, 12.1, 12.3, 12.4, 12.6, 13.4, 13.7, and 13.9 from course textbook. Due Saturday Apr. 22, 5pm on Gradescope. Solution.
Homework 6: Problems 14.1, 14.3, 14.6, and 14.7 from course textbook. Due Thursday May 4, 5pm on Gradescope. Solution.