Term: Fall 2020

Prerequisites: ECE 534 (Random Processes) and ECE 515 (Control System Theory and Design)

Instructor: Prof. R. Srikant, rsrikant@illinois.edu

TAs: Joseph Lubars (lubars2@illinois.edu) and Siddhartha Satpathi (ssatpth2@illinois.edu)

Office Hours: Joseph: 4-5 Tu, 5-6 Wed; Siddhartha: 5-6 Tu, 4-5 Wed; Srikant: after class

Lectures: 11-12:20 TuTh, via zoom (link is on Compass 2G under announcements)

Topics (Time Permitting):

• MDPs

• LQG Problems

• Controlled Diffusions

Grading :

• Homework: 60%; Midterm Exam (Nov. 5): 20%; Project: 20% (Due: Dec. 14, 10 pm)

References:

• D. P. Bertsekas, Dynamic Programming and Optimal Control, Vols. I and II, Athena Scientific, 2005, 2012

• D. P. Bertsekas and J. N. Tsitsiklis. Neuro-Dynamic Programming, Athena Scientific, 1996

• V. Krishnamurthy. Partially Observed Markov Decision Processes. Cambridge University Press, 2016.

• P. R. Kumar and P. Varaiya. Stochastic Systems: Estimation, Identification and Adaptive Control, SIAM, 2015

• M. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, 2005

• S. Ross. Applied Probability Models with Optimization Applications, Dover, 1970

• S. Ross. Introduction to Stochastic Dynamic Programming, Academic Press, 1983.

• The notes of Profs. Tamer Basar, Ali Belabbas, Aditya Mahajan, Max Raginsky, Ramon van Handel, and Richard Weber; except for the first one, the rest are available online