ECE 434 - Random Processes

• Quiz 2 solution can be found here .
• The second exam will cover material through problem set #5, and through Chapter 5 of the course notes. An old exam 2 can be found here. Another one can be found here with solution here .
• The first exam will cover material through problem set #3, and through Chapter 3 of the course notes. An old exam 1 (not quite the final version) can be found here.
• The Probability Quiz can be found here.
• The course text for this semester is "An Exploration of Random Processes for Engineers" by B. Hajek. This text is available for download here and is also available at TIS Copy Shop and the Illini Union Bookstore for a nominal xeroxing fee.
• An incomplete list of errata in the 3rd edition of the Stark and Woods text can be found here. A similar list for the 2nd edition of the Stark and Woods text can be found here.

Course Description:
This is a graduate-level course on random (stochastic) processes, which builds on a first-level (undergraduate) course on probability theory, such as ECE 313. It covers the basic concepts of random processes at a fairly rigorous level, and also discusses applications to communications, signal processing and control systems engineering. To follow the course, in addition to basic notions of probability theory, students are expected to have some familiarity with the basic notions of sets, sequences, convergence, linear algebra, linear systems, and Fourier transforms.

Required Text:

"An Exploration of Random Processes for Engineers" by B. Hajek. This text is available for download here and is also available at TIS Copy Shop and the Illini Union Bookstore for a nominal xeroxing fee.

Reserved References(in the engineering library):

*H. Stark and J. W. Woods, Probability and Random Processes with Applications to Signal Processing , third edition , Prentice Hall, 2001. (Available at the bookstore). Note errata for the second edition .

*R.G. Gallager, Discrete Stochastic Processes, Kluwer, 1996.

*H. Stark and J. W. Woods, Probability and Random Processes, and Estimation Theory for Engineers, second edition, Prentice Hall, 1994.

*W.B. Davenport, Jr. and W.L. Root, An Introduction to the Theory of Random Signals and Noise, McGraw Hill, 1987 edition.

*E. Wong and B. Hajek, Stochastic Processes in Engineering Systems, Springer Verlag, 1985.

*A. Papoulis, Probability, Random Variables and Stochastic Processes, 2nd edt., McGraw Hill, 1984.

*E. Wong, Introduction to Random Processes, Springer Verlag, 1983.

*B.D.O. Anderson and J.B. Moore, Optimal Filtering, Prentice Hall, 1979.

*W. Rudin, Principles of Mathematical Analysis, 3rd Edition, McGraw-Hill, New York, 1976.

*R.B. Ash, Basic Probability Theory, Academic Press, 1972.

*L. Breiman, Probability, Addison-Wesley, 1968.

*H. Cramer and M.R. Leadbetter, Stationary and Related Stochastic Processes, Wiley, 1967.

*E. Parzen, Stochastic Processes, Holden Day, 1962.

Grading Policy:Homework (10%), Probability Quiz 9/17 in class (10%), Midterm 1 10/22 in class (20%), Midterm 2 11/19 in class (20%) and Final Exam (40%). Up to 5% extra credit available, however credit only applied to students already receiving at least an A.

*Homework: Collaboration on the homework is permitted, however each student must write and submit independent solutions. Homework is due within the first 5 minutes of the class period on the due date. No late homework will be accepted (unless an extension is granted in advance by the instructor). The bottom homework grade will be dropped to accomodate 1 missed homework for every student.

*Exams: Closed book. The probability quiz is closed book. For the first midterm exam 1 8.5"x11" sheet of notes (both sides) is allowed, 2 sheets are permitted for exam 2, and 3 for the final.

ECE 434 COURSE OUTLINE -- FALL 2003

I. Review of Probability Theory

1. Basic axioms; probability space and measure; sigma fields
2. Conditional probability and independence
3. Random variable; probability distribution and density
4. Random vectors (multivariate random variables); independence; conditional distributions
5. Functions of random variables and random vectors
6. Expectation
7. Conditional expectation and its properties

II. Sequences of Random Variables

1. Different notions of convergence
2. Limit theorems
3. Large deviations

III. Random Vectors and Minimum Mean Squared Error (MMSE) Estimation

1. Best linear MMSE estimators
2. MMSE estimators
3. Orthogonality principle
4. Jointly Gaussian random variables and vectors

IV. Random Processes

1. Continuous- and discrete-time random processes
2. Stationarity and wide-sense stationarity (WSS)
3. Second-order processes; mean and correlation function spectrum
4. Markov processes and martingales; Gaussian, Wiener, and Poisson processes

V. Calculus for Random Processes

1. Continuity of random processes; differentiation and integration
2. Orthogonal representation of random processes (Karhunen-Loeve expansion)
3. Ergodicity

VI. Stationary Random Processes and Spectral Analysis

1. Wide sense stationary processes (WSS)
2. Power spectral density and its estimation
3. Random processes through linear systems
4. Spectral representation of random processes

VII. Minimum Mean Squared Error (MMSE) Estimation

1. MMSE estimation and linear MMSE estimation for random vectors (the orthogonality principle)
2. Discrete- and continuous-time Kalman filter
3. The Wiener filter; spectral factorization
4. Some applications

Back to top

1. Lecture 1: 8/27/03 Text Chapter 1 through 1.2
2. Lecture 2: 9/3/03 Text Chapter 1 through 1.3
3. Lecture 3: 9/8/03 Text Finish Chapter 1
4. Lecture 4: 9/10/03 Text Chapter 2 through 2.1
5. Lecture 5: 9/15/03 Text Finish Chapter 2
6. Lecture 6: 9/17/03 Probability Quiz
7. Lecture 7: 9/22/03 Text Finish Chapter 2
8. Lecture 8: 9/24/03 Text Chapter 3 through 3.2
9. Lecutre 9: 10/1/03 Text Finish Chapter 3
10. Lecture 10: 10/6/03 Text Chapter 4
11. Lecture 11: 10/8/03 Text Chapter 4
12. Lecture 12: 10/13/03 Text Chapter 4
13. Lecture 13: 10/15/03 Text Chapter 4
14. Lecture 14: 10/20/03 Text Chapter 5
15. Lecture 15: 10/27/03 Text Chapter 5
16. Lecture 16: 10/29/03 Text Chapter 5
17. Lecture 17: 11/3/03 Text Chapter 5
18. Lecture 18: 11/5/03 Text Chapter 6
19. Lecture 19: 11/10/03 Text Chapter 6
20. Lecture 20: 11/12/03 Text Chapter 6
21. Lecture 20: 11/17/03 Text Chapter 6
22. Lecture 21: 12/1/03 Text Chapter 7