ECE 434 - Random Processes

• An Old Final Exam and Solutions are available
• Solutions to the second midterm (in jpg format) can now be downloaded: page 1, page 2, page 3, page 4.
• The due date for problem set 6 has been postponed to Wednesday, December 11 in class.
• Solutions to the first midterm (in gif format) can now be downloaded: page 1, page 2, page 3, page 4, page 5, page 6, page 7, page 8.
• Solutions to the probability quiz are now available in pdf and ps format.
• Additional textbooks (written by Bob Gray) that may be helpful are available in pdf format via the following links:
  • Probability, Random Processes, and Ergodic Properties
  • Introduction to Statistical Signal Processing
• NOTE that in order to receive a passing grade on the first midterm, you must have your picture taken.
• The due date for Problem Set 2 has been postponed to Monday, September 30.
• The probability quiz is now available. Download it in either ps or pdf format. Remember that once you look at the quiz, you have two hours to complete it, and no textbooks, notes, etc. are allowed. The quiz is due Monday, September 30.
• The exam dates and locations have been determined. Please take note!
• A webboard has been established for the course. It can be accessed via the webboard login page.
• An incomplete list of errata in the 3rd edition of the Stark and Woods text can be found here. A similar list for the 2nd edition of the Stark and Woods text can be found here.

Course Description:
This is a graduate-level course on random (stochastic) processes, which builds on a first-level (undergraduate) course on probability theory, such as ECE 313. It covers the basic concepts of random processes at a fairly rigorous level, and also discusses applications to communications, signal processing and control systems engineering. To follow the course, in addition to basic notions of probability theory, students are expected to have some familiarity with the basic notions of sets, sequences, convergence, linear algebra, linear systems, and Fourier transforms.

Required Text:

*H. Stark and J. W. Woods, Probability and Random Processes with Applications to Signal Processing , third edition , Prentice Hall, 2001. (Available at the bookstore). Note errata for the second edition .

Reserved References (in the engineering library):

*R.G. Gallager, Discrete Stochastic Processes, Kluwer, 1996.

*H. Stark and J. W. Woods, Probability and Random Processes, and Estimation Theory for Engineers, second edition, Prentice Hall, 1994.

*W.B. Davenport, Jr. and W.L. Root, An Introduction to the Theory of Random Signals and Noise, McGraw Hill, 1987 edition.

*E. Wong and B. Hajek, Stochastic Processes in Engineering Systems, Springer Verlag, 1985.

*A. Papoulis, Probability, Random Variables and Stochastic Processes, 2nd edt., McGraw Hill, 1984.

*E. Wong, Introduction to Random Processes, Springer Verlag, 1983.

*B.D.O. Anderson and J.B. Moore, Optimal Filtering, Prentice Hall, 1979.

*W. Rudin, Principles of Mathematical Analysis, 3rd Edition, McGraw-Hill, New York, 1976.

*R.B. Ash, Basic Probability Theory, Academic Press, 1972.

*L. Breiman, Probability, Addison-Wesley, 1968.

*H. Cramer and M.R. Leadbetter, Stationary and Related Stochastic Processes, Wiley, 1967.

*E. Parzen, Stochastic Processes, Holden Day, 1962.

Grading Policy: Homework (10%), Midterms (25% each) and Final Exam (40%).

*There may be an additional (5%) available for extra credit; however, such credit will only count if you already have at least an A in the course.

*Homework: Collaboration on the homework is permitted, however each student must write and submit independent solutions. Homework is due within the first 5 minutes of the class period on the due date. No late homework will be accepted (unless an extension is granted in advance by the instructor).

*Exams: Closed book. However, 1 8.5"x11" sheet of notes (both sides) is allowed for each exam, and this is cumulative, i.e. 1 page for exam 1, and 2 for exam 2, and 3 for the final.

ECE 434 COURSE OUTLINE -- FALL 2002

I. Review of Probability Theory

1. Basic axioms; probability space and measure; sigma fields
2. Conditional probability and independence
3. Random variable; probability distribution and density
4. Random vectors (multivariate random variables); independence; conditional distributions
5. Functions of random variables and random vectors
6. Expectation
7. Conditional expectation and its properties

II. Sequences of Random Variables

1. Different notions of convergence
2. Limit theorems
3. Large deviations

III. Random Vectors and Minimum Mean Squared Error (MMSE) Estimation

1. Best linear MMSE estimators
2. MMSE estimators
3. Orthogonality principle
4. Jointly Gaussian random variables and vectors

IV. Random Processes

1. Continuous- and discrete-time random processes
2. Stationarity and wide-sense stationarity (WSS)
3. Second-order processes; mean and correlation function spectrum
4. Markov processes and martingales; Gaussian, Wiener, and Poisson processes

V. Calculus for Random Processes

1. Continuity of random processes; differentiation and integration
2. Orthogonal representation of random processes (Karhunen-Loeve expansion)
3. Ergodicity

VI. Stationary Random Processes and Spectral Analysis

1. Wide sense stationary processes (WSS)
2. Power spectral density and its estimation
3. Random processes through linear systems
4. Spectral representation of random processes

VII. Minimum Mean Squared Error (MMSE) Estimation

1. MMSE estimation and linear MMSE estimation for random vectors (the orthogonality principle)
2. Discrete- and continuous-time Kalman filter
3. The Wiener filter; spectral factorization
4. Some applications

1. Lecture 1: 8/28/02 Text Sec 1.1 - 1.5
2. Lecture 2: 9/4/02 Text Sec 1.6 - 2.4
3. Lecture 3: 9/9/02 Text Sec 2.4-2.5, 3.1-3.2, 4.1
4. Lecture 4: 9/11/02 Text Sec 3.2-3.4, 4.1, 4.3, 4.5, 4.7
5. Lecture 5: 9/16/02 Text Sec 2.6, 3.3-3.6, 4.2, 4.4, 4.6, 4.7
6. Lecture 6: 9/18/02 Text Sec 6.7
7. Lecture 7: 9/23/02 Text Sec 6.7
8. Lecture 8: 9/25/02 Text Sec 6.8
9. Lecture 9: 9/30/02 Text Sec 5.1-5.4, 9.1
10. Lecture 10: 10/7/02 Text Sec 5.5-5.11
11. Lecture 11: 10/9/02 Text Sec 5.5-5.11, Gray ISSP 4.6-4.10, notes
12. Lecture 12: 10/14/02 Text Sec 6.8, 6.1, 7.1
13. Lecture 13: 10/16/02 Text Sec 6.2-6.4 (through p351)
14. Lecture 14: 10/21/02 Text Sec 7.1-7.2 (through p421)
15. Lecture 15: 10/23/02 Text 7.3,7.4
16. Lecture 16: 10/28/02 Text Sec 6.5,7.2
17. Lecture 17: 10/30/02 Text Sec 8.1-8.2
18. Lecture 18: 11/04/02 Text Sec 8.2-8.3
19. Lecture 19: 11/06/02 Text Sec 8.4-8.5
20. Lecture 20: 11/11/02 Text Sec 7.3-7.5
21. Lecture 21: 11/13/02 Text Sec 6.4,7.5
22. Lecture 22: 11/18/02 Text Sec 8.6
23. Lecture 23: 12/02/02 Text Sec 7.6-7.7, 8.6
24. Lecture 24: 12/04/02 Text Sec 9.2
25. Lecture 25: 12/09/02 Text Sec 9.3
26. Lecture 26: 12/10/02 Text Sec 9.3