H. Stark and J. W. Woods, Probability
and Random Processes with Applications to Signal Processing.
Third Edition, Prentice Hall, 2001. [errata]
R. G. Gallager, Discrete
Stochastic Processes. Kluwer, 1996.
H. Stark and J. W. Woods, Probability
and Random Processes with Applications to Signal Processing.
Second Edition, Prentice Hall, 1994. [errata]
W. B. Davenport, Jr. and W. L. Root, An Introduction to the Theory of Random
Signals and Noise. McGraw Hill, 1987.
E. Wong and B. Hajek, Stochastic
Processes in Engineering Systems. Springer Verlag, 1985.
A. Papoulis, Probability,
Random Variables and Stochastic Processes. Second Edition,
McGraw Hill, 1984.
E. Wong, Introduction to
Random Processes. Springer Verlag, 1983.
B. D. O. Anderson and J. B. Moore, Optimal Filtering. Prentice Hall,
1979.
W. Rudin, Principles of
Mathematical Analysis. Third Edition, McGraw-Hill, New York,
1976.
R. B. Ash, Basic Probability
Theory. Academic Press, 1972.
L. Breiman, Probability.
Addison-Wesley, 1968.
H. Cramer and M. R. Leadbetter, Stationary
and Related Stochastic Processes. Wiley, 1967.
E. Parzen, Stochastic
Processes. Holden Day, 1962.
Grading and Exams:
Homework (10%), Probability Quiz on 02/11 (10%), Midterm 1 on 03/17
(20%), Midterm 2 on 04/21 (20%) and Final Exam on 05/07 (40%).
Exams are closed book; one 8.5"x11" sheet of notes (both sides)
is permitted for Midterm 1, two sheets are permitted for Midterm 2, and
three for the final. The Probability Quiz and the Midterm
Exams will be from 5-7pm on the above dates.
Homework Policy:
Collaboration on the homework is permitted but each student must write
and submit independent solutions. Homework is due within the
first five minutes of the class period on the due date.
Late homework will not be accepted without prior arrangement with the
instructor. The bottom homework grade will be dropped to
accommodate one missed homework for every student.
Lecture Topics:
1. Review of
Probability Theory
Basic axioms; probability space and measure; sigma algebras
Conditional probability and independence
Random variable; probability distribution and density
Random vectors; conditional distributions and independence
Functions of random variables and random vectors
Expectation; conditional expectation and properties
2. Sequences of Random Variables
Notions of convergence
Limit theorems; large deviations
3. Random Vectors and Minimum Mean
Squared Error (MMSE) Estimation
Linear MMSE and MMSE estimators
Orthogonality principle
Jointly Gaussian random variables and vectors
Kalman filtering
4. Random Processes
Continuous- and discrete-time random processes
Stationarity and wide-sense stationarity (WSS)
Second-order processes; mean and correlation function spectrum
Markov processes and martingales; Gaussian, Wiener, and Poisson
processes
5. Calculus for Random Processes
Continuity of random processes; differentiation and integration
Orthogonal representation of random processes (Karhunen-Loeve
expansion)
Ergodicity
6. Stationary Random Processes and
Spectral Analysis
Power spectral density and its estimation
Random processes through linear systems
Spectral representation of random processes
7. Minimum Mean Squared Error (MMSE)
Estimation
MMSE estimation and linear MMSE estimation for random vectors
Discrete- and continuous-time Kalman filter
The Wiener filter; spectral factorization and applications