Syllabus: ECE 417
Characteristics of speech and image signals; important analysis and synthesis tools for multimedia signal processing including subspace methods, Bayesian networks, hidden Markov models, and factor graphs; applications to biometrics (person identification), human-computer interaction (face and gesture recognition and synthesis), and audio-visual databases (indexing and retrieval). Emphasis on a set of python machine problems providing hands-on experience. 4 undergraduate hours. 4 graduate hours. Prerequisite: ECE 310 and ECE 313. Assignments | Staff | Resources
- 15%: Written homework
- 45%: Machine problems
- 10% each Midterm
- 20% Final Exam
- Up to 1% extra credit for answering the questions of other students on piazza.
Grade cutoffs are approximately as follows, where mu=class average, sigma=standard deviation.
- min(90%, mu+0.25sigma): A-, min(94%, mu+0.5sigma): A, min(99%, mu+1.5sigma): A+
- min(75%, mu-sigma): B-, min(80%, mu-0.75sigma): B, min(85%, mu): B+
- min(60%, mu-2.25sigma): C-, min(65%, mu-2sigma): C, min(70%, mu-1.25sigma): C+
- min(45%, mu-3sigma): D-, min(50%, mu-2.75sigma): D, min(55%, mu-2.5sigma): D+
Written homework will be due every two weeks; write your answers by hand, photograph, and submit to Gradescope.
Machine problems will be in python, once every two weeks, and will be autograded on gradescope.
HW and MPs are accepted up to 7 days late on Gradescope, with only a 5% penalty. If you're more than 7 days late, you'll have to submit by e-mail; in general, credit of up to 50% is possible for any submission, at any time before the end of the semester.
Extensions will not be given for software that didn't work; you should have checked that in advance. Extensions are possible in case of illness, on a case-by-case basis.
You are encouraged to consult with other students in your attempts to solve any of the MPs. The only thing that’s expressly forbidden is sharing code.
- Do not: share code. Don’t send lines of python by text message or e-mail, don’t post it on your blog, don’t share it on github, don’t post it in piazza.
- Do: consult with other students about how to solve a problem. You can discuss code on a teleconference, as long as nobody's copying the code verbatim. You are encouraged to share pseudo-code and natural language descriptions of algorithms. You are allowed to send chat messages etc. with pseudo-code – just don’t send actual lines of python.
Exams will be open-book. They will be taken on Compass. They will be timed, and scheduled at the regular lecture time, unless you notify me in advance of a conflict. Sample exams will be available, in advance, for study.
Lectures and Office Hours
Lectures are Tuesdays and Thursdays at 9:30am, at a zoom URL specified on the course Compass page. Lecture videos will be posted to Mediaspace and echo360 after class. If you have trouble accessing the Mediaspace or echo360 video, please send me e-mail.
Office hours will be Fridays and Mondays, 5-6pm, at a zoom URL specified on the course Compass page. You can also ask questions on piazza.
Readings will be listed at the top of each HW or MP. These will be published tutorials or research papers.
- Statements from the undergraduate advising office about sexual misconduct, academic integrity, religious observances, disability-related accommodations, and FERPA
- Home page of the undergraduate advising office
Lectures, Homework, MPs and Exams
- Lecture 1, T8/25 09:30
- Review of DTFT, Gaussians, and Linear Algebra
- Ad for a related course that some of you may find interesting
- Reading: Strang, Section 6.1
- Reading: Gallager, pp. 33-34, 36, 39-43, 45
- Reading: ECE 313 lecture notes
- Lecture 2, R8/27 09:30
- Principal Components and Eigenfaces
- Reading: Gallager, pp. 43-45
- Reading: Eigenfaces for Recognition, Turk and Pentland, 1991
- Homework 1, M8/31 23:59
- PDF: Review of DTFT, Gaussians, and Eigenvectors (solutions)
- Lecture 3, T9/1 09:30
- Lecture 4, R9/3 09:30
- Filtered Noise
- Machine Problem 1, M9/7 23:59
- Assignment web page: PCA
- Lecture 5, T9/8 09:30
- Job posting: Technical manager
- Short Time Fourier Transform
- Lecture 6, R9/10 09:30
- Musical, Perceptual, and Masking Frequency Scales
- Homework 2, M9/15 23:59
- Autocorrelation and Power Spectrum: homework, solutions.
- Lecture 7, T9/16 09:30
- Neural Nets
- Lecture 8, T9/18 09:30
- Monday 9/21: Machine Problem 2 due at 23:59
- Speech recognition with mel filterbank and gammatone features
- Extra credit: group assignment
- Tuesday 9/22 09:30: Exam 1 Review
- We'll review the practice exams on Compass
- Wednesday 9/23: Extra Office Hours
- Wednesday 9/23 10-11am and 6-7pm, for last-minute exam questions.
- Thursday 9/24: Midterm 1, 09:30-11:00 AM
- The midterm exam will be a timed, open-book, open-notes, open-internet exam, held on Compass. It will appear in your Compass folder at 9:15AM on Thursday 9/24, and will be available until 11:45AM; you may choose any 90-minute period during that time in which to take the exam. You may type your answers in any mixture of plaintext pseudo-math or pseudo-python syntax; as long as we can understand what you mean, you will get the points. Your answer should contain no integrals or infinite-length sums, but otherwise, you do not need to simplify explicit numerical expressions. Examples are available in the two sample exams that are currently available on Compass; you may also find it useful to look at exams from past semesters, though they are in a different format. Reference solutions to all practice exam problems are available on Compass after you submit your answers. Piazza will be open, during the real exam, for private questions to the instructors.
- Lecture 9, T9/29 09:30
- Convolutional Neural Networks
- Lecture 10, R10/1 09:30
- Faster-RCNN Object Detectors
- Homework 3, M10/5 23:59
- PDF: Back-propagation, loss functions, and nonlinearities. Solutions.
- Lecture 11, T10/6 09:30
- Discriminative vs. Bayesian Classifiers
- Lecture 12, R10/8 09:30
- Hidden Markov Models
- Machine Problem 3, M10/12 23:59
- Face detection using a Faster-RCNN object detector
- Lecture 13, T10/13 09:30
- How to train Observation Probability Densities
- Lecture 14, R10/15 09:30
- Log Viterbi and Scaled Forward Algorithms
- Homework 4, M10/19 23:59
- Hidden Markov models and Baum-Welch: assignment, solutions
- Lecture 15, T10/20 09:30
- Weighted Finite State Acceptors
- Lecture 16, R10/22 09:30
- Weighted Finite State Transducers
- Machine Problem 4, M10/26 23:59
- Assignment page: Speech segmentation using Hidden Markov models
- Midterm Review, T10/27 09:30
- See sample exams on Compass.
- Wednesday 10/28: Extra Office Hours
- Wednesday 10/28 11am-noon and 6-7pm, for last-minute exam questions.
- Midterm 2, R10/29 09:30
- The midterm exam will be a timed, open-book, open-notes, open-internet exam, held on Compass. It will appear in your Compass folder at 9:15AM on Thursday 9/24, and will be available until 12:15PM; you may choose any 90-minute period during that time in which to take the exam. You may type your answers in any mixture of plaintext pseudo-math or pseudo-python syntax; as long as we can understand what you mean, you will get the points. Your answer should contain no integrals or infinite-length sums, but otherwise, you do not need to simplify explicit numerical expressions. Examples are available in the two sample exams that are currently available on Compass; you may also find it useful to look at exams from past semesters, though they are in a different format. Reference solutions to all practice exam problems are available on Compass after you submit your answers. Piazza will be open, during the real exam, for private questions to the instructors.
- T 11/3
- No lecture
- R 11/5
- Lecture 17: Practical WFSTs
- Homework 5, W11/11 23:59
- Weighted Finite State Transducers: PDF
- T 11/10
- Lecture 18: Recurrent Neural Networks
- R 11/11
- Lecture 19: Long/Short-Term Memory
- Machine Problem 5, W11/18 23:59
- Continuous speech recognition using WFSTs (assignment page)
- T 11/17
- Lecture 20: Moving Pixels Around
- R 11/19
- Lecture 21: Barycentric Coordinates
- T 12/1
- Lecture 22: Variational Autoencoders
- Homework 6, W12/2 23:59
- Gated and recurrent neural networks: PDF
- R 12/3
- Lecture 23: Generative Adversarial Networks. PDF, Examples
- Machine Problem 6, W12/9 23:59
- Video synthesis using LSTMs
- Final Exam
- Friday, December 18, 13:30-16:30. If you need a conflict exam because of time zone, or because of a schedule conflict, please e-mail the instructor at least one week prior.