Mon Tue Wed Thu Fri
8/24 8/25 8/26 8/27 8/28
8/31 9/1 9/2 9/3 9/4
9/7 9/8 9/9 9/10 9/11
9/14 9/15 9/16 9/17 9/18
9/21 9/22 9/23 9/24 9/25
9/28 9/29 9/30 10/1 10/2
10/5 10/6 10/7 10/8 10/9
10/12 10/13 10/14 10/15 10/16
10/19 10/20 10/21 10/22 10/23
10/26 10/27 10/28 10/29 10/30
11/2 11/3 11/4 11/5 11/6
11/9 11/10 11/11 11/12 11/13
11/16 11/17 11/18 11/19 11/20
11/23 11/24 11/25 11/26 11/27
11/30 12/1 12/2 12/3 12/4
12/7 12/8 12/9 12/10 12/11
12/14 12/15 12/16 12/17 12/18

Syllabus: ECE 417

Characteristics of speech and image signals; important analysis and synthesis tools for multimedia signal processing including subspace methods, Bayesian networks, hidden Markov models, and factor graphs; applications to biometrics (person identification), human-computer interaction (face and gesture recognition and synthesis), and audio-visual databases (indexing and retrieval). Emphasis on a set of python machine problems providing hands-on experience. 4 undergraduate hours. 4 graduate hours. Prerequisite: ECE 310 and ECE 313. Assignments | Staff | Resources

Grading Scheme

Grade cutoffs are approximately as follows, where mu=class average, sigma=standard deviation.

Assignments

HomeWork

Written homework will be due every two weeks; write your answers by hand, photograph, and submit to Gradescope.

Directory of homework assignments: HW1 | HW2 | HW3 | HW4 | HW5 | HW6

Machine Problems

Machine problems will be in python, once every two weeks, and will be autograded on gradescope.

Directory of machine problems: MP1 | MP2 | MP3 | MP4 | MP5 | MP6

Late Policy

HW and MPs are accepted up to 7 days late on Gradescope, with only a 5% penalty. If you're more than 7 days late, you'll have to submit by e-mail; in general, credit of up to 50% is possible for any submission, at any time before the end of the semester.

Extensions will not be given for software that didn't work; you should have checked that in advance. Extensions are possible in case of illness, on a case-by-case basis.

Academic Integrity

You are encouraged to consult with other students in your attempts to solve any of the MPs. The only thing that’s expressly forbidden is sharing code.

Exams

Exams will be open-book. They will be taken on Compass. They will be timed, and scheduled at the regular lecture time, unless you notify me in advance of a conflict. Sample exams will be available, in advance, for study.

Directory of exams: Midterm 1 | Midterm 2 | Final

Staff

Lectures and Office Hours

Lectures are Tuesdays and Thursdays at 9:30am, at a zoom URL specified on the course Compass page. Lecture videos will be posted to Mediaspace and echo360 after class. If you have trouble accessing the Mediaspace or echo360 video, please send me e-mail.

Office hours will be Fridays and Mondays, 5-6pm, at a zoom URL specified on the course Compass page. You can also ask questions on piazza.

Resources

Readings

Readings will be listed at the top of each HW or MP. These will be published tutorials or research papers.

Software

Campus Resources


Lectures, Homework, MPs and Exams

Week 1

Lecture 1, T8/25 09:30
Review of DTFT, Gaussians, and Linear Algebra
Ad for a related course that some of you may find interesting
Reading: Strang, Section 6.1
Reading: Gallager, pp. 33-34, 36, 39-43, 45
Reading: ECE 313 lecture notes
Lecture 2, R8/27 09:30
Principal Components and Eigenfaces
Reading: Gallager, pp. 43-45
Reading: Eigenfaces for Recognition, Turk and Pentland, 1991

Week 2

Homework 1, M8/31 23:59
PDF: Review of DTFT, Gaussians, and Eigenvectors (solutions)
Lecture 3, T9/1 09:30
Noise
Lecture 4, R9/3 09:30
Filtered Noise

Week 3

Machine Problem 1, M9/7 23:59
Assignment web page: PCA
Lecture 5, T9/8 09:30
Job posting: Technical manager
Short Time Fourier Transform
Lecture 6, R9/10 09:30
Musical, Perceptual, and Masking Frequency Scales

Week 4

Homework 2, M9/15 23:59
Autocorrelation and Power Spectrum: homework, solutions.
Lecture 7, T9/16 09:30
Neural Nets
Lecture 8, T9/18 09:30
Nonlinearities

Week 5

Monday 9/21: Machine Problem 2 due at 23:59
Speech recognition with mel filterbank and gammatone features
Extra credit: group assignment
Tuesday 9/22 09:30: Exam 1 Review
We'll review the practice exams on Compass
Wednesday 9/23: Extra Office Hours
Wednesday 9/23 10-11am and 6-7pm, for last-minute exam questions.
Thursday 9/24: Midterm 1, 09:30-11:00 AM
The midterm exam will be a timed, open-book, open-notes, open-internet exam, held on Compass. It will appear in your Compass folder at 9:15AM on Thursday 9/24, and will be available until 11:45AM; you may choose any 90-minute period during that time in which to take the exam. You may type your answers in any mixture of plaintext pseudo-math or pseudo-python syntax; as long as we can understand what you mean, you will get the points. Your answer should contain no integrals or infinite-length sums, but otherwise, you do not need to simplify explicit numerical expressions. Examples are available in the two sample exams that are currently available on Compass; you may also find it useful to look at exams from past semesters, though they are in a different format. Reference solutions to all practice exam problems are available on Compass after you submit your answers. Piazza will be open, during the real exam, for private questions to the instructors.

Week 6

Lecture 9, T9/29 09:30
Convolutional Neural Networks
Lecture 10, R10/1 09:30
Faster-RCNN Object Detectors

Week 7

Homework 3, M10/5 23:59
PDF: Back-propagation, loss functions, and nonlinearities. Solutions.
Lecture 11, T10/6 09:30
Discriminative vs. Bayesian Classifiers
Lecture 12, R10/8 09:30
Hidden Markov Models

Week 8

Machine Problem 3, M10/12 23:59
Face detection using a Faster-RCNN object detector
Lecture 13, T10/13 09:30
How to train Observation Probability Densities
Lecture 14, R10/15 09:30
Log Viterbi and Scaled Forward Algorithms

Week 9

Homework 4, M10/19 23:59
Hidden Markov models and Baum-Welch: assignment, solutions
Lecture 15, T10/20 09:30
Weighted Finite State Acceptors
Lecture 16, R10/22 09:30
Weighted Finite State Transducers

Week 10

Machine Problem 4, M10/26 23:59
Assignment page: Speech segmentation using Hidden Markov models
Midterm Review, T10/27 09:30
See sample exams on Compass.
Wednesday 10/28: Extra Office Hours
Wednesday 10/28 11am-noon and 6-7pm, for last-minute exam questions.
Midterm 2, R10/29 09:30
The midterm exam will be a timed, open-book, open-notes, open-internet exam, held on Compass. It will appear in your Compass folder at 9:15AM on Thursday 9/24, and will be available until 12:15PM; you may choose any 90-minute period during that time in which to take the exam. You may type your answers in any mixture of plaintext pseudo-math or pseudo-python syntax; as long as we can understand what you mean, you will get the points. Your answer should contain no integrals or infinite-length sums, but otherwise, you do not need to simplify explicit numerical expressions. Examples are available in the two sample exams that are currently available on Compass; you may also find it useful to look at exams from past semesters, though they are in a different format. Reference solutions to all practice exam problems are available on Compass after you submit your answers. Piazza will be open, during the real exam, for private questions to the instructors.

Week 11

T 11/3
No lecture
R 11/5
Lecture 17: Practical WFSTs

Week 12

Homework 5, W11/11 23:59
Weighted Finite State Transducers: PDF
T 11/10
Lecture 18: Recurrent Neural Networks
R 11/11
Lecture 19: Long/Short-Term Memory

Week 13

Machine Problem 5, W11/18 23:59
Continuous speech recognition using WFSTs (assignment page)
T 11/17
Lecture 20: Moving Pixels Around
R 11/19
Lecture 21: Barycentric Coordinates

Week 14

T 12/1
Lecture 22: Variational Autoencoders
Homework 6, W12/2 23:59
Gated and recurrent neural networks: PDF
R 12/3
Lecture 23: Generative Adversarial Networks. PDF, Examples

Week 15

Machine Problem 6, W12/9 23:59
Video synthesis using LSTMs

Week 16

Final Exam
Friday, December 18, 13:30-16:30. If you need a conflict exam because of time zone, or because of a schedule conflict, please e-mail the instructor at least one week prior.