ECE 417

Mon	Tue	Wed	Thu	Fri
8/24	8/25	8/26	8/27	8/28
8/31	9/1	9/2	9/3	9/4
9/7	9/8	9/9	9/10	9/11
9/14	9/15	9/16	9/17	9/18
9/21	9/22	9/23	9/24	9/25
9/28	9/29	9/30	10/1	10/2
10/5	10/6	10/7	10/8	10/9
10/12	10/13	10/14	10/15	10/16
10/19	10/20	10/21	10/22	10/23
10/26	10/27	10/28	10/29	10/30
11/2	11/3	11/4	11/5	11/6
11/9	11/10	11/11	11/12	11/13
11/16	11/17	11/18	11/19	11/20
11/23	11/24	11/25	11/26	11/27
11/30	12/1	12/2	12/3	12/4
12/7	12/8	12/9	12/10	12/11
12/14	12/15	12/16	12/17	12/18

Syllabus: ECE 417

Characteristics of speech and image signals; important analysis and synthesis tools for multimedia signal processing including subspace methods, Bayesian networks, hidden Markov models, and factor graphs; applications to biometrics (person identification), human-computer interaction (face and gesture recognition and synthesis), and audio-visual databases (indexing and retrieval). Emphasis on a set of python machine problems providing hands-on experience. 4 undergraduate hours. 4 graduate hours. Prerequisite: ECE 310 and ECE 313. Assignments | Staff | Resources

Grading Scheme

15%: Written homework
45%: Machine problems
10% each Midterm
20% Final Exam
Up to 1% extra credit for answering the questions of other students on piazza.

Grade cutoffs are approximately as follows, where mu=class average, sigma=standard deviation.

min(90%, mu+0.25sigma): A-, min(94%, mu+0.5sigma): A, min(99%, mu+1.5sigma): A+
min(75%, mu-sigma): B-, min(80%, mu-0.75sigma): B, min(85%, mu): B+
min(60%, mu-2.25sigma): C-, min(65%, mu-2sigma): C, min(70%, mu-1.25sigma): C+
min(45%, mu-3sigma): D-, min(50%, mu-2.75sigma): D, min(55%, mu-2.5sigma): D+

Assignments

HomeWork

Written homework will be due every two weeks; write your answers by hand, photograph, and submit to Gradescope.

Directory of homework assignments: HW1 | HW2 | HW3 | HW4 | HW5 | HW6

Machine Problems

Machine problems will be in python, once every two weeks, and will be autograded on gradescope.

Directory of machine problems: MP1 | MP2 | MP3 | MP4 | MP5 | MP6

Late Policy

HW and MPs are accepted up to 7 days late on Gradescope, with only a 5% penalty. If you're more than 7 days late, you'll have to submit by e-mail; in general, credit of up to 50% is possible for any submission, at any time before the end of the semester.

Extensions will not be given for software that didn't work; you should have checked that in advance. Extensions are possible in case of illness, on a case-by-case basis.

Academic Integrity

You are encouraged to consult with other students in your attempts to solve any of the MPs. The only thing that’s expressly forbidden is sharing code.

Do not: share code. Don’t send lines of python by text message or e-mail, don’t post it on your blog, don’t share it on github, don’t post it in piazza.
Do: consult with other students about how to solve a problem. You can discuss code on a teleconference, as long as nobody's copying the code verbatim. You are encouraged to share pseudo-code and natural language descriptions of algorithms. You are allowed to send chat messages etc. with pseudo-code – just don’t send actual lines of python.

Exams

Exams will be open-book. They will be taken on Compass. They will be timed, and scheduled at the regular lecture time, unless you notify me in advance of a conflict. Sample exams will be available, in advance, for study.

Directory of exams: Midterm 1 | Midterm 2 | Final

Staff

Professor: Mark Hasegawa-Johnson
Teaching Assistant: Mahir Morshed

Lectures and Office Hours

Lectures are Tuesdays and Thursdays at 9:30am, at a zoom URL specified on the course Compass page. Lecture videos will be posted to Mediaspace and echo360 after class. If you have trouble accessing the Mediaspace or echo360 video, please send me e-mail.

Office hours will be Fridays and Mondays, 5-6pm, at a zoom URL specified on the course Compass page. You can also ask questions on piazza.

Resources

Readings

Readings will be listed at the top of each HW or MP. These will be published tutorials or research papers.

Software

Python and NumPy:

Campus Resources

COVID19
Statements from the undergraduate advising office about sexual misconduct, academic integrity, religious observances, disability-related accommodations, and FERPA
Home page of the undergraduate advising office

Lectures, Homework, MPs and Exams

Week 4

Homework 2, M9/15 23:59: Autocorrelation and Power Spectrum: homework, solutions.
Lecture 7, T9/16 09:30: Neural Nets
Lecture 8, T9/18 09:30: Nonlinearities

Week 5

Monday 9/21: Machine Problem 2 due at 23:59: Speech recognition with mel filterbank and gammatone features; Extra credit: group assignment
Tuesday 9/22 09:30: Exam 1 Review: We'll review the practice exams on Compass
Wednesday 9/23: Extra Office Hours: Wednesday 9/23 10-11am and 6-7pm, for last-minute exam questions.
Thursday 9/24: Midterm 1, 09:30-11:00 AM: The midterm exam will be a timed, open-book, open-notes, open-internet exam, held on Compass. It will appear in your Compass folder at 9:15AM on Thursday 9/24, and will be available until 11:45AM; you may choose any 90-minute period during that time in which to take the exam. You may type your answers in any mixture of plaintext pseudo-math or pseudo-python syntax; as long as we can understand what you mean, you will get the points. Your answer should contain no integrals or infinite-length sums, but otherwise, you do not need to simplify explicit numerical expressions. Examples are available in the two sample exams that are currently available on Compass; you may also find it useful to look at exams from past semesters, though they are in a different format. Reference solutions to all practice exam problems are available on Compass after you submit your answers. Piazza will be open, during the real exam, for private questions to the instructors.

Week 6

Lecture 9, T9/29 09:30: Convolutional Neural Networks
Lecture 10, R10/1 09:30: Faster-RCNN Object Detectors

Week 7

Homework 3, M10/5 23:59: PDF: Back-propagation, loss functions, and nonlinearities. Solutions.
Lecture 11, T10/6 09:30: Discriminative vs. Bayesian Classifiers
Lecture 12, R10/8 09:30: Hidden Markov Models

Week 8

Machine Problem 3, M10/12 23:59: Face detection using a Faster-RCNN object detector
Lecture 13, T10/13 09:30: How to train Observation Probability Densities
Lecture 14, R10/15 09:30: Log Viterbi and Scaled Forward Algorithms

Week 9

Homework 4, M10/19 23:59: Hidden Markov models and Baum-Welch: assignment, solutions
Lecture 15, T10/20 09:30: Weighted Finite State Acceptors
Lecture 16, R10/22 09:30: Weighted Finite State Transducers

Week 10

Machine Problem 4, M10/26 23:59: Assignment page: Speech segmentation using Hidden Markov models
Midterm Review, T10/27 09:30: See sample exams on Compass.
Wednesday 10/28: Extra Office Hours: Wednesday 10/28 11am-noon and 6-7pm, for last-minute exam questions.
Midterm 2, R10/29 09:30: The midterm exam will be a timed, open-book, open-notes, open-internet exam, held on Compass. It will appear in your Compass folder at 9:15AM on Thursday 9/24, and will be available until 12:15PM; you may choose any 90-minute period during that time in which to take the exam. You may type your answers in any mixture of plaintext pseudo-math or pseudo-python syntax; as long as we can understand what you mean, you will get the points. Your answer should contain no integrals or infinite-length sums, but otherwise, you do not need to simplify explicit numerical expressions. Examples are available in the two sample exams that are currently available on Compass; you may also find it useful to look at exams from past semesters, though they are in a different format. Reference solutions to all practice exam problems are available on Compass after you submit your answers. Piazza will be open, during the real exam, for private questions to the instructors.

Week 11

T 11/3: No lecture
R 11/5: Lecture 17: Practical WFSTs

Week 12

Homework 5, W11/11 23:59: Weighted Finite State Transducers: PDF, solutions
T 11/10: Lecture 18: Recurrent Neural Networks
R 11/11: Lecture 19: Long/Short-Term Memory

Week 13

Machine Problem 5, W11/18 23:59: Continuous speech recognition using WFSTs (assignment page)
T 11/17: Lecture 20: Moving Pixels Around
R 11/19: Lecture 21: Barycentric Coordinates

Week 14

T 12/1: Lecture 22: Variational Autoencoders
Homework 6, W12/2 23:59: Gated and recurrent neural networks: PDF, solutions.
R 12/3: Lecture 23: Generative Adversarial Networks. PDF, Examples

Week 15

Machine Problem 6, W12/9 23:59: Video synthesis using LSTMs: Assignment page

Week 16

Final Exam: Friday, December 18, 13:30-16:30. If you need a conflict exam because of time zone, or because of a schedule conflict, please e-mail the instructor at least one week prior.; Sample exam is currently available on Compass.; The actual final exam will be available from Friday 12/18 at 13:00 until Friday 12/18 at 19:00; you may choose any 180-minute period within that span for your exam.