ECE 563 - Information Theory (Fall 2021)

Department of Electrical and Computer Engineering
University of Illinois at Urbana-Champaign
Lectures: Tuesday and Thursday, 12:30PM - 1:50PM, ECEB 3081 (lecture videos available on echo360)
Instructor: Ilan Shomorony, ilans@illinois.edu. Office hours (via Zoom): Monday 10:30am-12:00pm (info)

TA: Seiyun Shin, seiyuns2@illinois.edu. Office hours (via Zoom): Wednesday 4:00pm-5:00pm (info)
Problem solving sessions: Biweekly on Friday 4:00pm-5:00pm via Zoom (info)
Recorded videos of the previous problem-solving sessions can be found here


Prerequisites: Solid background in probability (ECE 534, MATH 464, or MATH 564).
Textbook: T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed., Wiley, 2006.

Announcements

• Solutions to Final exam can be found here (mean: 69.5, std: 17.7)
• Final project blog posts can be found here
• Final exam information:
• Time: Friday, December 10, 7pm-10pm (via Zoom)
• You will be allowed three 8.5 x 11” sheets of handwritten notes. No calculators or other electronic devices will be allowed.
• The exam will cover all the course material.
• Solutions to Midterm 2 can be checked here
• No lecture on Tuesday, November 9.
• Information on the final project can be found here.
• Midterm 2 information:
• Time: Tuesday, November 9, 7pm-8:50pm (via Zoom)
• You will be allowed one 8.5 x 11” sheet of handwritten notes. No calculators or other electronic devices will be allowed.
• The exam will cover everything since Data Compression.
• No Office Hours on Monday, October 25.
• Solutions to Midterm 1 can be checked here
• There will be no lecture on Tuesday, October 5.
• Midterm 1 information:
• Time: Tuesday, October 5, 7pm-8:50pm (via Zoom)
• You will be allowed one 8.5 x 11” sheet of handwritten notes. No calculators or other electronic devices will be allowed.
• The exam will cover everything until Gaussian channels.
• First problem solving session will be on Friday, 09/17.
• You can access the class notes here.
• Lecture videos will be available at echo360.

Main topics

• Information measures and fundamental inequalities
• Typicality
• Channel Capacity and the Channel Coding Theorem
• Gaussian channels
• Lossless compression
  
• Huffman coding
  
• Universal compression of i.i.d. and Markov sources
  
• Lempel-Ziv coding
• Rate-distortion theory
• Large Deviation Theory and Hypothesis Testing
• Topics in multi-user information theory
  

Grading

• Homework 15%
• Exam 1 - 20% (10/05 7pm)
• Exam 2 - 20% (11/09 7pm)
• Project - 10%
• Final Exam - 35% (12/10 7pm-10pm)