ECE 563 - Information Theory (Fall 2023)

Department of Electrical and Computer Engineering
University of Illinois at Urbana-Champaign
Lectures: Tuesday and Thursday, 12:30PM - 1:50PM, ECEB 3081 (lecture videos available on mediaspace.illinois.edu)
Instructor: Ilan Shomorony, ilans@illinois.edu. Office hours: Monday 1:00pm-2:00pm, 313 CSL

TA: Aditya Narayan Ravi, anravi2@illinois.edu. Office hours: Wednesday 10:30AM-11:30AM, ECEB 3036
Problem solving sessions: Biweekly on Friday 3:30pm-4:30pm via Zoom (info)
Recorded videos of the previous problem-solving sessions can be found here.


Prerequisites: Solid background in probability (ECE 534, MATH 464, or MATH 564).
Textbook: T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed., Wiley, 2006.

Announcements

• Final exam information:
• Time: Tuesday, December 12, 8am-11am - 3081 ECEB
• You will be allowed three 8.5 x 11 sheets of handwritten notes. No calculators or other electronic devices will be allowed.
• The exam will cover all the course material.
• Solutions to Midterm 2 can be checked here
• Midterm 2 information:
• Time: Tuesday, November 7, 7pm-9:00pm, 3017 ECEB
• You will be allowed to bring in the sheet of notes from Midterm 1, plus one additional sheet. No calculators or other electronic devices will be allowed.
• The exam will cover everything in data compression (lectures 14 to 19).
• Solutions to Midterm 1 can be found here
• Midterm 1 information:
• Time: Tuesday, October 3, 7pm-9:00pm, 3017 ECEB
• You will be allowed one sheet of handwritten notes. No calculators or other electronic devices will be allowed.
• The exam will cover everything until (and including) lecture 11.
• First problem solving session will be on Friday, 09/08.

Main topics

• Information measures and fundamental inequalities
• Typicality
• Channel Capacity and the Channel Coding Theorem
• Gaussian channels
• Lossless compression
  
• Huffman coding
  
• Universal compression of i.i.d. and Markov sources
  
• Lempel-Ziv coding
• Rate-distortion theory
• Large Deviation Theory and Hypothesis Testing
• Topics in multi-user information theory
  

Grading

• Homework 15%
• Exam 1 (Oct 3) - 20%
• Exam 2 (Nov 7) - 20%
• Project - 10%
• Final Exam (Dec 12, 8am-11am) - 35%