Reading List: CS598KN-ReadingList
Course Details: Slides-About-Course-Details
Goal: Multimedia data and underlying systems and networks that service multimedia (multi-modal sensory) data are becoming ubiquitous. In the "Advanced Multimedia Systems" class we will explore major advances that have been made in multimedia data, systems and networks over the last 5-10 years to enable next generation multimedia applications such as YouTube, Video-360, AR/VR systems and others. We will take the end-to-end approach and explore an integrated view of multimedia systems ranging from 2D and 3D video and audio, advanced compression techniques H.264/H.265, MPEG4 and MPEG-7, new multimedia transport protocols such as DASH, Adaptive Bitrate Control, Quality of Service (QoS) preservation for mobile multimedia, Content Distribution, to multi-modal synchronization, Video 360 oncepts, Mulsemedia concepts, and subjective and objective Quality of Experience (QoE) evaluation methods for next generation multimedia applications.
New this semester: We will learn about new advances, metrics, protocols and algorithms in recent conferences regarding Video 360/AR/VR systems, Machine Learning in Multimedia Systems, Mulsemedia, and Quality of Experience evaluation methodology in multi-modal distributed systems, networks, and applications, as well as new developments in video/audio compression, multimedia communication, distribution and adaptive streaming.
Expected Workload: The students will present 1 research paper (or part of it if two presenters per paper), formulate (prior to class) and answer (after the class) research questions to a 2nd research paper from the Reading List, review three research paper (other than the presented paper and the question paper), work on a take-home midterm exam, and participate in zoom-discussion and online of presented papers. Furthermore, students will work on a semester-long project and present their results in a research paper and final presentation. Students can define their own project. Sample projects: design and development of multi-view and/or Video 360 video distribution algorithms, design and validation of QoE evaluation methodology for diverse mobile multimedia applications; development of DASH Adaptive control algorithms, QoS analytics algorithms and tool(s) for multimedia systems, caching of video in mobile environments, and others.
Topics: Basic Concepts of audio and video (2D, 3D, volumentric video, video-360), MPEG, HEVC, and H.265 compression algorithms, QoS and resource management algorithms, streaming protocols, feedback control and adaptation algorithms for A/V streaming, peer-to-peer streaming and content-distribution networks, DASH, SIP, WebRTC, multi-modal synchronization algorithms, machine learning and analytics in video systems, QoE evaluation methods, and design and integration principles for multimedia systems such as mobile multimedia, Video 360 streaming, VR, surround sound, Voice-over-IP, Video Conferencing, and Video-on-Demand.
Lecture Slides, Paper Reviews and Class Discussion are available at https://piazza.com/illinois/fall2020/cs598kn
Note for Illinois students: If you are not registered for the class and would like to attend the first few classes to make the final decision about your courses' registration, please, contact the instructor or the TA to share the zoom link with you.