CS 598: Deep Generative and Dynamical Models

Fall, 2021

 

 

Course Description

Recent years have seen considerable advances in generative models, which learn distributions from data and also generate new data instances from the learned distribution; and dynamical models, which model systems with a dynamical or temporal component. Both of these developments have been leveraging advances in deep learning. The course will cover key advances in generative and dynamical models, including variational auto-encoders, normalizing flows, generative adversarial networks, neural differential equations, physics guided machine learning, among other topics.

 

Basic Information

Classes: Tue, Thu 02:00 pm – 03:15 pm

Location: Zoom

Instructor: Arindam Banerjee, arindamb@illinois.edu

TA: Anant Dadu, adadu2@illinois.edu

Office hours:

·      Arindam Banerjee: Tue, Thu 5:00 – 6:00 pm (zoom)

·      Anant Dadu:  

Online resources:

·      Canvas

 

Schedule (Tentative, subject to mild adjustments)

·      Course Outline and Background (week 1)

·      Introduction to Generative Models (week 2)

·      Auto-regressive Models (weeks 3, 4)

·      Variational Auto-encoders (weeks 4, 5, 6)

·      Normalizing Flows (weeks 6, 7, 8)

·      Generative Adversarial Networks (weeks 8, 9, 10)

·      Neural Differential Equations (weeks 11, 12)

·      Score-based Models (weeks 12, 13)

·      Learning Operators, Physics guided ML (weeks 13, 14)

·      Project Presentations (weeks 14, 15)

 

Coursework and Grading

·      Paper Presentations:

o   20%: Presentation for 2 classes (of 23 classes), each class with 2 (or 3) presenters [noon, before class]

o   10%: Blog post for 2 classes, corresponding to presentations, ~10 min read [week after presentation]

·      Paper Reviews

o   30%: Reviews for 10 classes, excluding the 2 presentations, 4 pages core material (including figures/results) + references + appendix [day before presentation]

·      Project – in teams of 2 (or 3)

o   Proposal: 5%, 2 pages + references

o   Mid-term report: 10%, 4 pages + references + appendices

o   Final report: 20%, 8 pages + references + appendices

o   Presentation: 5%, 10-12 mins, in class

 

Key Dates

·      Presentation slides: noon, before class

·      Blog posts: 7 days after presentation

·      Paper reviews: End-of-day before presentation

·      Project

o   Team: Fri, Sept 17

o   Proposals due: Fri, October 1

o   Mid-term report due: Fri, Nov 5

o   Final report due: Wed, Dec 8

 

Grades

 

q Following cutoffs represent what will likely be used to generate letter grades:   

   

     A+   >= 95%                     A  >=90% & < 95%      A-  >=85% & < 90%      

     B+   >= 80% & < 85%      B  >=75% & < 80%      B- >=70% & < 75%      

     C+   >= 65% & < 70%      C  >=60% & < 75%      C- >=55% & <60%             

     D  >= 50%& <55%           F < 50% 

 

q The above cutoffs are tentative and may be adjusted slightly; if there is any adjustment to the above cutoffs, we will NOT curve down your letter grades

q However, there will be no general curve-fitting in assigning the final grades

 

 

Textbooks

No textbook, the material will be based on papers posted on the class webpage.

 

 

Mental Health

Diminished mental health, including significant stress, mood changes, excessive worry, substance/alcohol abuse, or problems with eating and/or sleeping can interfere with optimal academic performance, social development, and emotional wellbeing. The University of Illinois offers a variety of confidential services including individual and group counseling, crisis intervention, psychiatric services, and specialized screenings at no additional cost. If you or someone you know experiences any of the above mental health concerns, it is strongly encouraged to contact or visit any of the University’s resources provided below. Getting help is a smart and courageous thing to do -- for yourself and for those who care about you.

 

Counseling Center: 217-333-3704, 610 East John Street Champaign, IL 61820

 

McKinley Health Center:217-333-2700, 1109 South Lincoln Avenue, Urbana, Illinois 61801

 

Sexual Misconduct Reporting Obligation

 

The University of Illinois is committed to combating sexual misconduct. Faculty and staff members are required to report any instances of sexual misconduct to the University’s Title IX Office. In turn, an individual with the Title IX Office will provide information about rights and options, including accommodations, support services, the campus disciplinary process, and law enforcement options.

 

A list of the designated University employees who, as counselors, confidential advisors, and medical professionals, do not have this reporting responsibility and can maintain confidentiality, can be found here: http://wecare.illinois.edu/resources/students/#confidential

 

Other information about resources and reporting is available here: http://wecare.illinois.edu

 

Academic Integrity

The University of Illinois at Urbana-Champaign Student Code should also be considered as a part of this syllabus. Students should pay particular attention to Article 1, Part 4: Academic Integrity. Read the Code at the following URL: http://studentcode.illinois.edu/

 

Academic dishonesty may result in a failing grade. Every student is expected to review and abide by the Academic Integrity Policy: https://studentcode.illinois.edu/article1/part4/1-401/. Ignorance is not an excuse for any academic dishonesty. It is your responsibility to read this policy to avoid any misunderstanding. Do not hesitate to ask the instructor(s) if you are ever in doubt about what constitutes plagiarism, cheating, or any other breach of academic integrity.

 

Religious Observances

Illinois law requires the University to reasonably accommodate its students' religious beliefs, observances, and practices in regard to admissions, class attendance, and the scheduling of examinations and work requirements. You should examine this syllabus at the beginning of the semester for potential conflicts between course deadlines and any of your religious observances. If a conflict exists, you should notify your instructor of the conflict and follow the procedure at

https://odos.illinois.edu/community-ofcare/resources/students/religious-observances/

to request appropriate accommodations. This should be done in the first two weeks of classes.

 

Disability-Related Accommodations

To obtain disability-related academic adjustments and/or auxiliary aids, students with disabilities must contact the course instructor and the Disability Resources and Educational Services (DRES) as soon as possible. To contact DRES, you may visit 1207 S. Oak St., Champaign, call 333-4603, email disability@illinois.edu or go to https://www.disability.illinois.edu. If you are concerned you have a disability-related condition that is impacting your academic progress, there are academic screening appointments available that can help diagnosis a previously undiagnosed disability. You may access these by visiting the DRES website and selecting “Request an Academic Screening” at the bottom of the page.

 

Family Educational Rights and Privacy Act (FERPA)

Any student who has suppressed their directory information pursuant to Family Educational Rights and Privacy Act (FERPA) should self-identify to the instructor to ensure protection of the privacy of their attendance in this course. See https://registrar.illinois.edu/academic-records/ferpa/ for more information on FERPA.