IE 521 (Convex Optimization)  Fall 2023  Schedule
Course schedule
Course notes can be downloaded here:
Dong, 'IE 521’. This is a workinprogress so please expect to redownload this file a few times throughout the semester. In the readings, it will be listed as IE 521.
Dong, 'How to math’. This is a helper document which outlines some of the 'rules’ of mathematical reasoning which is needed for HW1.
Other readings will be taken from these textbooks:
BV: Boyd and Vandenberghe, 'Convex Optimization’.
Bertsekas, Nedich, and Ozdaglar, 'Convex Analysis and Optimization’.
BenTal and Nemirovski, 'Lectures on Optimization III: Convex Analysis, Nonlinear Programming Theory, Nonlinear Programming Algorithms’.
BenTal and Nemirovski, 'Lectures on Modern Convex Optimization: analysis, algorithms, and engineering applications’.
Nesterov, 'Lectures on Convex Optimization’.
HiriantUrruty and Lemarechal, 'Fundamentals of Convex Analysis’.
R: Rockafellar, 'Convex Analysis’.
Bauschke and Combettes, 'Convex Analysis and Monotone Operator Theory in Hilbert Spaces’
These books are not strictly about convex optimization but have certain sections that I think are a great resource:
Date  Topics covered  Related readings 
Week 1   
Lecture 1  Course overview, introduction to optimization  Course syllabus, IE 521 Sect. 23, How to math document, BV Ch. 1 
Lecture 2 [PDF]  Review of set theory, real analysis, and linear algebra  IE 521 Sect. 46, L Appendix A 
Week 2   
Lecture 3 [PDF]  Review of optimization, convex sets  IE 521 Sect. 78, BV Ch. 2.12.4 
Lecture 4 [PDF]  Convex functions  IE 521 Sect. 9, BV Ch. 2.5 
Week 3   
 No class due to Labor Day  
Lecture 5 [PDF]  Separating and supporting hyperplanes, convex conjugates  BV Ch. 3.13.3, R Sect. 1112 
Week 4   
Lecture 6 [PDF]  Separating hyperplane theorem  
Lecture 7 [PDF]  Cutting planes methods, tropical semirings  Boyd and Vandenberghe's cutting planes notes, My notes on Fenchel and Fourier duality (optional) 
Week 5   
Lecture 8 [PDF]  Fenchel inequality, proximal mappings  R Sect. 13, 23 
Lecture 9 [PDF]  Fenchel duality  R Sect. 31 
Week 6   
Lecture 10 [PDF]  Standard form optimization, examples of convex problems and applications  BV Ch. 68 
Lecture 11 [PDF]  Fenchel duality theorem, KuhnTucker conditions, Moreau envelope, gradient descent  BV Ch. 9, Karimi, Nutini, Schmidt 2016, Wilson's Optimization crash course: (pt. 1) (pt. 2 (optional)) 
Week 7   
Lecture 12 [PDF]  Gradient descent (continued), PL inequality, Newton's method, interior point methods, Convergence proofs  BV Ch. 11 
Lecture 13 [PDF]  Lagrangian duality  BV Ch. 5 
Week 8   
Lecture 14 [PDF]  Constraint qualification, Slater's condition  
Lecture 15 [PDF]  KarushKuhnTucker optimality conditions  
Week 9   
Lecture 16 [PDF]  Convex bifunctions, adjoint bifunctions  R Sect. 2930 
Lecture 17 [PDF]  Convex bifunctions, adjoint bifunctions (cont.)  
Week 10   
Lecture 18 [PDF]  Dual ascent, envelope theorem, dual decomposition, augmented Lagrangian, method of multipliers  Boyd, Parikh, Chu, Peleato, Eckstein, 'Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers’ 
Lecture 19 [PDF]  Alternating direction method of multipliers (ADMM)  
Week 11   
Lecture 20  Bregman divergences, mirror descent  Orabona, 'A Modern Introduction to Online Learning’ Ch. 6, Bubeck, 'Introduction to Online Optimization’, Ch. 5 (optional), (my messy lecture notes) 
Lecture 21  Bregman divergences, mirror descent (cont.)  
Week 12   
Lecture 22  Dual spaces  (my messy lecture notes) 
Lecture 23  Optimal transport  Villani, 'Topics in Optimal Transportation’, Ch. 1.1, 2.1, (my messy lecture notes) 
Week 13   
Lecture 24  Optimal transport (continued)  
 Midterm exam  
Week 14   
 Thanksgiving break  
 Thanksgiving break  
Week 15   
Lecture 25  Closing remarks  
 Final presentations  
Week 16   
 Final presentations  
 Final presentations 

Homeworks
Please see Gradescope for due dates.
Homework 0
Homework 1 (TeX source)
Homework 2 (TeX source)
Homework 3 (Jupyter Notebook) (.ipynb file)
Homework 4 (TeX source)
Homework 5 (Jupyter Notebook) (.ipynb file)
Homework 6 (TeX source)
Homework 7 (TeX source)
Homework 8 (Jupyter Notebook) (.ipynb file)
Data for HW 8: mnist_train.csv mnist_test.csv
Solutions:
Homework 1 Solutions
Homework 2 Solutions
Homework 3 Solutions
Homework 4 Solutions
Homework 5 Solutions
Homework 6 Solutions
Homework 7 Solutions
Homework 8 Solutions
Midterm Exam
The midterm exam will take place inclass on Wednesday, November 15th. It will cover all material prior to the exam date.
Final Project Presentations
As mentioned in the syllabus, you are expected to give a 10minute presentation. Final project presentations will take place the last 23 days of the semester, depending on the steadystate enrollment. Also as mentioned in the syllabus: you are expected to attend the presentations of your peers.
Final Reports
Final reports will be due Friday, December 15th at 5pm CT.
