IE 521 (Convex Optimization) - Fall 2023 - Schedule

Course schedule

Course notes can be downloaded here:

  • Dong, 'IE 521’. This is a work-in-progress so please expect to re-download this file a few times throughout the semester. In the readings, it will be listed as IE 521.

  • Dong, 'How to math’. This is a helper document which outlines some of the 'rules’ of mathematical reasoning which is needed for HW1.

Other readings will be taken from these textbooks:

  • BV: Boyd and Vandenberghe, 'Convex Optimization’.

  • Bertsekas, Nedich, and Ozdaglar, 'Convex Analysis and Optimization’.

  • Ben-Tal and Nemirovski, 'Lectures on Optimization III: Convex Analysis, Nonlinear Programming Theory, Nonlinear Programming Algorithms’.

  • Ben-Tal and Nemirovski, 'Lectures on Modern Convex Optimization: analysis, algorithms, and engineering applications’.

  • Nesterov, 'Lectures on Convex Optimization’.

  • Hiriant-Urruty and Lemarechal, 'Fundamentals of Convex Analysis’.

  • R: Rockafellar, 'Convex Analysis’.

  • Bauschke and Combettes, 'Convex Analysis and Monotone Operator Theory in Hilbert Spaces’

These books are not strictly about convex optimization but have certain sections that I think are a great resource:

  • L: Lee, 'Introduction to Topological Manifolds’. (I like Appendix A a lot as a review of basic mathematical concepts/notation.)

Date Topics covered Related readings
Week 1
Lecture 1 Course overview, introduction to optimization Course syllabus, IE 521 Sect. 2-3, How to math document, BV Ch. 1
Lecture 2 [PDF] Review of set theory, real analysis, and linear algebra IE 521 Sect. 4-6, L Appendix A
Week 2
Lecture 3 [PDF] Review of optimization, convex sets IE 521 Sect. 7-8, BV Ch. 2.1-2.4
Lecture 4 [PDF] Convex functions IE 521 Sect. 9, BV Ch. 2.5
Week 3
No class due to Labor Day
Lecture 5 [PDF] Separating and supporting hyperplanes, convex conjugates BV Ch. 3.1-3.3, R Sect. 11-12
Week 4
Lecture 6 [PDF] Separating hyperplane theorem
Lecture 7 [PDF] Cutting planes methods, tropical semirings Boyd and Vandenberghe's cutting planes notes, My notes on Fenchel and Fourier duality (optional)
Week 5
Lecture 8 [PDF] Fenchel inequality, proximal mappings R Sect. 13, 23
Lecture 9 [PDF] Fenchel duality R Sect. 31
Week 6
Lecture 10 [PDF] Standard form optimization, examples of convex problems and applications BV Ch. 6-8
Lecture 11 [PDF] Fenchel duality theorem, Kuhn-Tucker conditions, Moreau envelope, gradient descent BV Ch. 9, Karimi, Nutini, Schmidt 2016, Wilson's Optimization crash course: (pt. 1) (pt. 2 (optional))
Week 7
Lecture 12 [PDF] Gradient descent (continued), PL inequality, Newton's method, interior point methods, Convergence proofs BV Ch. 11
Lecture 13 [PDF] Lagrangian duality BV Ch. 5
Week 8
Lecture 14 [PDF] Constraint qualification, Slater's condition
Lecture 15 [PDF] Karush-Kuhn-Tucker optimality conditions
Week 9
Lecture 16 [PDF] Convex bifunctions, adjoint bifunctions R Sect. 29-30
Lecture 17 [PDF] Convex bifunctions, adjoint bifunctions (cont.)
Week 10
Lecture 18 [PDF] Dual ascent, envelope theorem, dual decomposition, augmented Lagrangian, method of multipliers Boyd, Parikh, Chu, Peleato, Eckstein, 'Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers’
Lecture 19 [PDF] Alternating direction method of multipliers (ADMM)
Week 11
Lecture 20 Bregman divergences, mirror descent Orabona, 'A Modern Introduction to Online Learning’ Ch. 6, Bubeck, 'Introduction to Online Optimization’, Ch. 5 (optional), (my messy lecture notes)
Lecture 21 Bregman divergences, mirror descent (cont.)
Week 12
Lecture 22 Dual spaces (my messy lecture notes)
Lecture 23 Optimal transport Villani, 'Topics in Optimal Transportation’, Ch. 1.1, 2.1, (my messy lecture notes)
Week 13
Lecture 24 Optimal transport (continued)
Midterm exam
Week 14
Thanksgiving break
Thanksgiving break
Week 15
Lecture 25 Closing remarks
Final presentations
Week 16
Final presentations
Final presentations


Please see Gradescope for due dates.

Homework 0

Homework 1 (TeX source)

Homework 2 (TeX source)

Homework 3 (Jupyter Notebook) (.ipynb file)

Homework 4 (TeX source)

Homework 5 (Jupyter Notebook) (.ipynb file)

Homework 6 (TeX source)

Homework 7 (TeX source)

Homework 8 (Jupyter Notebook) (.ipynb file)

Data for HW 8: mnist_train.csv mnist_test.csv


Homework 1 Solutions

Homework 2 Solutions

Homework 3 Solutions

Homework 4 Solutions

Homework 5 Solutions

Homework 6 Solutions

Homework 7 Solutions

Homework 8 Solutions

Midterm Exam

The midterm exam will take place in-class on Wednesday, November 15th. It will cover all material prior to the exam date.

Final Project Presentations

As mentioned in the syllabus, you are expected to give a 10-minute presentation. Final project presentations will take place the last 2-3 days of the semester, depending on the steady-state enrollment. Also as mentioned in the syllabus: you are expected to attend the presentations of your peers.

Final Reports

Final reports will be due Friday, December 15th at 5pm CT.