A first course in optimization theory.

**Instructor:**Elad Yarkony- yarkony2 [at] illinois [dot] edu
- Office hours: Tuesday 2pm–4pm, 167 CSL

**Teaching assistant:**Haoxiang Wang- hwang264 [at] illinois [dot] edu
- Office hours: Wed 1-3pm, ECEB3013

**Meetings:**Tuesday and Thursday, 11:00–12:20pm, 2017 ECEB**Schedule:***Midterm:*3/12/20 in class.*Final:*TBD

*1/21/20*: Welcome Students !*1/31/20*: Homework 1 released. Due 2/13 8PM CDT. Submit through Compass2g.*2/3/20*: TA Office Hours announced.

*2/3/20*: TA Office Hours of 2/5/20 moved to Mon. 2/10/20 ECEB4034, same hours*2/5/20*: Instructor’s additional office hours on Fri. 2/7/20 3-4pm, ECEB lobby- 2/20/20: Homework 2 released. Due 3/1 8PM CDT. Submit through Compass2g.
- 3/3/20: Homework 3 released. Due 3/10 8PM CDT. Submit through Compass2g.
- 3/3/20: Midterm 2 will be held on Mar 12th, regular class hours, and will cover HW1-HW3.

- Week 1 (1/20–1/24):
- Lecture 1: Introduction PDF
- Lecture 2: Mathematical Refresher PDF Jupyter notebook
- Reading assignment: Boyd, Appendix A

- Week 2 (1/27–1/31):
- Lecture 3: Positive Semi-Definite Matrices, Multivariate Calculus Refresher PDF Jupyter notebook
- Lecture 4: Affine Sets, Convex Sets, Cones PDF Jupyter notebook
- Homework 1 released. Due 2/13 8PM CDT. Submit through Compass2g.

- Week 3 (2\3–2/7):
- Week 4 (2\10–2/14):
- Lecture 7: Convex Cones Wrap up, Convex Functions PDF Jupyter notebook
- Lecture 8: First and Second Order Convexity Conditions, Minima of Convex Functions, Jensen’s Inequality, Convexity Preserving Operations PDF Jupyter notebook
- Reading assignment: Boyd, Chapter 3.1-3.2

- Week 5 (2/17-2/21):
- Lecture 9: Convex-preserving operations, Quasi-Convex and Log-Convex Functions PDF Jupyter notebook
- Lecture 10: Vector/Matrix-valued Convex Functions, Convex Conjugates PDF
- Reading assignment: Boyd, Chapter 3.3-3.6
- Homework 2 released. Due 3/1 8PM CDT. Submit through Compass2g.

- Week 6 (2/24-2/28):
- Lecture 11: Optimization Problems PDF, Numerical Conjugation Demo Jupyter notebook
- Lecture 12: First Order Optimality Condition for Convex Problems PDF, Jupyter notebook, Interactive Gradient Field (Python)
- Reading assignment: Boyd, Chapter 4.1-4.2

- Week 7 (3/2-3/6):
- Lecture 13: Problem Equivalence, Linear Programs PDF, Chebyshev Center Demo Jupyter notebook
- Homework 3 released. Due 3/10 8PM CDT. Submit through Compass2g.
- Lecture 14: Estimator Bounds, Quadratic Programming, Second Order Cone Programming, Geometric Programming PDF, Jupyter notebook,
- Reading assignment: Boyd, Chapter 4.3-4.5

- Week 8 (3/9-3/13):
- Midterm practice: homework review + the midterms in this link should be useful (note that we did not cover disciplined convex programming, so ignore everything DCP-related)
- Reading assignment: Boyd, Chapter 4.6-4.7

- Week 9 (3/23-3/27):
- Lecture 15: Midterm review, Lagrange Multipliers and Lagrangian Function PDF
- Lecture 16: Dual Problem, Dual Function PDF, Jupyter notebook,
- Reading assignment: Boyd, Chapter 5.1-5.4
- Homework 4 released. Due 4/7 8PM CDT. Submit through Compass2g.

- Week 10 (3/29-4/3):

- Introduction
- Mathematical Background (BV, Appendix A)
- Convex Sets, Cones (BV, Chapter 2)
- Convex Functions, Optimality Conditions (BV Chapter 3)

- Anatomy of Optimization Problems
- Optimization Problems Prototypes (BV, Chapter 4)
- Lagrange Duality (BV, Chapter 5)

- Common Applications
- Data Fitting, Approximation (BV, Chapter 6)
- Estimation (BV, Chapter 7)
- Geometric Problems (BV, Chapter 8)

- Standard Algorithms
- Unconstrained Optimization (BV, Chapter 9)
- Equality constraints (BV, Chapter 10)
- Inequalities (interior point methods, barrier methods) (BV, Chapter 11)

- 40% homeworks (submission instructions)
- 25% midterm

- 35% final exam
- 4 credits: final project (binary)
- Grades will be distributed on Compass 2g

- Stephen Boyd and Lieven Vandenberghe,
*“Convex Optimization”*, book (online version) - Osman Güler,
*“Foundations of Optimization”*, available as PDF through UIUC online library catalog (click “SpringerLink - Full text online” at the bottom of the page)

- Python/NumPy tutorials literally all over such as this beautiful lecture
- Plenty of Jupyter references such as this and this
- Download Python 3 here

- Elementary set theory e.g. \(A\subset B,x\in A\).
- Vector spaces
- Multivariate Calculus
- Partial Derivative, Gradient, Hessian
- Convex Function and Convex Set