University of Illinois Urbana-Champaign · Fall 2026

ECE 490Introduction to Optimization

A proof-oriented introduction to continuous optimization for seniors/first year graduate students. Mathematical foundations of unconstrained and constrained optimization, convexity vs nonconvexity, analysis of large-scale modern algorithms.

Instructor
Richard Y. Zhangryz@illinois.edu
Teaching Assistant
Hong-Ming Chiuhmchiu2@illinois.edu
Class
ECEB 3081, Tue and Thu, 11:00 AM–12:20 PM
Office Hours
CSL 115, Thu, 12:30–1:30 PM, or by appointment
Short URL
ryz.nz/490

Course

Course introduction

Tue Aug 25
Motivation and Formulation: Why optimization? Why take a proof-based class? Finding vs certifying. Basic problem classes; decision variables, objectives, and constraints. Formulation vs solution. Worked examples: formulate least-squares problems from verbal descriptions.

Foundational Overview

Thu Aug 27
Algorithmic Overview: How to find a good solution? Implement GD, SGD, momentum, and Newton's method. For constraints, quadratic penalties and log barriers. Worked examples: optimizer updates, one Newton step, penalty and barrier formulations.Blank
Tue Sep 1
Convex Functions: How to certify a good solution? Convexity by definition and by the Hessian test. Basic combination rules. Worked examples: prove and disprove convexity using both criteria.
Thu Sep 3
Convex Sets: What about constraints? Convex sets by line segments and sublevel sets. Intersections and affine transformations. Projected gradient descent. Worked examples: prove and disprove set convexity; perform one projected-gradient step.
Tue Sep 8Quiz 1: Foundational overviewHomework

Gradient Descent and First-order Optimality

Thu Sep 10
Optimality conditions and gradient descent
Tue Sep 15
Fixed-step gradient descent
Thu Sep 17
Line search and convergence
Tue Sep 22Quiz 2

Newton's Method and Superlinear Convergence

Thu Sep 24
Preconditioning and Newton's method
Tue Sep 29
Globalizing Newton
Thu Oct 1
Achieving second-order optimality
Tue Oct 6Quiz 3

Midterm

Thu Oct 8Midterm Review
Tue Oct 13Midterm

Duality and Infeasibility Certification

Thu Oct 15
Tue Oct 20
Thu Oct 22
Tue Oct 27Quiz 4: Duality and infeasibility certification

Local Optimality under Constraints

Thu Oct 29
Tue Nov 3
Thu Nov 5
Tue Nov 10Quiz 5: Local optimality under constraints

Constrained and Nonsmooth Algorithms

Thu Nov 12
Tue Nov 17
Thu Nov 19
Tue Nov 24Fall Break — No Class
Thu Nov 26Fall Break — No Class
Tue Dec 1Quiz 6: Constrained and nonsmooth algorithms

Final Review

Thu Dec 3Cumulative Review I
Tue Dec 8Cumulative Review II
Thu Dec 10Reading Day — No Class

Grading, Homework, Quiz

  • 0% - Biweekly homework
  • 50% - Biweekly quizzes (top 5 out of 6 scores)
  • 25% - Midterm exam
  • 25% - Final exam

Each quiz contains 4 problems sampled from homework, only 3 of which must be completed.

Quizzes last 50 minutes, and are held in class approximately biweekly on Tuesdays. The first 30 minutes of class before quiz will be used for pre-quiz review.

Lowest quiz score is dropped to accommodate travel, etc. It is in your interest to attend all quizzes due to make-up policy below.

Three-credit vs four-credit sections

Many HW problems have parts marked (G). These are optional extensions for 3-credit students and required material for 4-credit students.

All quizzes and exams will have one part that is taken from (G).

  • 3 credit student may ignore this part and earn 100%, or attempt it to earn bonus. Scores will be capped at 100%.
  • 4 credit student must attempt this part to earn 100%. Their lowest-scoring other part will be dropped, but the (G) part will never be dropped.

Example quiz

Problem 3-credit section 4-credit section
P1 Required Eligible for drop
P2 Required Eligible for drop
P3 Required Eligible for drop
P4 (G) Optional bonus Required

Let P1, P2, P3, P4 denote the normalized score from 0 to 1.

  • 3-credit score: min( P1*34 + P2*33 + P3*33, 100)
  • 4-credit score: max( P1, P2)*67 + P3*33

Missed assessments

Exceptional circumstances may necessitate make-up quiz or exam. The reasoning must be consistent with the student code.

Makeup quiz requests will only be considered after a student has been absent for one quiz. If a student has not been absent for a quiz, then the makeup request will be automatically declined.

The student must contact course staff with timestamp no later than 7 days (168 hours, 0 minutes, 0 seconds) before the starting time of the quiz or exam to inquire about make-up assessments.

AI policy

AI tools are strongly encouraged for studying and for working through the ungraded homework. Useful ways to use AI include:

  • asking for a different explanation of a theorem or proof;
  • requesting a hint when stuck;
  • asking AI to identify a gap in an argument;
  • checking algebra or intermediate calculations;
  • asking why a proposed proof is invalid;
  • generating additional problems equivalent to a homework problem;
  • asking for a complete solution and then working backward until every step can be reproduced independently.

Illinois students have access to Google Gemini through their University Google accounts. Illinois Google Gemini information

AI systems can make mathematical mistakes. Students remain responsible for checking AI output.

AI may not be used during any quiz or examination.

University policies

Disability-related accommodations

The University provides reasonable academic accommodations to students with disabilities through Disability Resources and Educational Services (DRES). Students seeking accommodations should register with DRES and provide their Letter of Academic Accommodations to the instructor as early as possible.

Approved accommodations will be implemented in accordance with Student Code § 1-110. Student Code § 1-110 — Disability accommodations

Religious observances

The University will reasonably accommodate religious beliefs, observances, and practices that conflict with class attendance, quizzes, examinations, or other course requirements.

Students should request accommodation sufficiently far in advance to allow alternate arrangements to be made. See Student Code § 1-107. Student Code § 1-107 — Religious accommodations

Emergency response

Students should familiarize themselves with University emergency procedures and with the evacuation and shelter procedures for ECEB. Instructions from University and emergency personnel should be followed during an emergency.

Other University policies

All applicable University of Illinois policies remain in effect, including policies concerning student privacy, nondiscrimination, community standards, sexual misconduct, student well-being, and military or veteran obligations. Students should consult the current University of Illinois Student Code for the governing policies and procedures.