Quick links

Gradescope: https://www.gradescope.com/courses/565145 (Entry Code: NX634V)

Media Space channel: Recorded lectures will be available at https://mediaspace.illinois.edu/channel/fa23-ie521

Piazza: https://piazza.com/illinois/fall2023/ie521

Times and places

• Course lecture: Mondays and Wednesdays at 1pm – 2:20pm in 218 Ceramics Building.

• Instructor office hours: Mondays and Wednesdays from 2:30pm – 3pm in 142 Coordinated Science Laboratory, or by appointment.

• TA office hours: Tuesdays from 2pm – 3pm in 202 Transportation Building.

Course folks

• Instructor: Roy Dong

  • email: roydong at illinois dot edu

• TA: Heling Zhang

  • email: hzhng120 at illinois dot edu

Course description

The goal of this course is to build both the foundations to understand modern convex optimization from both a theoretical and applied perspective. Students will be expected to be both capable of rigorous mathematical arguments as well as implementation and programming.

Topics covered include:

• fundamental concepts in optimization (e.g. optimality conditions, equivalence of optimizations)

• fundamental concepts in convex analysis (e.g. separating hyperplanes, Legendre-Fenchel conjugates)

• Lagrangian duality

• optimization methods and implementation (e.g. interior point, cutting planes, and operator splitting methods)

Course expectations

The expectations of the students in this course will be quite high. You will be expected to have an understanding of convex optimization from both a theoretical and applied perspective.

This course will be theoretical and rigorous, and you will be expected to give valid and detailed mathematical proofs. As this is a class, for pedagogical purposes we will expect more verbosity in proofs (e.g. explaining individual lines in derivations and what properties are invoked) than is typical of, say, a publication.

Additionally, some homework assignments will require programming. Programming assignments will be in Python. You are expected to set up your programming environment prior to the first programming homework.

If your mathematical foundations are a little rocky, it is expected that you will strengthen them through the course of this semester. If you haven't implemented algorithms before, you will be expected to learn enough about programming to be able to code up the algorithms covered in class.

You are encouraged to take this class even if your mathematical foundations are a bit rocky, or if you haven't had much experience implementing algorithms before. It's why I'm here as an instructor and you're supposed to learn from the course, not know everything prior to the course. However, if some of these more preliminary ideas outside the scope of the class are a bit foggy, please expect to put in a bit of extra time for ‘catch up’ and allocate accordingly. Some resources for self-learning will be provided, but if there's some aspects that are not familiar to you, feel free to reach out to me so I can try to find the appropriate resource to help fill in any gaps.

Overall, the goal of this course is to instill an intuition and understanding that will allow you to develop new algorithms, and not simply just implement existing ones. Furthermore, if you have a novel algorithm in hand, you are expected to have to tools to analyze this algorithm to see what properties it has.

Course structure

The course materials will be pulled from a variety of sources. I will post my own lecture notes, along with recommended readings for a more in-depth coverage of the concepts discussed in class. You are strongly recommended to look at the readings as well as my lectures for any concepts you are not already familiar with. Lectures and readings will be posted here: Course Schedule (which can also be accessed on the menu on the left of this webpage).

Assignment-wise, there will be 3 graded components: homeworks, a midterm exam, and a final project. There will not be a final exam.

Grading

The grades will be broken down as follows.

You may check your grades at any time on Gradescope (https://www.gradescope.com/), which we will be using as the gradebook for the class.

Homework

Homework assignments will be administered via Gradescope, i.e. HWs will be posted on Gradescope and submissions will be collected on Gradescope. The information to sign up for Gradescope is available at the top of this page under 'Quick Links’. Homework PDFs and TeX source will be posted on Course Schedule (which can also be accessed on the menu on the left of this webpage).

When you enroll, please use the following information.

• Email address: Use your illinois.edu email.

• Student ID #: Use your University Identification Number (UIN), which can be found on your ID card. This is NOT the same thing as your NetID. Failure to do so may cause your grade to not be correctly entered into the Registrar!

Homework due dates and time will be listed on the Gradescope assignment. No late homeworks will be accepted.

You may drop your lowest homework grade. This dropped homework is meant to account for any personal events or extenuating circumstances that may arise. No exceptions, extensions, or other variations will be given; if something happens that greatly interferes with your ability to complete assignments this semester, grading will be determined by the university's policy on such extenuating circumstances.

Allot yourself time to upload a PDF of your homework to the website. Failure to upload a homework assignment due to technical issues will also be absorbed by the one-dropped-homework policy.

Students are heavily, heavily encouraged to typeset their homeworks. LaTeX proficiency is an absolute requirement for many, many fields of research, and the homeworks for this course are a good place to begin building these skills. It will also make uploading PDFs of your homework to Gradescope much easier.

If you choose to hand-write homeworks instead, you may scan them. I recommend the Adobe Scan app (https://acrobat.adobe.com/us/en/mobile/scanner-app.html) to do so. Homeworks are required to be legible. This is a comment on both handwriting and scan quality. The definition of legibility is at our discretion, which is another reason typesetting your homework is recommended.

Midterm exam

There will be one in-class midterm exam, which will take place about 2/3 of the way through the semester. The exact date and topics covered will be announced later in the semester, on Course Schedule (which can also be accessed on the menu on the left of this webpage).

It will be an open-note exam and take the entire duration of the class period. Digital devices will not be allowed, and will not be needed.

Final project

At the end of the semester, we will have a final project with two deliverables: a 10-minute presentation and a written report.

This final project can either be novel research you are working on, or a review of a handful of related papers, providing an overview of existing results and their context within its field. If your final project is going over research which will appear in your thesis, please make sure your thesis advisor is aware and approves of this.

Regardless of whether your project covers your research or the work of others, your presentation and report should do the following:

• Identifying contributions

  • What is new in the works being presented? What was the research context prior to this work?

  • What is the main focus of the work? Is the theoretical contribution the most important aspect, or the application?

• Impact

  • What's the impact of the results? What can we do now with this result that we could not before?

  • For more theoretical works, what are the most useful implications of the results? (The implications can either be theoretical or applied, e.g. this result allows this new type of proof method for optimization algorithms, or this result guarantees exact recovery in phase-retrieval problems).

  • For more applied works, what are the real world consequences of this new technology existing? What real-world issues can be addressed with these results, and what are the consequences? Are there economic/financial or societal impacts?

• Technical content

  • For more theoretical works, you are expected to present the main proof ideas. I will expect more verbosity and ability to explain things beyond the text of what's in the paper itself, so make sure you understand the proof (and aren't simply reciting it).

  • For more applied works, you are expected to present the problem statement formally and informally, as well as the methodology. You are expected to present the reason why a particular tool was used to solve a problem and which parts are easy/difficult in applying the tool to this application. Similar to above, you will also be expected to answer questions about the methodology.

The presentations will be 10 minutes long, with 2-3 minutes for questions afterwards. You are expected to attend the presentations of your peers. You should demonstrate your understanding of the topic through your presentation; you are expected to be able to answer questions about the material you are presenting. It is probably best to think of this presentation as a simulated qualifying exam: you want to know the papers quite well and have knowledge beyond what you plan on presenting, so as to be able to answer questions.

Textbooks/materials

The following textbooks are a good reference for this course. Most of these are available for free digitally; for some, you may have to search through the UIUC library from an on-campus IP address to obtain the free PDF.

• Boyd and Vandenberghe, 'Convex Optimization’.

• Bertsekas, Nedich, and Ozdaglar, 'Convex Analysis and Optimization’.

• Ben-Tal and Nemirovski, 'Lectures on Optimization III: Convex Analysis, Nonlinear Programming Theory, Nonlinear Programming Algorithms’.

• Ben-Tal and Nemirovski, 'Lectures on Modern Convex Optimization: analysis, algorithms, and engineering applications’.

• Nesterov, 'Lectures on Convex Optimization’.

• Hiriant-Urruty and Lemarechal, 'Fundamentals of Convex Analysis’.

• Rockafellar, 'Convex Analysis’.

• Bauschke and Combettes, 'Convex Analysis and Monotone Operator Theory in Hilbert Spaces’.

Additionally, I will also post my own course notes here: Dong, 'IE 521’. This is a work-in-progress so please expect to re-download this file a few times throughout the semester.

Exact reading assignments will be posted on Course Schedule (which can also be accessed on the menu on the left of this webpage).

Course website

The central hub for this course will be the course website:

https://courses.grainger.illinois.edu/ie521/fa2023/

This webpage will be maintained and is the best resource for up-to-date information about the course. In the unlikely event of conflicting information, the information on this webpage will take precedence.

Course expectations (continued)

I expect all students to contribute to a supportive learning environment and a cooperative community. We are all here to learn, and, I'd like to emphasize this: help each other learn. Students are expected to be civil and respectful.

Throughout the course, you may freely ask questions at any time. There are no stupid questions, and everyone should feel comfortable asking anything during the class. However, I may request that discussions related to such questions be shifted to either office hours or Piazza if there is not enough in-class time to fully resolve any questions.

Academic integrity

All students are subject to the university's academic integrity policies. A quick reference guide, as well as links to the official student code, can be found at: https://provost.illinois.edu/policies/policies/academic-integrity/students-quick-reference-guide-to-academic-integrity/

I do not expect academic integrity will be an issue, but it is worth discussing briefly. If you find that you are struggling with the material in the course, do not hesitate at all to reach out to me. Send me an email, drop by my office hours, see me after class, post on Piazza, slip a note under my office door, whatever. One should not feel like they must resort to cheating in my class.

College-Wide Syllabi Information

The following text is standardized across all syllabi in the Grainger College of Engineering. This class stands behind and upholds the following statements and values therein.

Sexual Misconduct Reporting Obligation

The University of Illinois is committed to combating sexual misconduct. Faculty and staff members are required to report any instances of sexual misconduct to the University’s Title IX Office. In turn, an individual with the Title IX Office will provide information about rights and options, including accommodations, support services, the campus disciplinary process, and law enforcement options.

A list of the designated University employees who, as counselors, confidential advisors, and medical professionals, do not have this reporting responsibility and can maintain confidentiality, can be found here:

Other information about resources and reporting is available here: wecare.illinois.edu.

Academic Integrity

The University of Illinois at Urbana-Champaign Student Code should also be considered as a part of this syllabus. Students should pay particular attention to Article 1, Part 4: Academic Integrity. Read the Code at the following URL: http://studentcode.illinois.edu/.

Academic dishonesty may result in a failing grade. Every student is expected to review and abide by the Academic Integrity Policy: https://studentcode.illinois.edu/article1/part4/1-401/. Ignorance is not an excuse for any academic dishonesty. It is your responsibility to read this policy to avoid any misunderstanding. Do not hesitate to ask the instructor(s) if you are ever in doubt about what constitutes plagiarism, cheating, or any other breach of academic integrity.

Religious Observances

Illinois law requires the University to reasonably accommodate its students’ religious beliefs, observances, and practices in regard to admissions, class attendance, and the scheduling of examinations and work requirements. You should examine this syllabus at the beginning of the semester for potential conflicts between course deadlines and any of your religious observances. If a conflict exists, you should notify your instructor of the conflict and follow the procedure at https://odos.illinois.edu/community-of-care/resources/students/religious-observances/ to request appropriate accommodations. This should be done in the first two weeks of classes.

Disability-Related Accommodations

To obtain disability-related academic adjustments and/or auxiliary aids, students with disabilities must contact the course instructor and the Disability Resources and Educational Services (DRES) as soon as possible. To contact DRES, you may visit 1207 S. Oak St., Champaign, call 333-4603, e-mail disability@illinois.edu or go to https://www.disability.illinois.edu. If you are concerned you have a disability-related condition that is impacting your academic progress, there are academic screening appointments available that can help diagnosis a previously undiagnosed disability. You may access these by visiting the DRES website and selecting “Request an Academic Screening” at the bottom of the page.

Family Educational Rights and Privacy Act (FERPA)

Any student who has suppressed their directory information pursuant to Family Educational Rights and Privacy Act (FERPA) should self-identify to the instructor to ensure protection of the privacy of their attendance in this course. See https://registrar.illinois.edu/academic-records/ferpa/ for more information on FERPA.

Anti-Racism and Inclusivity Statement for Inclusion in Course Syllabi

The intent is to raise student and instructor awareness of the ongoing threat of bias and racism and of the need to take personal responsibility in creating an inclusive learning environment.

The Grainger College of Engineering is committed to the creation of an anti-racist, inclusive community that welcomes diversity along a number of dimensions, including, but not limited to, race, ethnicity and national origins, gender and gender identity, sexuality, disability status, class, age, or religious beliefs. The College recognizes that we are learning together in the midst of the Black Lives Matter movement, that Black, Hispanic, and Indigenous voices and contributions have largely either been excluded from, or not recognized in, science and engineering, and that both overt racism and micro-aggressions threaten the well-being of our students and our university community.

The effectiveness of this course is dependent upon each of us to create a safe and encouraging learning environment that allows for the open exchange of ideas while also ensuring equitable opportunities and respect for all of us. Everyone is expected to help establish and maintain an environment where students, staff, and faculty can contribute without fear of personal ridicule, or intolerant or offensive language. If you witness or experience racism, discrimination, micro-aggressions, or other offensive behavior, you are encouraged to bring this to the attention of the course director if you feel comfortable. You can also report these behaviors to the Bias Assessment and Response Team (BART) (https://bart.illinois.edu/). Based on your report, BART members will follow up and reach out to students to make sure they have the support they need to be healthy and safe. If the reported behavior also violates university policy, staff in the Office for Student Conflict Resolution may respond as well and will take appropriate action.