BIOE 485: Computational Mathematics for Machine Learning and Imaging

Course Description

The course will cover fundamental mathematical and computational methods needed to implement computational imaging and machine learning solutions. First, relevant aspects of probability theory, matrix decompositions, and vector calculus will be introduced. Subsequently, methods that underline approximate inference, such as stochastic sampling methods, are introduced. Finally, numerical optimization methods that represent core components of computed imaging and machine learning will be introduced. This will include numerical optimization-based formulations of inverse problems. An emphasis will be placed on first-order deterministic and stochastic gradient-based methods. Second-order optimization techniques including quasi-Newton and Hessian-free methods will also be surveyed. The application of these methods to computed imaging and machine learning problems will be addressed in detail.


Learning Objectives

Upon completion of this course, the students will be able to

  • Demonstrate the ability to formulate the solution of computed imaging and machine learning problems as numerical optimization problems

  • To enable students to apply cutting edge optimization techniques to solve computed imaging and machine learning problems

  • To enable students to understand the pros and cons of various numerical optimization techniques and to customize solutions for new problems that arise in practice

  • Understand the difference and relative advantages of deterministic vs stochastic optimization methods

  • Understand the basic principles of variational Bayesian inference and how it relates to a numerical optimization problem

  • Understand the basic principles of how to sample from a high-dimensional probability distribution


Grading

  • Homework: 30% (3 HWs, each 10%)

  • Machine Problems: 40% (4 MPs, each 10%)

  • Project: 30% (Final project presentation: 10%; Final project report and code: 20%)

  • (No exams)

Resources

  • Mathematics for Machine Learning, M. Deisenroth, A. Faisal, and C. Ong, Cambridge University Press (free online version)

  • Additional reading material will be assigned from a combination of book chapters, review articles, and primary research papers

Lecture notes & slides

Office hours & contact

  • Prof. Yogatheesan Varatharajah:

    • Tuesday - 3:30pm to 4:30pm (after class)

    • Location - Everitt Lab 3213

    • Contact - varatha[number2][at]illinois[dot]edu

      • Please include "[BIOE 485]" in email title!

  • (TA) Nimit Kapadia:

    • Wednesday - 3:30pm to 4:30pm

    • Location - Everitt Lab 3213

    • Contact - nimithk[number2][at]illinois[dot]edu

      • Please include "[BIOE 485]" in email title!