Lecture notes

• Lecture 1: Introduction
• Lecture 2: Concentration of sum of independent random variables I
• Lecture 3: Concentration of sum of independent random variables II
• Lecture 4: Random vectors in high dimensions I
• Lecture 5: Random vectors in high dimensions II
• Lecture 6: Application: Grothendieck's inequality and semidefinite programming
• Lecture 7: Max-Cut and Preliminaries on Matrices
• Lecture 8: Guides on Literature review, Nets and Covering Numbers
• Lecture 9: Random Sub-Gaussian Matrices and Community Detection
• Lecture 10: Covariance Estimation and PCA
• Lecture 11: Concentration without independence
• Lecture 12: Concentration of Lipschtz functions and Johnson-Lindenstrauss Lemma
• Lecture 13: Matrix Bernstein Inequality and Applications
• Lecture 14: Diffusion Maps
• Lecture 15: Diffusion Distance
• Lecture 16: Other Graph Metrics
• Lecture 17: Estimation on Graphs
• Lecture 18: Semi-supervised Learning and Harmonic Extension
• Lecture 19: Harmonic Extension and Supervised Learning on Graphs
• Lecture 20: Geometric deep learning
• Lecture 21--22: Graph neural networks, Vector Diffusion Maps
• Lecture 24: Harmonic analysis on groups
• Lecture 25: Harmonic analysis on group and homogeneous space
• Lecture 26: Generalization of equivariance and convolution in neural networks to the action of compact groups
• Lecture 27: Review

Useful Links and Recommended Texts

Textbooks

• High-Dimensional Probability, Roman Vershynin, Cambridge University Press, 2019.
• High-Dimensional Statistics, Martin Wainwright, Cambridge University Press, 2019.
• High-Dimensional Statistics, Philippe Rigollet and Jan-Christian Hutter, Lecture notes, 2019.
• G. Golub and C. Van Loan, Matrix Computations, Johns Hopkins University Press, 3rd ed. 1996.
• Computational Optimal Transport, Gabriel Peyré, Marco Cuturi, Foundations and Trends in Machine Learning, Vol 11, Issue 5-6, 2019.

Sparsity

• Sparse and redundant representations: from theory to applications in signal and image processing, Michael Elad, Springer, 2010.
• A mathematical introduction to compressive sensing, Simon Foucart, and Holger Rauhut, Springer, 2013.
• Mathematics of sparsity (and a few other things), Emmanuel Candes, International Congress of Mathematicians, 2014.
• Statistical learning with sparsity: the Lasso and generalizations, Trevor Hastie, Robert Tibshirani, and Martin Wainwright, Chapman and Hall/CRC, 2015.

Random Matrix

• Introduction to the non-asymptotic analysis of random matrices, Roman Vershynin, Compressed Sensing: Theory and Applications, 2010.
• Topics in random matrix theory, Terence Tao, American Mathematical Society, 2012.
• An Introduction to Matrix Concentration Inequalities, Joel Tropp, Foundations and Trends in Machine Learning, 2015.

Clustering

• Information-Theoretic Bounds and Phase Transitions in Clustering, Sparse PCA, and Submatrix Localization, Banks et al., IEEE Transactions on Information Theory, Vol. 64, No. 7, 4872 -- 4894, 2018

Nonlinear Dimensionality Reduction and Geometric Deep Learning

• Can one hear the shape of a drum , Mark Kac, The American Mathematical Monthly, Vol. 73, No. 4, pp. 1-23, 1966.
• A Global Geometric Framework for Nonlinear Dimensionality Reduction , Joshua B. Tenenbaum, Vin de Silva, John C. Langford, Science, vol. 290, pp. 2319--2323, 2000.
• Nonlinear Dimensionality Reduction by Locally Linear Embedding , Sam T. Roweis, Lawrence K. Saul, Science, vol. 290, pp.2323--2326, 2000.
• Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , Mikhail Belkin and Partha Niyogi, Neural Computation, Vol. 15, pp.1373--1396 2013.
• Diffusion Maps , Ronald R.Coifman Stephane Lafon, Applied and Computational Harmonic Analysis, pp. 5--30, 2006
• Vector Diffusion Maps , A. Singer H.‐T. Wu, vol.8, pp. 1067-1144 2012.
• Geometric deep learning: going beyond Euclidean data, M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, P. Vandergheynst, IEEE Signal Processing Magazine 2017
• Group theoretical methods in machine learning by Risi Kondor
• Geometric Deep Learning website
• Tutorial on geometric deep learning by Michael Bronstein

No other question has ever moved so profoundly the spirit; no other idea has so fruitfully stimulated the intellect; yet no other concept stands in greater need of clarification than that of the infinite. David Hilbert

ECE 598ZZ (High-Dimensional Geometric Data Analysis): This course aims to establish the mathematical foundation of many recent algorithms for tasks such as organization and visualization of data clouds, dimensionality reduction, clustering, and regression. Data analysis is an interdisciplinary field. It combines mathematics (both pure and applied), computer science (machine learning, theoretical CS, AI, computer vision), electrical engineering (signal and image processing), statistics, structural biology, neuroscience, computational biology (microarray data for gene expression), biophysics and chemical engineering (molecular dynamics simulations), and more. We will focus on a few particular methods and explain what they are good for, what are their limitations, what is the underlying math, in order to develop a good sense of when to apply them and develop a sound basis for designing new data analysis algorithms. The course will have three main sections: 1) high dimensional probability, 2) geometric data analysis, and 3) other recent advances with applications. The high-dimensional probability section of the course aims at getting insight into the behavior of random vectors, random matrices, random subspaces, and objects used to quantify uncertainty in high dimensions. In the second part of the course, we introduce spectral methods that are useful in the analysis of big data sets. Particular applications involve cryo-electron microscopy single particle reconstruction and density functional theory with strongly correlated electrons. Prerequisite: ECE 534.

 

Compass

© Copyright 2019, University of IllinoisDesign adapted from: Luka Cvrk, Solucija

---