Abstract: These lecture notes document ACM 204 as taught in Winter 2022, and they are primarily intended as a reference for students who have taken the class. The notes are prepared by student scribes with feedback from the instructor. The notes have been edited by the instructor to try to correct his own failures of presentation. All remaining errors and omissions are the fault of the instructor. Please be aware that these notes reflect material presented in a classroom, rather than a formal scholarly publication. In some places, the notes may lack appropriate citations to the literature. There is no claim that the arrangement or presentation of the material is primarily due to the instructor. The notes also contain the projects of students who wished to share their work. They received feedback and made revisions, but the projects have not been edited. They represent the students’ individual work.

No.: 2022-01
ID: CaltechAUTHORS:20220412-221559139

]]>

Abstract: ACM 217 is a second-year graduate course on high-dimensional probability, designed for students in computing and mathematical sciences. We discuss phenomena that emerge from probability models with many degrees of freedom, tools for working with these models, and a selection of applications to computational mathematics. The Winter 2021 edition of ACM 217 is the fourth instantiation of a class that initially focused on concentration inequalities and that has expanded to include other topics in high-dimensional probability. This year, the course was more mathematical than some previous editions, with less attention to tools and applications. This slant may not serve applied students well, and it is likely that future versions of the course will strike a different balance between theory and practice. These lecture notes document ACM 217 as it was taught in Winter 2021. The notes are being transcribed by the students as part of their coursework, and they are edited lightly by the instructor. They are intended as a record for the students who have taken the course. Other readers should beware that this course is neither refined nor especially coherent. There is no warranty about correctness. Furthermore, these notes have been prepared using many sources and without appropriate scholarly citations.

No.: 2021-01
ID: CaltechAUTHORS:20220412-221302767

]]>

Abstract: ACM 204 is a graduate course on randomized algorithms for matrix computations. It was taught for the first time in Winter 2020. The course begins with Monte Carlo algorithms for trace estimation. This is a relatively simple setting that allows us to explore how randomness can be used for matrix computations. We continue with a discussion of the randomized power method and the Lanczos method for estimating the largest eigenvalue of a symmetric matrix. For these algorithms, the randomized starting point regularizes the trajectory of the iterations. The Lanczos iteration and randomized trace estimation fuse together in the stochastic Lanczos quadrature method for estimating the trace of a matrix function. Then we turn to Monte Carlo sampling methods for matrix approximation. This approach is justified by the matrix Bernstein inequality, a powerful tool for matrix approximation. As a simple example, we develop sampling methods for approximate matrix multiplication. In the next part of the course, we study random linear embeddings. These are random matrices that can reduce the dimension of a dataset while approximately preserving its geometry. First, we treat Gaussian embeddings in detail, and then we discuss structured embeddings that can be implemented using fewer computational resources. Afterward, we describe several ways to use random embeddings to solve over-determined least-squares problems. We continue with a detailed treatment of the randomized SVD algorithm, the most widely used technique from this area. We give a complete a priori analysis with detailed error bounds. Then we show how to modify this algorithm for the streaming setting, where the matrix is presented as a sequence of linear updates. Last, we show how to develop an effective algorithm for selecting influential columns and rows from a matrix to obtain skeleton or CUR factorizations. The next section of the course studies kernel matrices that arise in high-dimensional data analysis. We discuss positive-definite kernels and outline the computational issues associated with solving linear algebra problems involving kernels. We introduce random feature approximations and Nyström approximations based on randomized sampling. This area is still not fully developed. The last part of the course gives a complete presentation of the sparse Cholesky algorithm of Kyng & Sachdeva [KS16], including a full proof of correctness.

No.: 2020-01
ID: CaltechAUTHORS:20210421-101607288

]]>

Abstract: These lecture notes were written to support the short course Matrix Concentration & Computational Linear Algebra delivered by the author at École Normale Supérieure in Paris from 1–5 July 2019 as part of the summer school “High-dimensional probability and algorithms.” The aim of this course is to present some practical computational applications of matrix concentration.

No.: 2019-01
ID: CaltechAUTHORS:20190715-125341188

]]>

Abstract: ACM 204, Winter 2019

No.: 2019-02
ID: CaltechAUTHORS:20220412-220319430

]]>