Source Guide: SVD and Low-Rank Approximation

A curated source guide for learning SVD and low-rank approximation from first pass through research bridge.

Keywords

source guide, svd, low-rank approximation

1 How To Use This Guide

Use this guide on two axes at once.

First choose the role you need:

• First pass for intuition and basic fluency
• Second pass for proofs, computation, and broader applications
• Paper bridge for research-facing reading

Then use the sections below as filters:

• Stable Core for durable first-pass and second-pass references
• Current Bridge for newer surveys or recent frontier notes
• Paper Bridge for concrete papers

Do not try to read everything in parallel.

2 Stable Core

• MIT 18.06 Lecture 29: Singular value decomposition - First pass - best official first stop if you want the geometric core without too much abstraction. Checked 2026-04-24.
• Introduction to Applied Linear Algebra – Vectors, Matrices, and Least Squares - First pass - strong official online book for the applied and computational viewpoint. Checked 2026-04-24.
• Hefferon, Linear Algebra - Second pass - useful for self-study, proofs, and exercises once the first intuition is in place. Checked 2026-04-24.
• CS168 Lecture 9: The Singular Value Decomposition and Low-Rank Matrix Approximations - Second pass - especially good for denoising, matrix completion intuition, and modern applications. Checked 2026-04-24.

3 Current Bridge

• Randomized Numerical Linear Algebra: Foundations & Algorithms - Second pass - modern survey context for low-rank approximation, streaming, and sketching. Checked 2026-04-24.
• Randomized low-rank approximations beyond Gaussian random matrices - Paper bridge - current example of how the assumptions behind randomized approximation are still broadening. Checked 2026-04-24.
• Low-Rank Adaptation for Foundation Models: A Comprehensive Review - Second pass - current survey showing how low-rank structure now appears in model adaptation workflows. Checked 2026-04-24.

4 Paper Bridge

• Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions - Paper bridge - classic starting point for approximate low-rank algorithms. Checked 2026-04-24.
• LoRA: Low-Rank Adaptation of Large Language Models - Paper bridge - useful if you want to see low-rank ideas in a modern ML setting immediately. Checked 2026-04-24.
• Streaming Low-Rank Matrix Approximation with an Application to Scientific Simulation - Paper bridge - good next paper if memory and pass constraints interest you. Checked 2026-04-24.

5 Recommended Reading Paths

5.1 Path 1: First serious understanding

1. MIT 18.06 Lecture 29
2. SVD and Low-Rank Approximation
3. Stanford VMLS or Hefferon for more exercises

5.2 Path 2: PCA and data analysis

1. SVD and Low-Rank Approximation
2. PCA Through SVD
3. Computation Lab: Rank-1 Approximation and PCA Geometry
4. Stanford VMLS

5.3 Path 3: Bridge to randomized algorithms

1. SVD and Low-Rank Approximation
2. Eckart-Young and Truncated SVD
3. Paper Lab: Finding Structure with Randomness
4. RNLA survey

5.4 Path 4: Bridge to modern model adaptation

1. SVD and Low-Rank Approximation
2. Research Direction: Low-Rank Structure from Randomized Algorithms to Model Adaptation
3. LoRA paper
4. 2025 low-rank adaptation review

6 Sources Checked Online

All links above were checked on 2026-04-24.

The online verification pass included:

• official course pages
• official author-maintained textbook pages
• official arXiv or author-hosted paper pages
• current survey pages for low-rank adaptation and randomized low-rank methods