Linear Algebra

Core linear algebra for research-facing work in CS, AI, and engineering.

Keywords

linear algebra, least squares, spectral methods

1 Why This Module Matters

Linear algebra is one of the most important modules on the whole site. Modern ML, optimization, signal processing, control, and statistics all keep returning to vectors, operators, geometry, and spectra.

Prerequisites Algebra and late single-variable calculus

Unlocks Multivariable calculus, optimization, numerics, matrix analysis

Research Use Least squares, SVD, covariance, kernels, spectral arguments

2 First Pass Through This Module

1. Vectors and Linear Combinations
2. Matrices and Linear Maps
3. Subspaces, Basis, and Dimension
4. Orthogonality and Least Squares
5. Eigenvalues and Diagonalization
6. SVD and Low-Rank Approximation

On a first pass, stay on the concept pages. You should not need proof, application, lab, paper-lab, research, or source-guide pages just to understand the main story of the module.

3 Navigation Rule

Use the module overview and core concept pages as the main first-pass route.

Proof, exercise, application, lab, paper-lab, research, and source-guide pages are deep-dive companions. They stay linked from this module page and the topic concept pages so the main study order remains readable.

4 Go Deeper By Topic

4.1 Vectors and Linear Combinations

Start with Vectors and Linear Combinations.

If you want more after the main page:

• Proof: Span Is a Subspace
• Application: Vector Mixtures in Embeddings and Attention
• Visual intuition: Computation Lab: Linear Combinations and Span Geometry
• Practice: Exercises: Vectors and Linear Combinations

4.2 Matrices and Linear Maps

Start with Matrices and Linear Maps.

If you want more after the main page:

• Proof: Basis Images Determine a Linear Map
• Application: Learned Linear Projections in Transformers
• Visual intuition: Computation Lab: Matrix Composition and Basis Action
• Practice: Exercises: Matrices and Linear Maps

4.3 Subspaces, Basis, and Dimension

Start with Subspaces, Basis, and Dimension.

If you want more after the main page:

• Proof: Exchange Argument and Why Dimension Is Well Defined
• Application: Low-Dimensional Subspace Models
• Visual intuition: Computation Lab: Basis and Column Space Geometry
• Practice: Exercises: Subspaces, Basis, and Dimension

4.4 Orthogonality and Least Squares

Start with Orthogonality and Least Squares.

If you want more after the main page:

• Proof: Projection Theorem and Normal Equations
• Application: Linear Regression Through Projection
• Visual intuition: Computation Lab: Projection Geometry and Regression Residuals
• Practice: Exercises: Orthogonality and Least Squares

4.5 Eigenvalues and Diagonalization

Start with Eigenvalues and Diagonalization.

If you want more after the main page:

• Proof: Distinct Eigenvalues Give Independent Eigenvectors
• Application: Spectral Modes in Consensus and Graphs
• Visual intuition: Computation Lab: Matrix Powers and Spectral Modes
• Practice: Exercises: Eigenvalues and Diagonalization

4.6 SVD and Low-Rank Approximation

Start with SVD and Low-Rank Approximation.

If you want more after the main page:

• Proof: Eckart-Young and Truncated SVD
• Application: PCA Through SVD
• Visual intuition: Computation Lab: Rank-1 Approximation and PCA Geometry
• Practice: Exercises: SVD and Low-Rank Approximation

5 Research Bridge

After the first pass, use the Optional Paper Bridge and Go Deeper sections inside each concept page to open the relevant:

• paper lab
• research direction
• source guide

That keeps the module landing page clean while still making every topic package discoverable from its own home page.

6 What You Want By The End

• geometric intuition
• symbolic fluency
• operator viewpoint
• ability to recognize where a paper is really using spectral structure

7 Application Door

If you want a first application target, start with least squares -> SVD -> PCA -> low-rank approximation. That chain alone will pay off again and again.

8 Sources and Further Reading

• MIT 18.06SC Linear Algebra resource index - First pass - strong official module spine. Checked 2026-04-24.
• Stanford Math 51 - First pass - current applied framing for the linear algebra side of the module. Checked 2026-04-24.
• Hefferon, Linear Algebra - Second pass - proof-and-exercise depth for self-study. Checked 2026-04-24.