The Gram-Schmidt Process
Given any linearly independent vectors, Gram-Schmidt builds an orthonormal basis — one direction at a time, subtracting what you already have.
From Oblique to Orthogonal
Real-world vectors are rarely perpendicular. Gram-Schmidt converts any basis into an orthonormal basis — same subspace, clean geometry.
Three Steps, Repeated
- Take the next input vector vₖ
- Subtract its projections onto q₁, …, qₖ₋₁
- Normalize the residual to unit length
- Repeat for each new vector
Subtract Every Projection
Each inner product qⱼᵀvₖ measures how much of vₖ lies along qⱼ. Subtracting them all leaves a vector orthogonal to everything before it.
The First Two Vectors
A = QR
Gram-Schmidt on the columns of A produces the QR decomposition — Q has orthonormal columns, R is upper triangular.
Sequential Structure
Each vₖ only involves q₁ through qₖ (earlier vectors can't see later ones). That "one-way" dependency makes R upper triangular.
Modified Gram-Schmidt
- Classical GS accumulates rounding errors
- Modified GS updates the running vector after each subtraction
- Same math, far better floating-point behavior
- MGS is the standard in practice (LAPACK, NumPy)
P = QQᵀ
Once you have an orthonormal basis Q, projection needs no matrix inversion. Just dot products.
- Least squares — solve via back-substitution on R
- Eigenvalues — QR iteration algorithm
- Fourier series — projections onto sinusoidal basis
- Signal processing — fast, stable inner products
M5-L3: Orthogonal Matrices
You've built orthonormal bases with Gram-Schmidt. Next: matrices whose columns are orthonormal — a special family with remarkable properties for rotations and reflections.