LA 101
M05 · L01
Module 5: Orthogonality
Orthogonal Vectors and Projections
When two vectors meet at a right angle, they share no information. Orthogonality is the geometric foundation of projections, least squares, and the Fourier transform.
01 / 11
LA 101
M05 · L01
The Definition
Zero Dot Product
Two vectors are orthogonal when their dot product is zero — they meet at a 90° angle in any dimension.
Orthogonality
\mathbf{v}^T\mathbf{w}=0
02 / 11
LA 101
M05 · L01
Why It Matters
Pythagoras in n Dimensions
- Dot product = ||v|| ||w|| cos θ, so v·w = 0 means θ = 90°
- If v ⊥ w then ||v + w||² = ||v||² + ||w||²
- The zero vector is orthogonal to every vector
- Standard basis vectors are orthogonal to each other
03 / 11
LA 101
M05 · L01
Orthogonal Complement
W and W⊥
The orthogonal complement W⊥ of a subspace W contains all vectors perpendicular to every vector in W.
dim W
subspace
+
dim W⊥
complement
= n
total
04 / 11
LA 101
M05 · L01
Projection onto a Line
The Shadow Formula
Projection of b onto line through a
proj = (aᵀb / aᵀa) · a
Key insight
Error b − proj is perpendicular to a
05 / 11
LA 101
M05 · L01
Projection Matrix
P = aaᵀ / aᵀa
The projection matrix P maps any vector to its projection onto the line through a.
Property 1
P² = P (project twice = project once)
Property 2
Pᵀ = P (symmetric matrix)
06 / 11
LA 101
M05 · L01
Projection onto a Subspace
The Normal Equations
Projection matrix
P=A(A^TA)^{-1}A^T
Solve AᵀA x̂ = Aᵀb to find the projection coordinates. When A has orthonormal columns Q, this simplifies to P = QQᵀ.
07 / 11
LA 101
M05 · L01
Orthogonal Decomposition
b = p + e
- Every vector splits uniquely: b = p + e
- p = Pb lies in the subspace W
- e = (I−P)b lies in the complement W⊥
- The two pieces are orthogonal: p · e = 0
- I − P is itself a projection onto W⊥
08 / 11
LA 101
M05 · L01
Why This Matters
Everywhere in Math
- Least squares — closest point in a subspace
- Gram-Schmidt — build orthogonal basis step by step
- QR decomposition — factor any matrix
- Fourier transform — orthogonal frequency basis
- Signal processing — projections onto sinusoids
10 / 11
LA 101
Up Next
Lesson 1 Complete
M5-L2: Gram-Schmidt
You've mastered orthogonality and projection. Next: the Gram-Schmidt process — an algorithm to build an orthogonal basis from any set of linearly independent vectors.
Coming Up
11 / 11