LA 101
M07 · L02
Module 7: Matrix Decompositions
QR Decomposition
Factor A = QR into an orthogonal Q and an upper triangular R. QR is the backbone of least-squares solvers and eigenvalue algorithms — and it is far more stable than the normal equations.
01 / 10
LA 101
M07 · L02
The QR Factorization
Orthogonal and Triangular
Any m × n matrix A with linearly independent columns factors as A = QR.
Q
orthonormal columns, QᵀQ = I
R
upper triangular, positive diagonal
A = QR
the factorization
Key insight
Works for rectangular matrices (m ≥ n). Q preserves lengths and angles — perfect conditioning.
02 / 10
LA 101
M07 · L02
Computing QR: Method 1
Gram-Schmidt Process
Gram-Schmidt Step
\tilde{e}_k = a_k - \sum_{i=1}^{k-1}(q_i^T a_k)q_i,\quad q_k = \frac{\tilde{e}_k}{\|\tilde{e}_k\|}
Modified vs. Classical
Classical GS loses orthogonality in floating point. Modified GS subtracts each projection immediately — much more stable.
03 / 10
LA 101
M07 · L02
Computing QR: Method 2
Householder Reflections
An orthogonal matrix H = I − 2vvᵀ/(vᵀv) reflects any vector across a hyperplane. Choose v to zero out an entire column below the diagonal in one step.
- H is orthogonal AND symmetric: H² = I, HᵀH = I
- Apply n reflectors from the left to get R = Hₙ⋯H₁A
- Recover Q = H₁⋯Hₙ (product of reflectors)
- Numerically stable — used in LAPACK and NumPy
04 / 10
LA 101
M07 · L02
Computing QR: Method 3
Givens Rotations
Givens rotations zero out one entry at a time by rotating two coordinates. The surgical scalpel of QR algorithms.
- Each rotation G(i, j, θ) affects only rows i and j
- Preserves sparsity — ideal for banded matrices
- Used for incremental QR updates (adding one row at a time)
- More flops than Householder for dense A, but beats it for sparse
05 / 10
LA 101
M07 · L02
Application: Least Squares
Solve min ‖Ax − b‖² Stably
QR Least Squares
A=QR \Rightarrow Rx = Q^T b \text{ (back sub)}
vs. Normal Equations
Normal equations form AᵀA, squaring the condition number: κ(AᵀA) = κ(A)². QR keeps κ(A). Far more accurate for ill-conditioned data.
06 / 10
LA 101
M07 · L02
Application: Eigenvalues
The QR Algorithm
QR Iteration
A_k = Q_k R_k \Rightarrow A_{k+1} = R_k Q_k
Why it works
Each step Aₖ₊₁ = RₖQₖ is similar to Aₖ — same eigenvalues. The sequence converges to upper triangular (Schur) form with eigenvalues on the diagonal.
07 / 10
LA 101
M07 · L02
Numerical Stability
Condition Number Advantage
Condition Numbers
\kappa(A^T A) = \kappa(A)^2 \gg \kappa(R) = \kappa(A)
- Normal equations: condition is κ(A)² — digits lost double
- QR: condition is κ(A) — half the precision loss
- For κ(A) = 10⁶: normal eq. loses 12 digits, QR loses 6
- Q is orthogonal → Q⁻¹ = Qᵀ, never amplifies errors
09 / 10
LA 101
M07 · L02
Module 7 Continues
QR: The Stable Solver
A = QR (orthogonal × upper triangular) enables least-squares with half the condition-number penalty of normal equations, and drives the QR eigenvalue algorithm. Householder reflections make it numerically bulletproof.
Module 7: Matrix Decompositions
QR Decomposition — Done ✓
10 / 10