LA 101
M07 · L01
Module 7: Matrix Decompositions
LU Decomposition
Factor A once into a lower triangular L and an upper triangular U. Then solve Ax = b for any right-hand side in O(n²) — the foundation of computational linear algebra.
01 / 10
LA 101
M07 · L01
The LU Factorization
Two Triangular Factors
Any square matrix A (under mild conditions) can be written as the product of two triangular matrices.
L
lower triangular, 1s on diagonal
U
upper triangular, row echelon
A = LU
the factorization
Motivation
Solve Ax = b for many right-hand sides b without repeating elimination each time.
02 / 10
LA 101
M07 · L01
L and U Factors
Structure of A = LU
LU Factorization
A = LU,\quad L_{ii}=1,\; L_{ij}=0\;(j>i),\quad U_{ij}=0\;(j<i)
Key property
L has 1s on its diagonal. The off-diagonal entries of L are the multipliers from Gaussian elimination.
03 / 10
LA 101
M07 · L01
Gaussian Elimination Connection
Elimination Repackaged
Gaussian elimination on A produces U and discards the multipliers. LU decomposition saves them in L.
- Multiplier mᵢₖ = aᵢₖ / aₖₖ is stored in L[i,k]
- U is the row echelon form after all steps
- L is the inverse of the product of elementary matrices
- Same arithmetic — just recorded differently
04 / 10
LA 101
M07 · L01
Two-Step Solve
Forward & Back Substitution
Ax = b via LU
Ly = b \;\text{(forward)},\quad Ux = y\;\text{(back)}
- Forward sub: solve Ly = b top-to-bottom, O(n²)
- Back sub: solve Ux = y bottom-to-top, O(n²)
- Both steps exploit triangular structure
- New b? Just run the two cheap substitutions again
05 / 10
LA 101
M07 · L01
Why LU? Multiple Right-Hand Sides
One Factorization, Many Solves
The cost asymmetry is the whole point. Factorize once, solve cheaply every time.
- n³/3 flops — one-time factorization cost
- n² flops — cost per new right-hand side
- For n = 10,000: factorization is 1,600× the cost of one solve
- Used in MATLAB \, NumPy linalg.solve, LAPACK dgesv
- Also enables: matrix inverse, determinant computation
06 / 10
LA 101
M07 · L01
Pivoting for Stability
PA = LU with Pivoting
Partial Pivoting
PA = LU,\quad |m_{ik}| \leq 1
Why pivoting?
Choose the largest-magnitude element as pivot at each step. All multipliers satisfy |mᵢₖ| ≤ 1, bounding error growth. Standard in every production solver.
07 / 10
LA 101
M07 · L01
Worked 3×3 Example
Seeing LU in Action
3×3 Factorization
\begin{pmatrix}2&1&1\\4&3&3\\8&7&9\end{pmatrix}=\begin{pmatrix}1&0&0\\2&1&0\\4&3&1\end{pmatrix}\begin{pmatrix}2&1&1\\0&1&1\\0&0&2\end{pmatrix}
Multipliers
m₂₁ = 2, m₃₁ = 4, m₃₂ = 3 — stored in L below the diagonal. U is the resulting row echelon form.
09 / 10
LA 101
M07 · L01
Module 7 Started
LU: The Workhorse
A = LU (or PA = LU with pivoting) is Gaussian elimination repackaged as two reusable triangular factors. It costs n³/3 once and n² per solve — the key to efficient computational linear algebra.
Module 7: Matrix Decompositions
LU Decomposition — Done ✓
10 / 10