LA 101
M06 · L03
Module 6: Inner Product Spaces
Fourier Series Connection
The Fourier series is not a mysterious transform — it is orthogonal projection in L². Every coefficient is a dot product, every partial sum is a best approximation. Linear algebra makes it all transparent.
01 / 10
LA 101
M06 · L03
The Orthonormal Basis
The Trigonometric System
L²([−π, π]) has a complete orthonormal basis: the trigonometric functions.
1/√(2π)
constant
cos(nt)/√π
cosines
sin(nt)/√π
sines
Orthogonality
⟨cos(mt), cos(nt)⟩ = π δₘₙ — distinct harmonics have zero inner product.
02 / 10
LA 101
M06 · L03
Coefficients as Inner Products
Fourier Coefficients
Fourier Coefficients
a_n = \tfrac{1}{\pi}\!\int_{-\pi}^{\pi}\!f(t)\cos(nt)\,dt,\quad b_n = \tfrac{1}{\pi}\!\int_{-\pi}^{\pi}\!f(t)\sin(nt)\,dt
Same as ℝⁿ
Extracting coordinates via inner products — the direct analogue of v_i = ⟨v, eᵢ⟩.
03 / 10
LA 101
M06 · L03
Minimum-Error Approximation
Partial Sums as Projections
Best Approximation
S_N f = \underset{g\in V_N}{\arg\min}\,\|f-g\|_{L^2}
Sₙf is the orthogonal projection of f onto Vₙ = span{1, cos t, sin t, …, cos(Nt), sin(Nt)}. The error f − Sₙf is orthogonal to Vₙ.
04 / 10
LA 101
M06 · L03
Complex Form
Complex Exponentials
Complex Coefficients
c_n = \tfrac{1}{2\pi}\!\int_{-\pi}^{\pi}\!f(t)e^{-int}\,dt
- Basis: eⁱⁿᵗ/√(2π) for n ∈ ℤ
- Complete orthonormal in complex L²([−π, π])
- Unifies cosine and sine into one formula
- Extends naturally to the Fourier transform
05 / 10
LA 101
M06 · L03
Limits of L² Optimality
Gibbs Phenomenon
Near a jump discontinuity, partial sums overshoot by ~9% of the jump height — no matter how many terms.
- L² best ≠ small everywhere
- Overshoot moves closer to jump, but never vanishes
- Dirichlet theorem: pointwise convergence to average at jumps
- Windowing (Hamming, Hanning) smooths this in practice
06 / 10
LA 101
M06 · L03
Finite-Dimensional Version
The DFT Matrix
DFT
\mathbf{X} = F\mathbf{x},\quad F_{kn}=\omega^{kn}/\!\sqrt{N},\quad FF^*=I
Unitary
F F* = I — the DFT is a change of basis in ℂᴺ to an orthonormal DFT basis. The FFT computes it in O(N log N).
07 / 10
LA 101
M06 · L03
Filtering is Diagonalization
Convolution Theorem
Convolution = Multiplication
F(\mathbf{x}\circledast\mathbf{h}) = \sqrt{N}\,(F\mathbf{x}) \odot (F\mathbf{h}) \;\Leftrightarrow\; Y[k]=H[k]X[k]
- The √N comes from the unitary 1/√N normalization of F
- Complex exponentials are eigenfunctions of LTI systems
- Filter's frequency response = eigenvalues
- OFDM: N independent scalar channels via DFT
- Matched filtering = inner product with template
09 / 10
LA 101
M06 · L03
Lesson 3 Complete
Fourier is Linear Algebra!
Coefficients are inner products. Partial sums are projections. DFT is a unitary basis change. Filtering is diagonalization. The entire framework of Fourier analysis is just linear algebra in infinite (or finite) dimensions.
Module 6 Complete
Inner Product Spaces — Done ✓
10 / 10