Home / LA 101 / Module 6 / Lesson 3
Stories Mode

Fourier Series Connection

The classical Fourier series is not a mysterious transform — it is orthogonal projection in an inner product space. Every coefficient is a dot product, every partial sum is a best approximation, and convergence is measured by the norm. Linear algebra makes it all transparent.

~18 min read M6 · L3 Intermediate

The Trigonometric Basis

In Lesson 2 we established that L²([−π, π]) is a Hilbert space — a complete inner product space. Now we ask: does it have an orthonormal basis? The answer is yes, and the basis is the trigonometric system.

Define the functions φ₀(t) = 1/√(2π) and, for n ≥ 1, φ_n^c(t) = cos(nt)/√π and φ_n^s(t) = sin(nt)/√π. These are pairwise orthogonal under the L² inner product ⟨f, g⟩ = ∫₋π^π f(t)g(t) dt, and each has unit norm. Together they form a countably infinite orthonormal set that is also complete — no nonzero function is orthogonal to all of them.

Orthogonality of the Trigonometric System
\int_{-\pi}^{\pi} \cos(mt)\cos(nt)\,dt = \pi\,\delta_{mn}, \quad \int_{-\pi}^{\pi}\sin(mt)\sin(nt)\,dt = \pi\,\delta_{mn}, \quad \int_{-\pi}^{\pi}\cos(mt)\sin(nt)\,dt = 0
The integrals that establish mutual orthogonality. Each pair of distinct basis functions has zero inner product. These identities follow from the product-to-sum formulas for trigonometric functions and are the foundation on which all Fourier analysis rests.

Fourier Coefficients as Inner Products

Because the trigonometric system is a complete orthonormal set in L²([−π, π]), every f ∈ L² can be expanded as an infinite series. The expansion coefficients are precisely the inner products of f with each basis function:

Fourier Coefficients
a_n = \frac{1}{\pi}\int_{-\pi}^{\pi} f(t)\cos(nt)\,dt, \quad b_n = \frac{1}{\pi}\int_{-\pi}^{\pi} f(t)\sin(nt)\,dt
The Fourier coefficients a₀, aₙ, bₙ are inner products in L². They measure how much of each basis frequency is present in f. This is the direct analogue of v_i = ⟨v, eᵢ⟩ in ℝⁿ — coordinates extracted by projecting onto basis vectors.

The Fourier series is the resulting infinite expansion:

Fourier Series
f(t) = \frac{a_0}{2} + \sum_{n=1}^{\infty}\bigl[a_n\cos(nt) + b_n\sin(nt)\bigr]
The series converges in L² norm: ‖f − Sₙf‖ → 0 as N → ∞. This does not guarantee pointwise convergence everywhere, but for piecewise smooth functions the series converges pointwise except at jump discontinuities, where it converges to the average of the left and right limits.

Partial Sums as Orthogonal Projections

The N-th partial sum Sₙf = a₀/2 + Σᵢ₌₁ᴺ [aᵢ cos(it) + bᵢ sin(it)] is not just a truncation — it is the orthogonal projection of f onto the finite-dimensional subspace Vₙ = span{1, cos t, sin t, …, cos(Nt), sin(Nt)}.

By the Projection Theorem, Sₙf is the unique element of Vₙ closest to f in the L² norm. No other linear combination of the first N+1 harmonics can do better. This geometric interpretation explains why Fourier series are the natural tool for approximating functions with periodic structure.

Best Approximation Property
S_N f = \underset{g\,\in\,V_N}{\arg\min}\,\|f - g\|_{L^2}, \qquad \langle f - S_N f,\, g\rangle = 0 \;\forall\, g \in V_N
The N-th partial sum is the minimizer of the L² error over all linear combinations of the first N+1 basis functions. The error f − Sₙf is orthogonal to Vₙ — this is the orthogonality condition that characterizes the projection.

The Residual is Orthogonal to the Subspace

A key property of orthogonal projection: the error f − Sₙf satisfies ⟨f − Sₙf, g⟩ = 0 for every g ∈ Vₙ. This means the error has no component in the directions we have already approximated. Each additional harmonic we include captures a bit of the remaining error, and that captured piece is always orthogonal to everything already included.

Complex Fourier Series

Working with complex exponentials eⁱⁿᵗ simplifies the algebra considerably. Define ψₙ(t) = eⁱⁿᵗ/√(2π) for n ∈ ℤ. These form a complete orthonormal set in the complex L²([−π, π]) equipped with the inner product ⟨f, g⟩ = ∫₋π^π f(t)g̅(t) dt (where g̅ is the complex conjugate).

Complex Fourier Coefficients
c_n = \frac{1}{2\pi}\int_{-\pi}^{\pi} f(t)\,e^{-int}\,dt, \qquad f(t) = \sum_{n=-\infty}^{\infty} c_n\,e^{int}
The complex Fourier coefficient cₙ is the inner product of f with the n-th complex exponential basis function. The real and complex forms are related by aₙ = 2Re(cₙ) and bₙ = −2Im(cₙ). Complex notation unifies cosine and sine modes into a single compact formula indexed over all integers.

Convergence and the Gibbs Phenomenon

L² convergence — the norm of the error going to zero — is a global statement about average accuracy. For smooth functions, the Fourier series also converges uniformly (the error goes to zero everywhere). For functions with jump discontinuities, the story is more subtle.

Pointwise Convergence

The Dirichlet theorem states: if f is piecewise smooth on [−π, π], then the Fourier series converges at every point of continuity to f(t), and at each jump discontinuity x₀ to the average (f(x₀⁺) + f(x₀⁻))/2. This is a pointwise statement, stronger than L² convergence but requiring more smoothness.

Gibbs Phenomenon

Near a jump discontinuity, the partial sums overshoot by approximately 9% of the jump height, no matter how many terms are included. This overshoot — the Gibbs phenomenon — is a fundamental limitation: the partial sum Sₙf is the best L² approximation from Vₙ, but "best in L²" does not mean "small everywhere." Adding more terms moves the overshoot closer to the jump but does not eliminate it.

Engineering Implication

The Gibbs phenomenon explains why ideal lowpass filtering (which sets all Fourier coefficients beyond frequency N to zero) produces ringing at sharp signal edges. In practice, windowed Fourier methods (Hamming, Hanning, Blackman windows) taper the coefficients smoothly to zero, trading some approximation accuracy for elimination of ringing. This tradeoff is the frequency-domain incarnation of orthogonal projection onto a smoothed subspace.

The Discrete Fourier Transform

In practice, signals are sampled at N discrete time points t_k = 2πk/N for k = 0, 1, …, N−1. The N-dimensional complex vector space ℂᴺ has its own discrete analogue of the trigonometric system: the DFT basis vectors w_n = (1, ωⁿ, ω²ⁿ, …, ω^((N−1)n))/√N where ω = e^(i2π/N).

DFT as Matrix-Vector Product
X[k] = \sum_{n=0}^{N-1} x[n]\,\omega^{kn}, \quad \omega = e^{-i2\pi/N}, \qquad \mathbf{X} = F\mathbf{x},\; F F^* = I
The DFT matrix F has entries F_{kn} = ω^(kn)/√N. It is a unitary matrix: F*F = I. Computing the DFT is therefore equivalent to changing to an orthonormal basis in ℂᴺ. The FFT algorithm computes this matrix-vector product in O(N log N) instead of O(N²) by exploiting the recursive structure of the DFT matrix.

The DFT is exactly the finite-dimensional version of the Fourier series. The signal vector x ∈ ℂᴺ plays the role of f ∈ L², and the DFT coefficients X = Fx are the coordinates of x in the DFT orthonormal basis. Parseval's theorem in this setting becomes ‖X‖² = ‖x‖², the preservation of the Euclidean norm under unitary transformation.

Applications Revisited Through Linear Algebra

Spectral Analysis

Computing the power spectral density of a signal is computing ‖Px‖² for each projection Px onto the subspace spanned by a single DFT basis vector. The power spectrum is the squared magnitude of the coordinates in the DFT basis. Linear algebra makes clear why power spectra are non-negative (they are squared norms) and why they add across orthogonal subspaces (Parseval).

Circular Convolution as Diagonal Multiplication

In the DFT basis, circular convolution becomes pointwise multiplication: F(x ⊛ h) = √N · (Fx) ⊙ (Fh), where ⊙ is the componentwise product. The √N is the price of the unitary 1/√N normalization we adopted for F; with the unnormalized DFT the factor disappears and the identity reads F(x ⊛ h) = (Fx) ⊙ (Fh), which is the form you will usually see quoted. Either way, the structural fact is the same, and it holds because the DFT basis vectors are the eigenvectors of every circular convolution operator. Applying a filter is therefore: (1) project onto DFT basis, (2) scale each component by the filter's frequency response, (3) project back. This is diagonalization — the most important operation in linear algebra, now applied to signal processing.

Convolution Theorem
F(\mathbf{x} \circledast \mathbf{h}) = \sqrt{N}\,(F\mathbf{x}) \odot (F\mathbf{h}) \quad\Longleftrightarrow\quad Y[k] = H[k]\,X[k]
Circular convolution in the time domain equals pointwise (Hadamard) multiplication in the frequency domain. The √N appears because we normalized F by 1/√N to make it unitary; the right-hand form Y[k] = H[k]X[k] is stated in terms of the unnormalized DFTs, where the factor is absorbed. This is the eigenvalue-eigenvector relationship: the complex exponentials eⁱⁿᵗ are eigenfunctions of any linear time-invariant operation, and the filter's frequency response H(n) is the corresponding eigenvalue. Filtering is diagonalization.

OFDM in One Equation

OFDM modulates independent data symbols X[0], X[1], …, X[N−1] onto the DFT basis vectors, then transmits the time-domain signal x = F⁻¹X. At the receiver, the channel applies a circular convolution with h. The received signal y = h * x + noise satisfies Fy = (Fh) ⊙ (Fx) + F(noise). After removing the cyclic prefix, the receiver applies F, yielding Y[k] = H[k] X[k] + W[k] — N independent scalar channels, each with its own complex gain H[k]. This one-tap equalization is possible precisely because the DFT diagonalizes circular convolution.

Matched Filtering as Inner Product

In radar and communications, a matched filter computes the inner product ⟨received signal, template⟩ for each possible template (delay, Doppler shift). The output is the projection of the received signal onto the template. Choosing the template that maximizes this inner product is exactly maximum likelihood detection — finding the closest point in the signal space to the observed data.

Key Takeaways

The Fourier series is orthogonal projection: each coefficient aₙ or bₙ is an inner product in L², and the N-th partial sum is the closest element of the N-harmonic subspace to f. Complex exponentials form a complete orthonormal basis; the DFT is the finite-dimensional version with a unitary change-of-basis matrix. Circular convolution is diagonalized by the DFT — filtering is changing basis, scaling by eigenvalues, and changing back. The Gibbs phenomenon shows that L² optimality does not mean pointwise accuracy at discontinuities. This unification — Fourier analysis as linear algebra — is the conceptual bridge between abstract inner product theory and practical signal processing.