Function Spaces

Functions as Vectors

In earlier modules, vectors were lists of numbers — elements of ℝⁿ. The ten vector space axioms (closure under addition and scalar multiplication, associativity, commutativity, identity, inverses, and the four distributive/compatibility laws) were stated for ℝⁿ but are satisfied by a much wider class of objects.

Consider the set of all continuous functions on an interval [a, b]. Given two continuous functions f and g, we can form f + g (pointwise addition) and αf (pointwise scaling) — and both are again continuous. All ten axioms hold. So continuous functions on [a, b] form a vector space, which we call C([a, b]).

This is not a curiosity. It means every concept we built for ℝⁿ — linear combinations, bases, dimension, orthogonality, projections — either carries over directly or has a natural infinite-dimensional analogue.

The L² Space

For signal processing and physics, the most important function space is L²([a, b]): the set of all functions f such that the integral of f² is finite.

The L² Space

L^2([a,b]) = \left\{ f : [a,b] \to \mathbb{R} \;\Big|\; \int_a^b |f(t)|^2\,dt < \infty \right\}

A function belongs to L²([a, b]) precisely when its "energy" — the integral of its square — is finite. This includes all continuous functions on a closed interval, all piecewise-continuous functions, and many more. It excludes functions that blow up too wildly, like 1/x near 0.

L²([a, b]) is a vector space: the sum of two finite-energy functions has finite energy (by the triangle inequality for integrals), and scaling a finite-energy function keeps it in L². Crucially, it carries the L² inner product ⟨f, g⟩ = ∫f(t)g(t)dt that we introduced in Lesson 1, making it an inner product space.

Why L² and Not Something Else?

Physical signals have finite energy: a transmitted waveform cannot carry infinite power. The condition ∫f²(t)dt < ∞ directly captures this. Moreover, L² is the unique function space that is both an inner product space and complete (every Cauchy sequence converges inside it) — making it a Hilbert space, the infinite-dimensional analogue of Euclidean space.

Hilbert Spaces

A Hilbert space is a complete inner product space. Completeness means that sequences that "should converge" (Cauchy sequences) always converge to a limit that is still in the space. ℝⁿ is trivially complete; L²([a, b]) is the infinite-dimensional completion of the space of continuous functions under the L² norm.

Completeness Matters

The space C([a, b]) of continuous functions with the L² inner product is not complete: you can build Cauchy sequences of continuous functions that converge to a discontinuous limit (e.g., the square wave as a limit of smooth approximations). L² is the right space because it includes those limit functions, giving a closed arena in which analysis works.

Orthogonal Functions and Function Bases

In ℝⁿ, an orthonormal basis {e₁, …, eₙ} lets us represent any vector as v = Σ ⟨v, eᵢ⟩ eᵢ. Exactly the same works in L²: an orthonormal sequence {φ₁, φ₂, …} in L² satisfies ⟨φₘ, φₙ⟩ = δₘₙ (1 if m = n, 0 otherwise). If this sequence is also complete (no nonzero function is orthogonal to all of them), then any f ∈ L² can be written:

Generalized Fourier Series

f = \sum_{n=1}^{\infty} c_n \varphi_n, \qquad c_n = \langle f,\, \varphi_n \rangle = \int_a^b f(t)\,\varphi_n(t)\,dt

The coefficients cₙ = ⟨f, φₙ⟩ are exactly the inner products of f with each basis function — the direct analogue of extracting coordinates via dot products in ℝⁿ. The series converges in the L² sense (convergence of the norm of the error to zero).

The Trigonometric System

The most famous orthogonal system in L²([0, 2π]) is the trigonometric system: the functions {1/√(2π), cos(nt)/√π, sin(nt)/√π} for n = 1, 2, 3, … are mutually orthogonal and form a complete orthonormal set. Expanding f in this basis gives the classical Fourier series.

Fourier Coefficients as Inner Products

a_n = \langle f,\, \tfrac{\cos(nt)}{\sqrt{\pi}} \rangle = \frac{1}{\pi}\int_0^{2\pi} f(t)\cos(nt)\,dt

The Fourier coefficient aₙ is the L² inner product of f with cos(nt)/√π — measuring how much of the nth cosine mode is present in f. This is precisely a projection in L². The inner product framework makes Fourier analysis just "linear algebra in infinite dimensions."

Legendre and Hermite Polynomials

The trigonometric functions are not the only choice. On the interval [−1, 1] with the standard L² inner product, the Legendre polynomials {P₀, P₁, P₂, …} form a complete orthogonal system. They arise naturally in solving differential equations with spherical symmetry (e.g., in antenna analysis and quantum mechanics).

On (−∞, ∞) with the weighted inner product ⟨f, g⟩ = ∫e^(−t²)f(t)g(t)dt, the Hermite polynomials play the same role. They appear in the quantum harmonic oscillator and in Gaussian channel analysis in communications. Each family is the "natural" basis for its specific function space.

Parseval's Theorem and Energy Conservation

When an orthonormal basis fully spans L², the norm of f equals the ℓ² norm of its coefficients:

Parseval's Theorem

\|f\|^2 = \int_a^b |f(t)|^2\,dt = \sum_{n=1}^{\infty} |c_n|^2

The total energy of a signal equals the sum of squares of its Fourier (or generalized Fourier) coefficients. This is the infinite-dimensional Pythagorean theorem: the norm is preserved when passing to an orthonormal basis. In signal processing, this is the statement that a signal's time-domain energy equals its frequency-domain energy.

Parseval's theorem is the infinite-dimensional Pythagorean theorem. It says that an isometric isomorphism exists between L²([a, b]) and the sequence space ℓ² (square-summable sequences) — the two spaces are geometrically identical, just described with different "coordinates."

The Sampling Connection

Parseval's theorem underpins the Nyquist-Shannon sampling theorem: a bandlimited signal (one whose Fourier coefficients beyond frequency W are zero) can be reconstructed exactly from samples at rate 2W. Sampling is nothing more than reading off specific inner products of the signal with sinc functions — a finite set of projections that captures all the signal's content.

Subspaces of L²

Function spaces have rich subspace structure. Some important examples:

Bandlimited signals: functions whose Fourier transform vanishes outside [−W, W]. This is a closed subspace of L²(ℝ), and orthogonal projection onto it is exactly ideal lowpass filtering.
Polynomials of degree ≤ n: a finite-dimensional subspace. The least-squares polynomial fit is the orthogonal projection of a function onto this subspace under the L² inner product.
Even (or odd) functions: closed subspaces of L²([−π, π]). Any function decomposes uniquely into its even and odd parts — an orthogonal direct sum.

Projection and Best Approximation

The projection theorem for Hilbert spaces says: for any closed subspace W ⊂ L² and any function f, there is a unique best approximation p ∈ W minimizing ‖f − p‖. This p is the orthogonal projection of f onto W, and (f − p) ⊥ W. The Fourier series partial sum is exactly this projection onto the subspace spanned by the first N basis functions.

Best Approximation in L²

S_N f = \sum_{n=1}^{N} \langle f,\, \varphi_n \rangle\, \varphi_n = \underset{g \in \mathrm{span}\{\varphi_1,\ldots,\varphi_N\}}{\arg\min}\, \|f - g\|

The N-term partial sum Sₙf minimizes the L² error ‖f − g‖ over all linear combinations g of {φ₁, …, φₙ}. More terms means a better approximation; completeness guarantees the error goes to zero as N → ∞.

Applications in Signal Processing and Communications

OFDM Subcarrier Decomposition

OFDM decomposes a wideband channel into N narrow subchannels by projecting the received signal onto N orthogonal complex exponentials. Each projection is an inner product — a single point in the DFT output. The orthogonality of the subcarriers (in the L² sense) guarantees that these projections are independent, enabling parallel data recovery without inter-carrier interference.

Channel Estimation via Basis Expansion

A time-varying channel impulse response h(t, τ) can be expanded in a basis of Doppler-delay functions. Estimating the channel reduces to estimating the expansion coefficients — a finite set of inner products. The choice of basis (and hence the function subspace) determines the tradeoff between estimation accuracy and pilot overhead.

Wavelet Analysis

The Fourier basis is frequency-localized but not time-localized. Wavelets provide orthonormal bases for L²(ℝ) that are simultaneously localized in both time and frequency. The wavelet expansion coefficients ⟨f, ψⱼₖ⟩ capture the signal's local frequency content at time 2⁻ʲk and scale 2⁻ʲ. This makes wavelets the tool of choice for compression (JPEG 2000) and denoising, where signal features are sparse in a wavelet basis.

Key Takeaways

Functions form vector spaces: C([a, b]) and L²([a, b]) satisfy all ten vector space axioms. L² is a Hilbert space — an inner product space that is also complete. An orthonormal basis {φₙ} in L² yields the generalized Fourier expansion f = Σ ⟨f, φₙ⟩ φₙ, with Parseval's theorem ‖f‖² = Σ |⟨f, φₙ⟩|² preserving energy. Orthogonal projection onto a subspace gives the best approximation — the same geometry as in ℝⁿ, now in infinite dimensions. This framework unifies Fourier analysis, spectral estimation, optimal filtering, and channel representation under a single algebraic roof.