Inner Products

Beyond the Dot Product

You already know the dot product: u · v = u₁v₁ + u₂v₂ + … + uₙvₙ. It lets you compute lengths (‖u‖ = √(u · u)) and angles (cos θ = u·v / ‖u‖‖v‖), and it defines orthogonality (u · v = 0). But why must we restrict ourselves to this one formula?

An inner product is an abstraction: any operation ⟨·,·⟩ on a vector space that satisfies four axioms gets to play the same role as the dot product. The axioms capture the essential algebraic properties without fixing a specific formula. This lets us measure angles between polynomials, between functions, even between matrices — and do geometry in infinite-dimensional spaces.

The Four Axioms

Let V be a real vector space. A function ⟨·,·⟩ : V × V → ℝ is an inner product on V if it satisfies, for all u, v, w ∈ V and scalars α:

Inner Product Axioms

Four axioms, in order: linearity in the first argument, symmetry (commutativity), non-negativity, and definiteness. The last two together are called positive definiteness; they rule out trivially zero operations and ensure the induced norm is genuine.

These four properties are exactly what you need to do geometry. Linearity lets you distribute and scale. Symmetry means order doesn't matter. Non-negativity means no vector has an imaginary "length". And definiteness guarantees that ⟨v,v⟩ = 0 implies v = 0 — the only vector with zero "length" is the zero vector.

Complex Inner Products

For complex vector spaces, symmetry is replaced by conjugate symmetry: ⟨u,v⟩ = ⟨v,u⟩* (complex conjugate). Linearity stays in the first argument, and the second becomes conjugate-linear: ⟨u, αv⟩ = α*⟨u,v⟩. The inner product used throughout this course is ⟨u,v⟩ = Σ uₖ vₖ* = v*u — the second argument is conjugated. (Physics texts adopt the mirror-image convention and conjugate the first argument; both are in common use, so always check which one a source means.) We focus on real spaces here, where the two coincide.

The Norm Induced by an Inner Product

Every inner product induces a norm — a way to measure the length of vectors:

Induced Norm

\|\mathbf{v}\| = \sqrt{\langle \mathbf{v},\, \mathbf{v} \rangle}

The norm ‖v‖ is the square root of the inner product of v with itself. Positive definiteness of the inner product guarantees ‖v‖ ≥ 0, with equality only when v = 0. For the standard dot product this gives the familiar Euclidean length.

Not every norm comes from an inner product — for example, the ℓ¹ norm (sum of absolute values) does not. A norm arises from an inner product if and only if it satisfies the parallelogram law: ‖u+v‖² + ‖u−v‖² = 2‖u‖² + 2‖v‖².

Distance and the Metric

The norm induces a distance d(u, v) = ‖u − v‖. This makes any inner product space a metric space. Convergence, continuity, and limits all become meaningful — a crucial step toward infinite-dimensional analysis.

Cauchy-Schwarz and Triangle Inequalities

Two foundational inequalities follow from the axioms alone — no specific formula needed:

Cauchy-Schwarz Inequality

|\langle \mathbf{u},\, \mathbf{v} \rangle| \leq \|\mathbf{u}\|\,\|\mathbf{v}\|

The absolute value of the inner product never exceeds the product of the norms. Equality holds if and only if u and v are parallel (one is a scalar multiple of the other). This is the most important inequality in all of analysis.

The Cauchy-Schwarz inequality makes the definition of angle coherent: it guarantees that |⟨u,v⟩| / (‖u‖‖v‖) ≤ 1, so we can legitimately define:

Angle Between Vectors

\cos\theta = \frac{\langle \mathbf{u},\, \mathbf{v} \rangle}{\|\mathbf{u}\|\,\|\mathbf{v}\|}

The angle θ ∈ [0, π] between two nonzero vectors. Two vectors are orthogonal (⟨u,v⟩ = 0) precisely when θ = π/2. This definition works in any inner product space — including function spaces.

The triangle inequality follows from Cauchy-Schwarz and states ‖u + v‖ ≤ ‖u‖ + ‖v‖ — the straight-line distance is never longer than the sum of the two sides. It is what makes distance a genuine metric.

Inner Products on Function Spaces

Here is where the abstraction pays off. Consider the space of square-integrable functions on an interval [a, b]. We define:

L² Inner Product

\langle f,\, g \rangle = \int_a^b f(t)\,g(t)\,dt

This satisfies all four axioms. The induced norm ‖f‖ = √∫f²(t)dt is the RMS (root-mean-square) value of f. Two functions are orthogonal when their integral product is zero — a statement about their "similarity" over the whole interval.

This inner product gives functions the same geometric vocabulary we use for vectors. A sine and cosine of the same frequency are orthogonal on [0, 2π]. A set of orthogonal functions can serve as a basis — each component is extracted via an inner product, exactly as dot products extract coordinates in ℝⁿ.

Weighted Inner Products

We can generalize further: ⟨f, g⟩_w = ∫w(t)f(t)g(t)dt for a non-negative weight function w(t). Different weights give rise to different families of orthogonal polynomials — Legendre polynomials use w = 1 on [−1,1], Hermite polynomials use w = e^(−t²) on ℝ, Laguerre polynomials use w = e^(−t) on [0,∞). Each family is the "natural" basis for its weighted space.

Inner Products on Matrices

The Frobenius inner product ⟨A, B⟩_F = tr(AᵀB) = Σᵢⱼ AᵢⱼBᵢⱼ treats matrices as long vectors. It gives rise to the Frobenius norm ‖A‖_F = √tr(AᵀA). This inner product appears in low-rank approximation and matrix completion problems.

Orthogonality and Projections in Inner Product Spaces

Once we have an inner product, we can project any vector onto any subspace — not just column spaces of matrices. The projection of v onto a subspace W is the unique element p ∈ W such that (v − p) ⊥ W, i.e., ⟨v − p, w⟩ = 0 for all w ∈ W.

For a subspace spanned by an orthonormal set {e₁, e₂, …, eₖ}, the projection is simply:

Projection onto Orthonormal Basis

\mathrm{proj}_W \mathbf{v} = \sum_{i=1}^{k} \langle \mathbf{v},\, \mathbf{e}_i \rangle\, \mathbf{e}_i

Each coefficient ⟨v, eᵢ⟩ measures "how much of eᵢ is in v." This is the abstract generalization of coordinate extraction. Fourier coefficients are exactly these inner products with the sine/cosine basis functions.

Applications in Signal Processing and Communications

Matched Filtering

A matched filter computes the inner product ⟨r(t), s(t)⟩ = ∫r(t)s(t)dt between the received signal r and a template s. By Cauchy-Schwarz, this is the optimal detector for a known signal in additive white Gaussian noise — it maximizes signal-to-noise ratio.

Correlation and Spectral Analysis

The cross-correlation of two signals is an inner product parameterized by a time lag. In the frequency domain, inner products between signals and complex exponentials give the Fourier transform coefficients — showing the "angle" between the signal and each frequency component.

OFDM and Orthogonal Subcarriers

Orthogonal Frequency Division Multiplexing (OFDM) rests entirely on the orthogonality of complex exponentials e^(j2πnΔft) under the L² inner product. Each subcarrier is orthogonal to all others over one symbol period, enabling parallel transmission without inter-carrier interference — all because of inner product structure.

Key Takeaways

An inner product ⟨·,·⟩ on a vector space must satisfy: linearity in the first argument, symmetry, and positive definiteness. It induces a norm ‖v‖ = √⟨v,v⟩ and a distance d(u,v) = ‖u−v‖. The Cauchy-Schwarz inequality |⟨u,v⟩| ≤ ‖u‖‖v‖ makes the angle definition rigorous. The L² inner product ⟨f,g⟩ = ∫f(t)g(t)dt extends all of this to function spaces. Orthogonality, projection, and basis decomposition work exactly as in ℝⁿ — the inner product is the universal language of geometry.