Orthogonal Matrices

What Makes a Matrix Orthogonal?

In the previous lesson we used Gram-Schmidt to build orthonormal bases — sets of vectors that are mutually perpendicular and each have unit length. Now we ask: what happens when we pack those orthonormal vectors into the columns of a matrix? The result is an orthogonal matrix, and it has one of the most useful properties in all of linear algebra.

A square matrix Q is called orthogonal if its columns form an orthonormal set. The defining condition is deceptively simple: Q^TQ = I. Since Q is square, this also implies QQ^T = I, meaning Q^T = Q^-1. The transpose is the inverse — no Gaussian elimination, no determinant computation, just a transpose.

Defining Property

Q^T Q = I \quad \Longleftrightarrow \quad Q^{-1} = Q^T

Q is orthogonal if and only if Q^TQ = I. This encodes two facts at once: (i,i) entries say each column has unit length; (i,j) entries for i ≠ j say distinct columns are perpendicular. Q^T = Q^-1 follows immediately since Q is square.

Naming Convention

The term "orthogonal matrix" is a historical quirk — what it really means is orthonormal columns (both orthogonal and unit length). Pure orthogonality (zero dot products only) is not enough. Keep this in mind when reading textbooks.

Orthogonal Matrices Preserve Geometry

The most important property of an orthogonal matrix is what it does to vectors: it preserves both lengths and angles. If Q is orthogonal and you transform vectors x and y to Qx and Qy, then:

Lengths are preserved: ‖Qx‖ = ‖x‖ for every vector x.
Angles are preserved: the angle between Qx and Qy equals the angle between x and y.
Dot products are preserved: (Qx)·(Qy) = x·y.

Length and Inner Product Preservation

\|Q\mathbf{x}\|^2 = \mathbf{x}^T Q^T Q \mathbf{x} = \mathbf{x}^T \mathbf{x} = \|\mathbf{x}\|^2, \qquad (Q\mathbf{x})^T(Q\mathbf{y}) = \mathbf{x}^T \mathbf{y}

‖Qx‖² = (Qx)ᵀ(Qx) = xᵀQᵀQx = xᵀIx = ‖x‖². Similarly, (Qx)ᵀ(Qy) = xᵀQᵀQy = xᵀy. Both follow directly from Q^TQ = I.

Geometrically, orthogonal matrices are the rigid motions of linear algebra — they rotate and reflect space without stretching or squishing it. This is why they appear everywhere in physics, computer graphics, signal processing, and numerical computation.

The Inverse Is Free: Q⁻¹ = Qᵀ

For a general n×n matrix A, computing A^-1 costs O(n³) operations. For an orthogonal matrix Q, the inverse is the transpose — an O(n²) operation (or even O(1) if you think of it as just a reindexing). This is the computational miracle of orthogonal matrices.

The Free Inverse

Q^{-1} = Q^T \quad \Rightarrow \quad Q\mathbf{x} = \mathbf{b} \text{ solved by } \mathbf{x} = Q^T \mathbf{b}

Q^-1 = Q^T. Solving Qx = b becomes x = Q^Tb — just a matrix-vector multiply. Solving systems with orthogonal matrices is trivially cheap and perfectly numerically stable.

This property makes orthogonal matrices the preferred factorization in numerical linear algebra. The QR decomposition from the last lesson is powerful precisely because Q^-1 is free — solving QRx = b reduces to back-substitution on R after computing Q^Tb.

Determinant of an Orthogonal Matrix

Since Q^TQ = I, taking determinants on both sides gives det(Q^T)·det(Q) = 1. Since det(Q^T) = det(Q), we get det(Q)² = 1, so det(Q) = ±1. This splits orthogonal matrices into two families:

det(Q) = +1 — proper rotations (orientation-preserving)
det(Q) = −1 — improper rotations: a rotation combined with a reflection (orientation-reversing)

Rotation Matrices

The most familiar orthogonal matrices are 2D rotation matrices. Rotating every vector in the plane by angle θ counterclockwise is a linear transformation represented by:

2D Rotation Matrix

R(\theta) = \begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix}, \qquad R(\theta)^T = R(-\theta) = R(\theta)^{-1}

R(θ) rotates every vector by angle θ counterclockwise. Its columns are [cos θ, sin θ]ᵀ and [−sin θ, cos θ]ᵀ — orthonormal vectors. det(R) = cos²θ + sin²θ = 1, confirming it is a proper rotation.

Verify: R(θ)ᵀR(θ) = I by direct computation. The inverse of a rotation by θ is a rotation by −θ, which equals R(−θ) = R(θ)ᵀ — confirming Q^-1 = Q^T.

In 3D, rotation matrices are similarly orthogonal. A rotation by angle θ about the z-axis extends R(θ) by adding a 1 in the (3,3) position. Composing two rotations Q₁ and Q₂ gives Q₁Q₂, which is again orthogonal: (Q₁Q₂)ᵀ(Q₁Q₂) = Q₂ᵀQ₁ᵀQ₁Q₂ = Q₂ᵀQ₂ = I.

Reflection Matrices

Reflections are the other family of orthogonal matrices (det = −1). Reflecting all vectors across a line through the origin (in 2D) or a plane through the origin (in 3D) is an orthogonal transformation. The general formula for reflection across the hyperplane perpendicular to a unit vector u is the Householder reflection:

Householder Reflection

H = I - 2\mathbf{u}\mathbf{u}^T, \quad \|\mathbf{u}\| = 1, \qquad H^T = H, \quad H^2 = I, \quad \det(H) = -1

H = I − 2uuᵀ reflects vectors across the hyperplane perpendicular to unit vector u. Check: Hᵀ = H (symmetric), H² = I (reflecting twice returns to start), det(H) = −1. Householder reflections are the workhorse of QR decomposition in practice.

Householder vs. Gram-Schmidt QR

Gram-Schmidt builds Q column by column. Householder QR builds Q as a product of reflections H₁H₂…Hₙ, each zeroing one subdiagonal of A. Householder is twice as stable in floating-point and is the algorithm used in LAPACK, MATLAB, and NumPy for large matrices.

Applications in Signal Processing

Orthogonal matrices appear throughout signal processing and communications because they preserve signal energy and enable fast, exact inversion.

The DFT Matrix

The Discrete Fourier Transform (DFT) matrix F has entries Fₖₙ = e^{−j2πkn/N}/√N. With the 1/√N normalization, F is unitary (the complex generalization of orthogonal): F*F = I where F* is the conjugate transpose. So the inverse DFT is F⁻¹ = F*. And because F is symmetric — Fₖₙ depends only on the product kn, so Fᵀ = F — the conjugate transpose collapses to a plain entrywise conjugate: F⁻¹ = F* = F̄, with entries e^{+j2πkn/N}/√N. Frequency to time is the same computation as time to frequency, just conjugated.

Beamforming and MIMO

In antenna arrays, the steering vector matrix is often approximately orthogonal for well-separated directions. Multiplying received signals by Q^T (matched filtering) is the maximum-ratio combining step. In MIMO communications, the precoding matrix Q rotates the transmitted signal into the eigenvectors of the channel — a rotation that does not change the power of the signal.

Wavelet and Transform Coding

The JPEG-style DCT (Discrete Cosine Transform) uses an orthogonal matrix. Transforming an image block Ax = b by an orthogonal transform Q preserves the total signal energy (‖Qb‖ = ‖b‖) while concentrating it in fewer large coefficients. The inverse is free (Q^T), so reconstruction is exact and cheap.

The Orthogonal Group O(n)

The set of all n×n orthogonal matrices forms a group under matrix multiplication — called the orthogonal group O(n). The subgroup with det = +1 is the special orthogonal group SO(n), which contains all proper rotations. These groups are fundamental in physics (symmetry groups of space), robotics (joint rotations), and machine learning (equivariant networks).

Key group properties:

Closure: the product of two orthogonal matrices is orthogonal.
Identity: the identity matrix I is orthogonal.
Inverse: the inverse Q^-1 = Q^T is orthogonal.
Associativity: matrix multiplication is always associative.

Key Takeaways

A square matrix Q is orthogonal if Q^TQ = I — its columns are orthonormal. Orthogonal matrices preserve lengths and angles: ‖Qx‖ = ‖x‖ and (Qx)·(Qy) = x·y. The inverse is free: Q^-1 = Q^T. det(Q) = ±1, splitting into rotations (+1) and reflections (−1). Rotation matrices R(θ) and Householder reflections H = I − 2uuᵀ are canonical examples. In signal processing, orthogonal transforms (DFT, DCT, Householder QR) preserve energy and invert at zero cost.