The Eigenvalue Problem

A Special Direction

Most vectors are transformed in two ways when a matrix acts on them: they get rotated and stretched (or shrunk). But there are special vectors where something remarkable happens — the matrix only stretches or shrinks them, leaving their direction completely unchanged. These are called eigenvectors, and the scaling factor is called the eigenvalue.

Formally, a nonzero vector v is an eigenvector of a square matrix A if there exists a scalar λ (lambda) such that:

The Eigenvalue Equation

A\mathbf{v} = \lambda\mathbf{v}

A acts on v and produces λv — the same vector v, just scaled by λ. If λ = 2, the vector doubles in length. If λ = −1, it flips direction. If λ = 0, the vector collapses to the zero vector (but v itself must be nonzero).

This equation is called the eigenvalue problem. The scalar λ is the eigenvalue and the nonzero vector v is the associated eigenvector. Together, (λ, v) is an eigenpair.

Geometric Intuition

To understand why eigenvectors matter, think about what a matrix does geometrically. A rotation matrix spins every vector by the same angle — no direction is preserved, so a pure rotation has no real eigenvectors. A scaling matrix stretches everything along the coordinate axes — those axes are eigenvectors. A shear matrix is more subtle: only one direction stays fixed.

For a general 2×2 matrix, the eigenvectors are the natural axes of the transformation. They are the directions the transformation "respects." If you align your coordinate system with the eigenvectors, the matrix looks like a simple diagonal scaling — it just multiplies each axis by its eigenvalue.

A Concrete Picture

Consider the matrix A = [[3, 1], [0, 2]]. Apply it to the vector [1, 0]ᵀ: the result is [3, 0]ᵀ — the same direction, scaled by 3. So [1, 0]ᵀ is an eigenvector with eigenvalue λ = 3. Apply A to [1, −1]ᵀ: the result is [2, −2]ᵀ = 2 · [1, −1]ᵀ — same direction, scaled by 2. So [1, −1]ᵀ is an eigenvector with eigenvalue λ = 2. Every other direction gets both rotated and stretched.

Why "Eigen"?

The word "eigen" comes from German and means "own" or "characteristic." Eigenvectors are the matrix's own special directions — the directions that belong to it intrinsically, the ones it acts on most simply.

The Characteristic Equation

To find the eigenvalues of a matrix, we start from the eigenvalue equation Av = λv and rearrange:

Av = λv → Av − λv = 0 → Av − λIv = 0 → (A − λI)v = 0

This is a homogeneous linear system in v. For a nonzero solution to exist, the matrix (A − λI) must be singular — it cannot be invertible, otherwise the only solution would be v = 0. A matrix is singular if and only if its determinant is zero. This gives us the characteristic equation:

Characteristic Equation

\det(A - \lambda I) = 0

This equation must hold for λ to be an eigenvalue. Expanding the determinant of (A − λI) produces a polynomial in λ — the characteristic polynomial. Its roots are the eigenvalues of A.

The expression det(A − λI) is a polynomial of degree n in λ, where n is the size of the matrix. For a 2×2 matrix, it is a quadratic. For a 3×3, a cubic. And so on. Finding the eigenvalues reduces to solving this polynomial equation.

Finding Eigenvalues: Step by Step

Let's work through a complete example. Take the upper-triangular matrix:

A = [[3, 1], [0, 2]]

Step 1: Form A − λI. Subtract λ from each diagonal entry:

A − λI = [[3−λ, 1], [0, 2−λ]]

Step 2: Set det(A − λI) = 0. For a 2×2 triangular matrix, the determinant is simply the product of the diagonal entries:

Characteristic Polynomial

(3-\lambda)(2-\lambda) = \lambda^2 - 5\lambda + 6 = (\lambda-3)(\lambda-2)

The characteristic polynomial factors neatly into (λ−3)(λ−2). The roots are the eigenvalues: λ₁ = 3 and λ₂ = 2. Note that for a triangular matrix, the eigenvalues are always the diagonal entries — this is a general rule.

Step 3: Identify the eigenvalues. The roots of (λ−3)(λ−2) = 0 are λ₁ = 3 and λ₂ = 2. These are the two eigenvalues of A.

For 3×3 Matrices

For an n×n matrix, the characteristic polynomial has degree n. A 3×3 matrix has a cubic characteristic polynomial, yielding up to 3 eigenvalues. By the Fundamental Theorem of Algebra, a degree-n polynomial has exactly n roots in the complex numbers (counting multiplicity). So every n×n matrix has exactly n eigenvalues — some may be complex, some may repeat.

Algebraic Multiplicity

A polynomial root may appear more than once. The algebraic multiplicity of an eigenvalue λ₀ is the number of times (λ − λ₀) appears as a factor of the characteristic polynomial.

For example, if the characteristic polynomial of a 3×3 matrix is (λ−2)²(λ−5), then λ = 2 has algebraic multiplicity 2 and λ = 5 has algebraic multiplicity 1. The total count is 2 + 1 = 3 = n. This is always the case: the sum of all algebraic multiplicities equals n.

Real vs. Complex Eigenvalues

Not every real matrix has real eigenvalues. A rotation matrix by 90° has characteristic polynomial λ² + 1 = 0, whose roots are ±i — purely imaginary. However, real symmetric matrices (where Aᵀ = A) always have real eigenvalues. This is a crucial theorem in spectral theory, with deep implications for physics and data science.

Key Properties of Eigenvalues

Two elegant identities connect the eigenvalues to the matrix entries directly:

Eigenvalue Relations

\det(A) = \prod_i \lambda_i, \quad \text{tr}(A) = \sum_i \lambda_i

The determinant of A equals the product of all eigenvalues, and the trace of A (sum of diagonal entries) equals the sum of all eigenvalues. These identities are immediate consequences of expanding the characteristic polynomial and comparing coefficients.

Additional properties worth knowing:

Similar matrices (B = P⁻¹AP for invertible P) share the same eigenvalues — the eigenvalues are a property of the linear transformation, not the particular matrix representation.
Triangular matrices (upper or lower) have their eigenvalues on the diagonal — the characteristic polynomial factors immediately.
Symmetric matrices (Aᵀ = A) always have real eigenvalues and a full set of orthogonal eigenvectors — a property used constantly in data science (PCA) and physics (quantum mechanics).
Singular matrices always have 0 as an eigenvalue, because det(A) = 0 = product of eigenvalues.

A Quick Sanity Check

For our matrix A = [[3, 1], [0, 2]] with eigenvalues λ₁ = 3 and λ₂ = 2: the product is 3 × 2 = 6 = det(A) = 3·2 − 1·0 = 6. The sum is 3 + 2 = 5 = tr(A) = 3 + 2. Both identities check out perfectly.

Key Takeaways

An eigenvector of A is a nonzero vector that A merely scales: Av = λv. The scalar λ is the eigenvalue. To find all eigenvalues, solve the characteristic equation det(A − λI) = 0 — the roots of the resulting polynomial are the eigenvalues. Every n×n matrix has exactly n eigenvalues counting algebraic multiplicity (possibly complex). The product of all eigenvalues equals det(A) and the sum equals tr(A). Real symmetric matrices always have real eigenvalues — a cornerstone of spectral theory.