Finding Eigenvectors

From Eigenvalues to Eigenvectors

In the previous lesson we learned to find eigenvalues by solving the characteristic equation det(A − λI) = 0. Now comes the natural next step: for each eigenvalue λ, find the nonzero vectors v that satisfy Av = λv. Rearranging this gives (A − λI)v = 0 — a homogeneous linear system. The solutions are the eigenvectors, and they form the null space of (A − λI).

The null space of (A − λI) is called the eigenspace corresponding to λ, written E_λ or sometimes ker(A − λI). It always contains the zero vector (trivially), but eigenvectors are the nonzero elements. The eigenspace is a subspace — it is closed under addition and scalar multiplication.

Eigenspace Definition

E_{\lambda} = \ker(A - \lambda I) = \{\mathbf{v} : (A - \lambda I)\mathbf{v} = \mathbf{0}\}

The eigenspace E_λ is the null space of (A − λI). To find it, row-reduce (A − λI) and express the solutions in terms of free variables. Each free variable yields one basis eigenvector.

The Procedure: Step by Step

To find eigenvectors for a given eigenvalue λ:

Form the matrix (A − λI). Subtract the eigenvalue λ from every diagonal entry of A, leaving all off-diagonal entries unchanged.
Row-reduce (A − λI). Apply Gaussian elimination to obtain the reduced row echelon form (RREF). Because λ is an eigenvalue, this matrix is singular — you will always get at least one free variable.
Express pivot variables in terms of free variables. Write out the system of equations and solve for each pivot variable.
Write the general solution. The solution vector, expressed in terms of free variables, gives you the eigenvectors — one basis vector per free variable.

A Complete Example

Let's work with the matrix A = [[3, 1], [0, 2]], whose eigenvalues we found in the previous lesson: λ₁ = 3 and λ₂ = 2.

Eigenvectors for λ₁ = 3: Form A − 3I:

A − 3I = [[3−3, 1], [0, 2−3]] = [[0, 1], [0, −1]]

Row-reducing: the second row is −1 times the first, so eliminate it: [[0, 1], [0, 0]]. The system is 0·x₁ + 1·x₂ = 0, so x₂ = 0. The variable x₁ is free. Setting x₁ = 1 gives the eigenvector v₁ = [1, 0]ᵀ. Any nonzero multiple of this vector is also an eigenvector for λ = 3.

Eigenvectors for λ = 3

E_3 = \text{span}\left\{\begin{bmatrix}1\\0\end{bmatrix}\right\}

Every nonzero scalar multiple of [1, 0]ᵀ is an eigenvector for λ = 3. The eigenspace E₃ is the x-axis — a one-dimensional subspace of ℝ².

Eigenvectors for λ₂ = 2: Form A − 2I:

A − 2I = [[3−2, 1], [0, 2−2]] = [[1, 1], [0, 0]]

The system is x₁ + x₂ = 0, so x₁ = −x₂. The variable x₂ is free. Setting x₂ = 1 gives x₁ = −1, so the eigenvector is v₂ = [−1, 1]ᵀ. The eigenspace E₂ is the line through [−1, 1]ᵀ.

Eigenvectors for λ = 2

E_2 = \text{span}\left\{\begin{bmatrix}-1\\1\end{bmatrix}\right\}

The eigenspace E₂ is the span of [−1, 1]ᵀ — the line at 135° from the positive x-axis. Every nonzero vector on this line gets scaled by 2 when A acts on it.

Geometric Multiplicity

The geometric multiplicity of an eigenvalue λ is the dimension of its eigenspace — equivalently, the number of linearly independent eigenvectors for that eigenvalue. Geometrically, it is the number of independent "special directions" associated with λ.

A key inequality governs the relationship between algebraic and geometric multiplicity:

Multiplicity Inequality

1 \leq \text{geo. mult.}(\lambda) \leq \text{alg. mult.}(\lambda)

The geometric multiplicity is always at least 1 (since λ is an eigenvalue, the eigenspace is nontrivial) and never exceeds the algebraic multiplicity. Equality (geometric = algebraic) for every eigenvalue is the condition for diagonalizability.

Defective Eigenvalues

When the geometric multiplicity of an eigenvalue is strictly less than its algebraic multiplicity, the eigenvalue is called defective. For example, the matrix [[2, 1], [0, 2]] has a repeated eigenvalue λ = 2 with algebraic multiplicity 2 but only one independent eigenvector [1, 0]ᵀ — geometric multiplicity 1. Defective matrices cannot be diagonalized, requiring the more general Jordan normal form.

Linear Independence of Eigenvectors

One of the most beautiful theorems in linear algebra: eigenvectors from distinct eigenvalues are always linearly independent. If λ₁ ≠ λ₂ ≠ … ≠ λₖ are distinct eigenvalues with eigenvectors v₁, v₂, …, vₖ, then the set {v₁, v₂, …, vₖ} is linearly independent.

Why does this matter? For an n×n matrix with n distinct eigenvalues, we automatically get n linearly independent eigenvectors, forming a basis for ℝⁿ. This is the ideal case for diagonalization (covered in the next lesson).

Proof Sketch

Suppose c₁v₁ + c₂v₂ = 0 for eigenvectors v₁, v₂ with distinct eigenvalues λ₁, λ₂. Apply A to both sides: c₁λ₁v₁ + c₂λ₂v₂ = 0. Subtract λ₂ times the original equation: c₁(λ₁ − λ₂)v₁ = 0. Since λ₁ ≠ λ₂ and v₁ ≠ 0, we get c₁ = 0. Similarly c₂ = 0. The argument extends by induction to any number of distinct eigenvalues.

Complex Eigenvalues and Eigenvectors

Real matrices can have complex eigenvalues. When they occur, they always come in conjugate pairs: if λ = a + bi is an eigenvalue, then so is λ̄ = a − bi. Their eigenvectors are also complex conjugates of each other.

Geometrically, complex eigenvalues correspond to rotation-and-scaling transformations. A 2×2 rotation matrix by angle θ has eigenvalues e^{iθ} and e^{−iθ} — no real eigenvectors exist because a rotation of the plane preserves no real direction (unless θ = 0 or π).

This is a statement about the plane, not about rotations in general. A rotation of ℝ³ by angle θ about an axis u has eigenvalues 1, e^{iθ}, e^{−iθ}: the axis itself satisfies Ru = u, so a real 3×3 rotation always has a real eigenvector, the one at λ = 1. Only in the plane is there no axis left over to fix.

Example: Rotation Matrix

The matrix R = [[cos θ, −sin θ], [sin θ, cos θ]] has characteristic polynomial λ² − 2cos(θ)λ + 1 = 0, with roots e^{±iθ}. The complex eigenvectors are [1, ∓i]ᵀ — these are complex directions in ℂ² that R maps to rotated versions of themselves.

When Eigenspaces Have Higher Dimension

For symmetric matrices (and more generally for normal matrices), eigenspaces can have dimension greater than 1. The identity matrix I is an extreme case: every nonzero vector is an eigenvector with eigenvalue 1, so E₁ = ℝⁿ — dimension n.

A more practical example: a symmetric 3×3 matrix might have eigenvalues λ₁ = 5 (with a 1D eigenspace) and λ₂ = 2 (with a 2D eigenspace, meaning two independent eigenvectors). The 2D eigenspace is a plane in ℝ³ — every vector in that plane gets scaled by 2 when A acts on it.

Eigenspace Basis

\dim(E_{\lambda}) = n - \text{rank}(A - \lambda I)

The dimension of the eigenspace E_λ equals the number of free variables after row-reducing (A − λI). A basis for E_λ is found by setting each free variable to 1 and all others to 0, producing one basis vector per free variable.

Key Fact for Symmetric Matrices

Real symmetric matrices (Aᵀ = A) have a spectacular property: not only are all eigenvalues real, but eigenvectors from different eigenspaces are always orthogonal. This means the eigenspaces partition ℝⁿ into perpendicular subspaces — a complete orthogonal decomposition. This is the Spectral Theorem, and it underlies PCA, quantum mechanics, and many other applications.

Key Takeaways

To find eigenvectors for eigenvalue λ, row-reduce (A − λI) and find the null space — the eigenspace E_λ. Each free variable yields one basis eigenvector. The geometric multiplicity (dimension of E_λ) is always between 1 and the algebraic multiplicity. Eigenvectors from distinct eigenvalues are always linearly independent. Complex eigenvalues come in conjugate pairs and correspond to rotation-scaling behavior. For symmetric matrices, eigenspaces from different eigenvalues are orthogonal — the Spectral Theorem.