Matrix Operations
Knowing what a matrix is is just the start. Now we learn what we can do with them — the operations that make matrices the engine of modern science.
Matrix Addition
Add corresponding entries. Both matrices must have the same dimensions. The result has the same shape. Subtraction works identically. Simple, component-wise.
Scalar Multiplication
Multiply every entry by the scalar. This scales the matrix uniformly — think of stretching or compressing the whole grid. Works for any matrix shape.
Matrix–Vector Multiply
Ax is a linear combination of A's columns, weighted by x. This is the most important operation in linear algebra — it describes every linear transformation.
Matrix Multiplication
The (i,j) entry of AB is the dot product of row i of A and column j of B. For this to work, A must be m×k and B must be k×n — inner dimensions must match.
Order Matters
Unlike numbers, matrix multiplication is NOT commutative: AB ≠ BA in general. But it IS associative: (AB)C = A(BC). And distributive: A(B+C) = AB + AC.
The Transpose
Swapping rows and columns gives the transpose AT. Key properties: (AB)T = BTAT (note the reversal!), and (AT)T = A.
The Trace
The trace of a square matrix is the sum of its diagonal entries. Simple but powerful: trace(AB) = trace(BA) even though AB ≠ BA. The trace equals the sum of eigenvalues.
Row × Column Pattern
Entry (i,j) = dot product of row i of A with column j of B. Highlight shows how A's first row and B's first column combine:
Why This Matters
- Neural networks: each layer is Ax + b
- Computer graphics: 3D rotation = matrix multiply
- Statistics: covariance = XTX
- Signal processing: filters as matrix products
- Physics: state evolution Ax = x’
Cost of Multiplication
Multiplying an m×k matrix by a k×n matrix takes O(mkn) operations — roughly m×k×n multiplications and additions. For n×n square matrices: O(n3). Large matrices are expensive!
What You Learned
Matrix addition is entry-wise. Scalar multiplication scales all entries. Matrix-vector and matrix-matrix products are dot-product operations. Multiplication is not commutative but is associative.