Scalars, Vectors & Matrices | LA-101

The Building Blocks

Before we can do anything meaningful in linear algebra, we need to understand its three fundamental objects: scalars, vectors, and matrices. These are the atoms of the subject — every concept, operation, and application in linear algebra is built from these three ingredients.

Think of them as a hierarchy. A scalar is a single number. A vector is an ordered list of numbers. A matrix is a rectangular grid of numbers. Simple definitions, but they unlock an extraordinary range of mathematical power. Let us explore each one.

Scalars: Just a Number

A scalar is the simplest object in linear algebra — it is just a single number. It could be an integer like 5, a real number like 3.14, or even a complex number. Scalars have magnitude only — they tell you "how much" but not "which direction."

Despite their simplicity, scalars play a crucial role. They act as scaling factors: when you multiply a vector by a scalar, you stretch or shrink it. When you multiply a matrix by a scalar, you scale every entry. The word "scalar" literally comes from "scale" — these numbers control size.

Scalars in Context

Temperature, mass, speed, energy — these are all scalar quantities. They have a value but no inherent direction. Compare this to velocity, force, or displacement, which are vector quantities because they carry both magnitude and direction.

Vectors: Direction and Magnitude

A vector is an ordered list of numbers. In two dimensions, a vector has two components. In three dimensions, three. In machine learning, vectors can have thousands or even millions of components. What makes a vector special is that it encodes both direction and magnitude.

We typically write vectors as columns enclosed in brackets. The entries are called components, and they describe the displacement along each axis. A 2D vector tells you how far to go horizontally and vertically. A 3D vector adds depth.

Vector Notation

\mathbf{v} = \begin{bmatrix} v_1 \\ v_2 \end{bmatrix}, \quad \mathbf{w} = \begin{bmatrix} w_1 \\ w_2 \\ w_3 \end{bmatrix}, \quad \mathbf{u} = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix}

A 2D vector with components x and y, and its extension to n dimensions

Vectors support two fundamental operations. Vector addition combines two vectors component by component — geometrically, this is the tip-to-tail rule. Scalar multiplication stretches or shrinks a vector by a constant factor. If you multiply by a negative scalar, the vector flips direction.

Vector Addition

\begin{bmatrix} a_1 \\ a_2 \end{bmatrix} + \begin{bmatrix} b_1 \\ b_2 \end{bmatrix} = \begin{bmatrix} a_1+b_1 \\ a_2+b_2 \end{bmatrix}

Add corresponding components

Scalar Multiplication

c \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} = \begin{bmatrix} cv_1 \\ cv_2 \end{bmatrix}

Scale every component by c

Matrices: Rectangular Arrays

A matrix is a rectangular array of numbers arranged in rows and columns. An m × n matrix has m rows and n columns. We refer to individual entries using subscript notation: a_ij is the element in row i, column j.

Matrices are far more than tables of data. They represent linear transformations, systems of equations, datasets, and network connections. When you multiply a matrix by a vector, you transform that vector. When you multiply two matrices, you compose two transformations into one.

Matrix Notation

A = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix}

An m × n matrix with entries a_ij — row i, column j

Special Matrices

Certain matrices appear so frequently that they have their own names. The identity matrix I is a square matrix with ones on the diagonal and zeros everywhere else. Multiplying any vector or matrix by I leaves it unchanged — it is the "do nothing" transformation.

The zero matrix has all entries equal to zero — it maps every vector to the zero vector. A diagonal matrix has nonzero entries only on the main diagonal, scaling each axis independently. A symmetric matrix equals its own transpose: A = A^T, meaning it is mirrored across the diagonal.

The Identity Matrix

I = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}, \quad I\mathbf{v} = \mathbf{v}

The identity matrix — ones on the diagonal, zeros elsewhere. I·v = v for any vector v.

Matrix Operations

Matrices support several key operations. Addition works element by element, just like vector addition — but both matrices must have the same dimensions. Scalar multiplication scales every entry by a constant.

The transpose of a matrix, written A^T, flips the matrix over its diagonal — rows become columns and columns become rows. If A is m × n, then A^T is n × m. The transpose is fundamental in statistics, optimization, and anywhere symmetric relationships arise.

Transpose

A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix} \;\Rightarrow\; A^T = \begin{bmatrix} 1 & 4 \\ 2 & 5 \\ 3 & 6 \end{bmatrix}

Transposing a matrix swaps rows and columns — (A^T)_ij = A_ji

Why These Building Blocks Matter

Scalars, vectors, and matrices are not just abstract objects — they are the fundamental language of engineering and science. In signal processing, a signal is a vector. In machine learning, a dataset is a matrix. In physics, forces are vectors and transformations are matrices.

Understanding these building blocks deeply is essential because every advanced topic in linear algebra — eigenvalues, decompositions, least squares, singular value decomposition — is expressed in terms of scalars, vectors, and matrices. Master them now, and every future concept will feel natural.

In the next lesson, we will explore vector spaces and linear independence — the rules that govern how vectors combine and the concept of dimension that emerges from those rules.

Key Takeaways

A scalar is a single number with magnitude only — it scales vectors and matrices.
A vector is an ordered list of numbers encoding direction and magnitude — the fundamental object of linear algebra.
A matrix is a rectangular array of numbers that represents transformations, systems, and data.
Special matrices like the identity, zero, diagonal, and symmetric matrices appear throughout science and engineering.
The transpose flips a matrix over its diagonal, turning rows into columns — a key operation in optimization and statistics.