Home / LA 101 / Module 3 / Lesson 4
Stories Mode

Applications of Linear Systems

Linear systems are not just a mathematical abstraction — they model circuits, chemical reactions, traffic flows, and data fitting. Every real-world balance condition becomes a row in a matrix.

~14 min read M3 · L4 Intermediate

From Abstraction to the Real World

Every application in this lesson follows the same template: identify the unknown quantities, write down the balance or conservation laws that relate them, and organize those laws as a matrix equation Ax = b. From there, Gaussian elimination takes over.

Electrical Engineering
Circuit Analysis
Kirchhoff's current and voltage laws produce one equation per node and loop — a system in the branch currents.
Chemistry
Balancing Equations
Conservation of each element type is a linear constraint on the stoichiometric coefficients.
Operations Research
Network Flow
Flow conservation at every node — what enters a junction must leave — is a linear system over the edge flows.
Data Science
Least Squares Fitting
When more data points than unknowns exist, the normal equations give the best-fit solution.

Electrical Circuits: Kirchhoff's Laws

Any resistive circuit is fully described by two of Kirchhoff's laws. KCL (current law): the sum of currents entering any node equals the sum leaving — charge is conserved. KVL (voltage law): the sum of voltage drops around any closed loop is zero — energy is conserved.

For a circuit with n nodes and m independent loops, these laws produce a system of equations. The unknowns are the branch currents I₁, I₂, …, Iₖ. Each resistor contributes a term IᵢRᵢ to the voltage equations (Ohm's law), and each current source or battery appears on the right-hand side.

A Three-Branch Example

Consider a circuit with a 12V battery and three resistors R₁ = 2Ω, R₂ = 3Ω, R₃ = 6Ω arranged so that R₂ and R₃ share a middle node. Assigning branch currents I₁, I₂, I₃:

Circuit as Matrix System
\begin{bmatrix}1&-1&-1\\2&3&0\\0&3&-6\end{bmatrix}\begin{bmatrix}I_1\\I_2\\I_3\end{bmatrix}=\begin{bmatrix}0\\12\\0\end{bmatrix}
The three Kirchhoff equations become one matrix equation. Gaussian elimination yields I₁ = 3A, I₂ = 2A, I₃ = 1A. Each entry of A is either a resistance value or a ±1 from a current balance.
Why This Works

KCL at every node and KVL for every independent loop produce exactly as many independent equations as there are unknown currents — provided you use the right number of independent loops (mesh analysis). The resulting matrix is always consistent and typically has a unique solution for a well-designed circuit.

Balancing Chemical Equations

A chemical equation must balance: the number of atoms of each element is identical on both sides. The stoichiometric coefficients (the numbers in front of each molecule) are the unknowns. Conservation of each element type produces one linear equation.

Combustion of Ethane

Consider: a C₂H₆ + b O₂ → c CO₂ + d H₂O. We need to find positive integers a, b, c, d. Conservation of each element:

This is a homogeneous system Ax = 0 with four unknowns and three equations — so rank(A) = 3 and nullity = 1. The solution has one free variable: set a = 1, and the unique solution (up to scaling) is a = 2, b = 7, c = 4, d = 6, giving the balanced equation 2 C₂H₆ + 7 O₂ → 4 CO₂ + 6 H₂O.

Balancing System
\begin{bmatrix}2&0&-1&0\\6&0&0&-2\\0&2&-2&-1\end{bmatrix}\begin{bmatrix}a\\b\\c\\d\end{bmatrix}=\begin{bmatrix}0\\0\\0\end{bmatrix}
Setting a = 2 as a free variable and solving gives the balanced coefficients. The null space of this matrix contains all scalar multiples of the balancing vector — physically meaning we can scale all coefficients by the same integer and the equation stays balanced.

Network Flow Problems

In any network — road traffic, internet packets, supply chains, water pipes — flow is conserved at every node: inflow equals outflow. Label the flow on each edge as an unknown. At each internal node, write: sum of incoming flows = sum of outgoing flows. This immediately produces a linear system.

A Four-Node Traffic Network

Suppose a city block has four intersections A, B, C, D with one-way streets. Known inflows enter at A (100 vehicles/hr) and B (80 vehicles/hr), and known outflows leave at C (90 vehicles/hr) and D (90 vehicles/hr). The edge flows x₁, x₂, x₃, x₄ on the interior streets are unknown.

KCL-style conservation at each intersection gives four equations. One will be redundant (total inflow = total outflow is guaranteed), leaving three independent equations in four unknowns — consistent, with one free variable. This means the traffic can be routed in a one-parameter family of ways, all satisfying conservation.

Underdetermined Flow Problems

Network flow systems are often underdetermined — there are more streets than intersections, so rank(A) < n. The free variables represent design choices: which routes to prioritize. The entire feasible set (all non-negative flow assignments satisfying conservation) is the intersection of the affine solution space with the non-negativity constraints — this is the starting point for linear programming.

Least Squares Fitting: Preview

In practice, measurement data rarely satisfies any linear model exactly. If we try to fit m data points with a model having n parameters (m > n), we get an overdetermined system Ax = b with no exact solution — there is no x that makes all m equations true simultaneously.

The least squares solution minimizes the total squared residual ||Ax − b||². Geometrically, we project the vector b onto the column space of A. The optimal x̂ satisfies the normal equations:

Normal Equations
A^T A\,\hat{x} = A^T b
Aᵀb is the projection of b onto the column space of A. When AᵀA is invertible (which it is when A has full column rank), the least squares solution is unique: x̂ = (AᵀA)⁻¹Aᵀb. The matrix (AᵀA)⁻¹Aᵀ is called the pseudoinverse of A.

Linear Regression as Least Squares

Fitting a line y = mx + c to n data points (x₁,y₁), …, (xₙ,yₙ) is a least squares problem. Build the matrix A with rows [xᵢ 1] and the vector b with entries yᵢ. The unknown vector is [m, c]ᵀ. The system Ax = b is overdetermined (n ≥ 2 equations in 2 unknowns), and the normal equations give the best-fit slope and intercept in closed form.

Best-Fit Line
\begin{bmatrix}\sum x_i^2 & \sum x_i \\ \sum x_i & n\end{bmatrix}\begin{bmatrix}m\\c\end{bmatrix}=\begin{bmatrix}\sum x_i y_i\\ \sum y_i\end{bmatrix}
The normal equations for the two-parameter linear fit reduce to a 2×2 system. The solution gives the slope m and intercept c that minimize the sum of squared vertical distances from the data points to the line.

Signal Processing: Filter Design

A finite impulse response (FIR) filter computes each output sample as a weighted sum of the current and past input samples: y[n] = h₀x[n] + h₁x[n−1] + … + h_{L−1}x[n−L+1]. The filter coefficients h₀, …, h_{L−1} are the unknowns.

If we want the filter to produce specific output values for a set of test inputs, each test case gives one equation. Enough test cases produce a square or overdetermined system for the coefficients. More commonly, frequency-domain specifications (pass certain frequencies, reject others) translate directly into linear constraints on the coefficients.

From Specification to Matrix

Requiring the filter's frequency response H(f) to equal target values at L specific frequencies produces L equations in L unknowns — exactly a square linear system. When the target is achievable exactly, Gaussian elimination finds the coefficients. When it is not (more frequency constraints than coefficients), the least squares approach finds the best-fit filter in the L² sense. This is the foundation of digital filter design in DSP.

The Common Thread

Across all these applications, the workflow is identical:

  1. Identify unknowns — currents, coefficients, flows, parameters.
  2. Write balance laws — conservation of charge, atoms, flow, or minimum error.
  3. Assemble Ax = b — each law becomes a row; each unknown becomes a column.
  4. Classify and solve — is the system square, overdetermined, or underdetermined? Apply Gaussian elimination, normal equations, or parametric solution accordingly.

The theory from the previous three lessons — setting up systems, elimination, and solution types — is precisely what makes all of these applications tractable. Linear algebra is not a prerequisite for engineering; it is the language engineering is written in.

Key Takeaways

Conservation laws in physics, chemistry, and engineering are linear equations — and their collection is a matrix system. Kirchhoff's laws produce the current vector from circuit topology. Chemical balancing finds the null space of the atom-conservation matrix. Network flow uses KCL-style node equations with free variables representing routing flexibility. When data exceeds unknowns, the normal equations Aᵀax = Aᵀb give the least squares optimum — the foundation of regression and filter design. The mathematical machinery is always the same: Ax = b, analyze rank, solve.