The Sampling Theorem

From Continuous to Discrete

The physical world is relentlessly analog. Sound waves ripple through air as smooth, continuous pressure variations. Temperature drifts slowly and without interruption. Light arrives in a seamless flood of photons. Yet the computers that process these phenomena are fundamentally discrete — they work with finite, countable numbers stored in memory. Bridging this gap is the central challenge of digital signal processing, and the Nyquist-Shannon Sampling Theorem is the mathematical key that makes it possible.

The theorem tells us something remarkable: a continuous signal can be perfectly captured and later perfectly reconstructed from a finite set of discrete samples — but only if we sample fast enough. Understanding exactly how fast, and what happens when we don’t, is the subject of this lesson.

What is Sampling?

Sampling is the process of measuring a continuous signal at regular time intervals and recording each measurement as a number. Each measurement is called a sample. The number of samples captured per second is the sampling rate (or sampling frequency), denoted f_s and measured in hertz (Hz).

Imagine a sine wave oscillating at 1 Hz — one full cycle per second. If we sample it at 10 Hz, we take 10 measurements per cycle, enough to see the wave’s shape clearly. But if we sample at only 1.5 Hz — fewer than 2 samples per cycle — the samples cannot possibly convey the original frequency. Something goes wrong.

The Nyquist Rate

Harry Nyquist (1928) and Claude Shannon (1949) established the precise minimum sampling rate needed to capture a bandlimited signal without loss. Their theorem states: if a signal contains no frequency components above f_max, then sampling at any rate f_s ≥ 2 f_max is sufficient for perfect reconstruction.

The quantity 2 f_max is called the Nyquist rate. Half the sampling frequency — f_s/2 — is called the Nyquist frequency or folding frequency. It represents the highest frequency that can be unambiguously represented at a given sampling rate.

Why Twice the Frequency?

The factor of 2 can be understood intuitively. A sine wave at frequency f completes one full cycle in 1/f seconds. To uniquely identify this cycle — to distinguish it from a completely different waveform — we need at least two samples per cycle: one to capture the rising portion and one to capture the falling portion. One sample per cycle leaves ambiguity; we can’t tell whether the signal is rising or falling at that point.

More formally, the sampling process creates multiple copies of the signal’s spectrum, spaced apart by f_s in the frequency domain. If f_s ≥ 2 f_max, these copies don’t overlap, and the original spectrum can be isolated with a simple lowpass filter. If f_s < 2 f_max, the copies collide and mix, creating aliasing — a permanent, unrecoverable distortion.

Aliasing: When Sampling Goes Wrong

When the sampling rate falls below the Nyquist rate, high-frequency components of the signal appear as spurious low-frequency components in the sampled data. This phenomenon is called aliasing. The name comes from the fact that a high-frequency signal masquerades under a low-frequency “alias.”

The alias frequency is determined by the formula: if a signal at frequency f is sampled at f_s and f > f_s/2, it aliases to a frequency |f − k f_s| for some integer k that places the result below f_s/2. In simpler terms, the high frequency folds back into the audible or visible range.

The wagon-wheel effect is the most famous visual example of aliasing. In old Western movies, wagon wheels sometimes appear to spin backward. A camera records at 24 frames per second, so its sampling rate is f_s = 24 Hz and its Nyquist frequency is 12 Hz. Pick out one spoke and follow it. If the wheel turns at 23 revolutions per second — far above 12 Hz, and just below the frame rate — then between frames that spoke advances 23/24 of a turn, stopping just short of where it started. Your eye takes the shortest path and reads it as 1/24 of a turn backward, so the wheel appears to creep in reverse at 1 revolution per second. The rotation has aliased.

The sign matters. Put the wheel at 25 rev/s instead — just above the frame rate — and each frame the spoke overshoots by 1/24 of a turn, so the wheel crawls slowly forward at 1 rev/s. The formula above gives the same 1 Hz magnitude for both (|23 − 24| = |25 − 24| = 1), because it takes an absolute value; the direction comes from the sign of f − round(f/f_s)·f_s, which is negative at 23 Hz and positive at 25 Hz. Either way the camera cannot tell 23 Hz, 25 Hz and 1 Hz apart.

In audio, aliasing produces phantom tones at incorrect frequencies, audible as clicks, buzzes, or distorted harmonics. In a recording studio, aliasing is disastrous because it pollutes the recorded signal with artifacts that are indistinguishable from real content.

Anti-Aliasing Filters

The standard defense against aliasing is to bandlimit the signal before sampling by applying a lowpass filter — commonly called an anti-aliasing filter. This filter attenuates all frequencies above f_s/2, ensuring that whatever reaches the sampler satisfies the Nyquist condition.

Every ADC (Analog-to-Digital Converter) in professional equipment includes such a filter. In practice, filters cannot have perfectly sharp cutoffs, so designers choose a sampling rate slightly higher than the strict minimum. CD audio uses 44.1 kHz partly because the anti-aliasing filter of the era worked reliably up to about 20 kHz with margin to spare.

Perfect Reconstruction

The sampling theorem is not just a statement about capture — it is equally a statement about reconstruction. Given samples taken at or above the Nyquist rate, the original continuous signal can be recovered exactly using a process called sinc interpolation.

Mathematically, reconstruction multiplies each sample by a sinc function — a sin(x)/x curve centered on that sample’s time location — and sums the result. The sinc function is the impulse response of an ideal lowpass filter, and this reconstruction is exactly equivalent to lowpass filtering the impulse train of samples.

In practice, ideal sinc interpolation is computationally expensive because the sinc function extends infinitely in time. Real systems use windowed sinc filters or other polynomial interpolation methods that trade a small amount of theoretical precision for manageable computation.

Practical Implications

The sampling theorem governs virtually every digital system that interacts with the physical world. Audio: CD players sample at 44.1 kHz, capturing all frequencies audible to humans (up to ~20 kHz). Professional studios use 48 kHz or 96 kHz for additional margin. Telecommunications: Voice over phone lines is bandlimited to 3.4 kHz and sampled at 8 kHz — exactly twice. Video codecs sample color and luminance channels at rates matched to the human visual system’s spatial frequency response.

Medical imaging: An MRI machine acquires raw data in the frequency domain (k-space) and must sample densely enough to reconstruct the desired spatial resolution. CT scanners rotate a detector array around the patient and must sample at sufficient angular intervals. Both are direct applications of the sampling theorem in two dimensions.

Wireless communications: A 5G base station may receive signals spanning 100 MHz of bandwidth. Its ADC must sample at over 200 megasamples per second. The sampling theorem determines the minimum hardware specification — and with it, the cost and power consumption of every base station on every tower.