STAT 101
M07 · L01
Module 7 · Lesson 1

Simple Linear Regression

One line, two parameters, and the most widely used predictive model in statistics — fitting a straight line through data to describe how one variable drives another.

01 / 12
STAT 101
M07 · L01
The equation

The Model

Simple Linear Regression
Y = \beta_0 + \beta_1 X + \varepsilon
  • β₀ — intercept (Y when X = 0)
  • β₁ — slope (change in Y per unit X)
  • ε — random error (irreducible noise)
02 / 12
STAT 101
M07 · L01
What we minimize

Least Squares

Sum of Squared Residuals
\text{SSR} = \sum_{i=1}^{n}(y_i - \hat{y}_i)^2

A residual is the vertical gap between an observed point and the fitted line: eᵢ = yᵢ − ŷᵢ. Ordinary Least Squares (OLS) finds the line that minimizes the total squared gap.

Interactive: drag the points and watch the squared gaps it is minimising — AI-generated
03 / 12
STAT 101
M07 · L01
Closed-form solution

OLS Estimators

Slope
\hat{\beta}_1 = \dfrac{\sum(x_i-\bar{x})(y_i-\bar{y})}{\sum(x_i-\bar{x})^2}
Intercept
\hat{\beta}_0 = \bar{y} - \hat{\beta}_1\bar{x}
04 / 12
STAT 101
M07 · L01
What do they mean?

Reading Coefficients

Example: House prices
Price = 50,000 + 120 × Area

β₀ = $50k: baseline cost
β₁ = $120 per sq ft: marginal value of space

The intercept is meaningful only if X = 0 is in the data range. The slope is always interpretable: a one-unit increase in X predicts a β₁ change in Y.

05 / 12
STAT 101
M07 · L01
Goodness of fit

R² — Variance Explained

R-squared
R^2 = 1 - \dfrac{\text{SSR}}{\text{SST}} = 1 - \dfrac{\sum(y_i-\hat{y}_i)^2}{\sum(y_i-\bar{y})^2}
R² = 0
Line useless
R² = 1
Perfect fit
06 / 12
STAT 101
M07 · L01
Are assumptions met?

Residual Analysis

  • Linearity — residuals show no pattern vs. X
  • Normality — residuals roughly bell-shaped (QQ plot)
  • Homoscedasticity — variance constant across X
  • Independence — no autocorrelation in residuals
07 / 12
STAT 101
M07 · L01
Constant variance

Homo­scedasticity

The assumption
Var(ε | X = x) = σ² for all x

Violation = heteroscedasticity: spread of residuals fans out or funnels as X grows. Standard errors become unreliable.

Detect with a residuals vs. fitted-values plot. Fix with log transforms or robust standard errors.

08 / 12
STAT 101
M07 · L01
Two types of uncertainty

Prediction vs. Confidence

Confidence interval
Mean of Y
Prediction interval
Single Y

A confidence interval bounds the true mean response at X = x*. A prediction interval bounds a new individual observation — always wider because it adds irreducible ε variance.

09 / 12
STAT 101
M07 · L01
Testing the slope

Slope Significance

t-statistic for β₁
t = \dfrac{\hat{\beta}_1}{\text{SE}(\hat{\beta}_1)}

H₀: β₁ = 0 (X has no linear effect). Reject if |t| > t₋₍ᴰ₎ₓ(n−2). A significant slope means X is a useful linear predictor of Y.

10 / 12
STAT 101
Simple Linear Regression

Check what stuck

Four quick checks on fitting and reading a straight-line model.

Question 1 of 0
Score 0/0

11 / 12
STAT 101
Summary
Recap

What you learned

  • Model: Y = β₀ + β₁X + ε — line plus noise
  • OLS minimizes ∑(yᵢ − ŷᵢ)² to find β̂₀ and β̂₁
  • β₁ = slope: unit change in Y per unit X
  • R² measures fraction of Y variance explained by X
  • Check residuals for linearity, normality, homoscedasticity
  • Prediction intervals > confidence intervals — add ε noise
  • t-test on β₁ checks whether X is a useful predictor
Up Next
Module 7, Lesson 2 → Multiple Regression
All lessons Read article
12 / 12