Designing Experiments
From observation to intervention. How to arrange a study so that differences in outcomes can be attributed to the treatment — and nothing else.
Causation, not Correlation
Observational studies can show associations but cannot rule out confounding. An experiment deliberately intervenes and controls confounders.
Randomization
Randomly assigning participants to groups balances all confounders — measured and unmeasured — across groups on average. It is the cornerstone of causal inference.
- Simple — coin flip per participant
- Stratified — balance within subgroups
- Cluster — randomize groups, not individuals
Control Groups
The baseline for comparison. Without a control group you cannot know if change is due to the treatment or would have happened anyway.
Blinding & Placebo
The placebo effect is real. Blinding participants prevents it. Double-blinding also prevents experimenter bias — subtle differences in how researchers treat or measure each group.
Blocking
Group similar units into blocks before randomizing. This removes known variability from the comparison and makes your estimate more precise.
Block & Randomize
These two principles work in tandem. Together they eliminate bias from both known and unknown confounders.
Factorial Designs
Study multiple factors at once. Cross all levels of each factor. This is more efficient than separate experiments and reveals interaction effects.
- 2×2 design: 2 factors × 2 levels = 4 groups
- Main effects: impact of each factor alone
- Interactions: effect of A depends on level of B
Power Analysis
Power = probability of detecting a true effect. Low power → missed discoveries. Compute required sample size before you start collecting data.
How many subjects?
For a two-group comparison, the required sample size per group is:
Ethical Obligations
Deliberately assigning treatments creates ethical responsibilities. These safeguards are essential, not optional:
- Equipoise — genuine uncertainty must exist before randomizing
- Informed consent — participants know the risks and can withdraw
- IRB approval — independent ethics review before data collection
What you learned
Randomization eliminates confounding. Control groups provide the baseline. Blocking removes known variability. Factorial designs reveal interactions. Power analysis sizes the study. Ethics protects participants.