The short answer
Separate the global question from the pairwise questions.
Ordinary one-way ANOVA compares variation among group means with residual variation within groups. Its null hypothesis is that every population mean is equal. Rejecting that single null says at least one mean differs under the model; it does not say that every pair differs or identify a particular pair.
The complete PlantGrowth benchmark
R's built-in PlantGrowth dataset contains 30 dried
plant weights: ten control observations and ten under each of
two treatment conditions. The three means are 5.032, 4.661 and
5.526. Plotting all observations shows both the shift in means
and the within-group variation used as the ANOVA error term.
What the omnibus ANOVA establishes
| Source | SS | df | MS | F | P |
|---|---|---|---|---|---|
| Treatment | 3.7663 | 2 | 1.8832 | 4.846 | 0.01591 |
| Residual | 10.4921 | 27 | 0.3886 | — | — |
| Total | 14.2584 | 29 | — | — | — |
The result is F(2, 27) = 4.846, P = 0.01591. The sample effect sizes are η² = 0.264 and ω² = 0.204. The F test rejects equality of all three means under the ordinary one-way model. It does not turn each pairwise comparison into a positive finding.
What the Tukey comparisons add
Tukey's method treats all three pairwise differences as one family and uses the studentized-range distribution. Its 95% intervals have simultaneous family-wise coverage for this balanced design.
| Difference | Estimate | Simultaneous 95% CI | Adjusted P |
|---|---|---|---|
| Treatment 1 − control | −0.371 | −1.062 to 0.320 | 0.3909 |
| Treatment 2 − control | 0.494 | −0.197 to 1.185 | 0.1980 |
| Treatment 2 − treatment 1 | 0.865 | 0.174 to 1.556 | 0.01201 |
Only the treatment 2 versus treatment 1 interval excludes zero. The omnibus result therefore cannot be rewritten as “both treatments differ from control.” Those two adjusted comparisons remain compatible with no mean difference at this sample size.
Choose the comparison family from the question
- Use all-pairs Tukey inference when every pairwise mean difference belongs to the scientific question.
- Use a control-focused family when the design asks only whether each treatment differs from a designated control.
- Use planned contrasts for specific combinations or trends defined before inspecting the outcomes.
- Do not run every unadjusted t test and report only the smallest P value; that changes the false-positive rate of the family.
If the design asks only whether each treatment differs from one pre-specified control, see the worked Tukey-versus-Dunnett comparison. It runs both families on these same 30 observations.
Check the design before the diagnostics
- Observations must represent independent experimental units. Technical replicates do not increase the biological sample size.
- Residual spread should be reasonably stable across groups and residual structure should be inspected graphically.
- The Gaussian error model should be plausible for the residuals, especially when groups are small or the design is unbalanced.
- If variance assumptions are not credible, define an unequal-variance analysis and compatible comparisons rather than switching methods only because a diagnostic crossed 0.05.
For PlantGrowth, the Brown–Forsythe equal-variance diagnostic is F(2, 27) = 1.119, P = 0.341. That result does not prove equal variances; it simply does not flag strong contrary evidence in this small balanced sample.
A compact reporting template
“Dried plant weight was compared among three groups using an ordinary one-way ANOVA (n = 10 observations per group), F(2, 27) = 4.846, P = 0.01591, η² = 0.264. Tukey-adjusted comparisons found a treatment 2 − treatment 1 difference of 0.865 (simultaneous 95% CI 0.174 to 1.556, adjusted P = 0.0120); the two treatment-versus-control intervals included zero.”
What Plotwright checks
Plotwright submits all 30 observations through the production one-way ANOVA engine. The executable suite checks the published F statistic and P value, while the worked project also computes sums of squares, effect sizes, residual diagnostics, variance checks and the exact Tukey comparisons shown above.
Numerical agreement does not determine the experimental unit, select the intended comparison family or make a causal claim. Those remain study-design decisions.
Sources and reproducibility
- R datasets documentation: PlantGrowth—the 30 values, group labels and one-factor ANOVA example.
- NIST/SEMATECH: one-way ANOVA—the model, variance partition, F test and assumptions.
- NIST/SEMATECH: Tukey's method—simultaneous all-pairs confidence intervals using the studentized range.
- R stats documentation: TukeyHSD—family-wise intervals and adjusted pairwise results.
- Plotwright statistical validation—published expected values, tolerance and evidence boundaries.