The short answer
Define the comparisons before looking at the results.
Use Tukey when
Every pair belongs to the question
With k groups, Tukey protects all k(k − 1) / 2 pairwise mean comparisons. For three groups, that is three comparisons.
Use Dunnett when
Each treatment is compared with one control
With k groups, Dunnett protects the k − 1 treatment-versus-control comparisons. For three groups, that is two comparisons.
One dataset, two legitimate families
R's complete PlantGrowth dataset has ten dried plant weights in a
control group and ten in each of two treatment groups. The means are 5.032, 4.661
and 5.526. Keeping the observations fixed isolates the consequence of changing the
comparison family.
| Row | Control | Treatment 1 | Treatment 2 |
|---|---|---|---|
| 1 | 4.17 | 4.81 | 6.31 |
| 2 | 5.58 | 4.17 | 5.12 |
| 3 | 5.18 | 4.41 | 5.54 |
| 4 | 6.11 | 3.59 | 5.50 |
| 5 | 4.50 | 5.87 | 5.37 |
| 6 | 4.61 | 3.83 | 5.29 |
| 7 | 5.17 | 6.03 | 4.92 |
| 8 | 4.53 | 4.89 | 6.15 |
| 9 | 5.33 | 4.32 | 5.80 |
| 10 | 5.14 | 4.69 | 5.26 |
The ordinary one-way ANOVA gives F(2, 27) = 4.846, P = 0.01591. That global result does not define which pairwise family should be reported. The study question does.
Tukey protects all three pairwise claims
Tukey's method includes both treatment-versus-control comparisons and the treatment 2 versus treatment 1 comparison. The intervals and adjusted P values apply simultaneously to that complete family.
| Difference | Estimate | Simultaneous 95% CI | Adjusted P |
|---|---|---|---|
| Treatment 1 − control | −0.371 | −1.062 to 0.320 | 0.3909 |
| Treatment 2 − control | 0.494 | −0.197 to 1.185 | 0.1980 |
| Treatment 2 − treatment 1 | 0.865 | 0.174 to 1.556 | 0.01201 |
Only treatment 2 versus treatment 1 excludes zero. That is a valid Tukey-family result, but it is irrelevant if the protocol defined only treatment-versus-control claims.
Dunnett protects the two control comparisons
Dunnett omits treatment 2 versus treatment 1. Because it protects a smaller, correlated family, its simultaneous intervals are narrower here and its adjusted P values are smaller for the two retained comparisons. Neither interval excludes zero in this dataset.
| Difference | Estimate | Simultaneous 95% CI | Adjusted P |
|---|---|---|---|
| Treatment 1 − control | −0.371 | −1.021 to 0.279 | 0.3227 |
| Treatment 2 − control | 0.494 | −0.156 to 1.144 | 0.1536 |
Does Dunnett require a significant omnibus ANOVA?
Tukey and Dunnett are simultaneous multiple-comparison procedures, not a license to search after an F test becomes significant. A protocol may use an omnibus test as a formal gate, or it may pre-specify the comparison family as the primary inference. State that rule in advance. Do not treat a significant omnibus P value as proof that every comparison is meaningful, and do not hide a pre-specified comparison merely because the omnibus test missed 0.05.
The family changes; the model assumptions do not
- Observations must be independent within and among groups; technical replicates do not create new experimental units.
- The ordinary procedures use a common within-group variance estimate and require a credible equal-variance Gaussian error model.
- The control must be defined by the design. Picking the most convenient observed group as “control” changes the family.
- One-sided Dunnett inference is justified only by a directional hypothesis specified before observing the outcomes.
PlantGrowth is a calculation benchmark with sparse experimental metadata. It cannot establish the original experimental unit, randomization quality, model adequacy or a biological interpretation.
Compact reporting examples
Tukey family: “All pairwise mean differences were estimated with Tukey simultaneous 95% confidence intervals. Treatment 2 exceeded treatment 1 by 0.865 (95% CI 0.174 to 1.556, adjusted P = 0.0120); both treatment-versus-control intervals included zero.”
Dunnett family: “Each treatment was compared with the pre-specified control using two-sided Dunnett inference. The treatment 1 − control difference was −0.371 (simultaneous 95% CI −1.021 to 0.279, adjusted P = 0.3227), and treatment 2 − control was 0.494 (−0.156 to 1.144, adjusted P = 0.1536).”
What Plotwright checks
Plotwright sends the same 30 observations through its production one-way ANOVA engine twice. The Tukey project requests all-pairs comparisons. The Dunnett project designates column zero as control and requests control-versus-all comparisons. The executable suite checks the implementation against SciPy's Tukey and deterministic Dunnett procedures; the public numbers above reproduce that configured engine output.
Numerical agreement does not select the family, validate the design or make a scientific conclusion. Those remain researcher responsibilities.
Sources and reproducibility
- R datasets documentation: PlantGrowth—all 30 values and group labels.
- Tukey (1949), Comparing Individual Means in the Analysis of Variance—the original all-pairs procedure.
- Dunnett (1955), A Multiple Comparison Procedure for Comparing Several Treatments with a Control—the original control-comparison family.
- SciPy documentation: Dunnett's test—implementation, confidence intervals, assumptions and the distinction from Tukey.
- Plotwright one-way ANOVA and Tukey guide—the omnibus calculation, effect sizes and complete Tukey interpretation.
- Plotwright statistical validation—published expected values, tolerances and evidence boundaries.