Statistical validation

Trust the result.
Inspect the evidence.

Plotwright checks its numerical output against published examples, independent library implementations and closed-form identities. Each check names its dataset, expected value and tolerance.

Review the benchmarks Read the evidence limits

How a result earns trust

Three independent anchors, used where they fit.

A passing test is useful only when its reference is meaningfully independent of the implementation under test.

01 / PUBLISHED

Reproduce reported results

Canonical datasets from textbooks, papers and R documentation are run end to end and compared with the reported statistics.

02 / INDEPENDENT

Cross-check another implementation

Results are compared with established numerical libraries such as SciPy, statsmodels, lifelines and scikit-learn when an equivalent implementation exists.

03 / IDENTITIES

Pin the underlying mathematics

Closed-form calculations and non-circular identities catch cases where two software implementations could share the same mistake.

Worked benchmarks

Recognizable datasets and transparent stress tests.

This readable subset combines published examples with explicitly labeled synthetic stress tests. Each entry identifies its source, expected result and numerical anchor.

Paired t test

Student's sleep data

t = −4.0621 · df = 9 · P = 0.002833

Absolute tolerance: t 5×10⁻⁴; P 1×10⁻⁵ R datasets reference → Worked paired-vs-unpaired guide →

Welch + pooled two-sample t test

Unequal variance × unequal n

Welch P = 0.00684 · pooled P = 0.0751

Transparent synthetic stress test; both formulas independently checked NIST/SEMATECH formulas → Read the variance-model guide → Open all 24 editable values →

Descriptive uncertainty

SD, SEM and 95% confidence intervals

n=5: SD 1.581 · SEM 0.707
n=20: SD 1.477 · SEM 0.330

SciPy SEM anchor plus closed-form t intervals Cumming, Fidler & Vaux (2007) → Read the error-bar guide → Open all 25 values and three graphs →

Mann–Whitney U test

Chorioamnion permeability

U = 35 · exact P = 0.2544

Exact-null enumeration across all 3,003 rank allocations R stats reference → Read what the rank test measures → Open the complete editable benchmark →

Pearson + Spearman correlation

Tuna quality

r = 0.5712 · ρ = 0.6000

Independent centered-product and rank-identity checks R stats reference → Read the coefficient-choice guide → Open all nine editable pairs →

Bland–Altman method agreement

Peak expiratory flow meters

bias = −2.118 · LoA −78.096 to 73.861

Original 17-subject PEFR data; 1.96-SD limits Bland & Altman (1986) → Read why high correlation is not agreement → Open all 17 editable pairs →

Fisher exact + chi-square

Tea tasting

χ² = 2.000 · Fisher P = 0.4857

Exact 34/70; every expected count = 2 R stats reference → Read the expected-count guide → Open the exact editable table →

One-way ANOVA

PlantGrowth

F(2, 27) = 4.846 · P = 0.0159

Absolute tolerance: F 1×10⁻³; P 1×10⁻⁴ R datasets reference → Read the ANOVA and Tukey guide → Choose ANOVA or Kruskal–Wallis → Choose Tukey or Dunnett → Open the complete editable benchmark →

Regression + correlation

Anscombe's quartet

slope = 0.500 · R² = 0.667 · r = 0.816

Absolute tolerance: 1×10⁻³ Anscombe (1973) → Explore all four datasets → Open the editable quartet →

Two-way ANOVA

ToothGrowth

Finteraction = 4.107 · P = 0.02186

Absolute tolerance: F 1×10⁻²; P 1×10⁻⁴ R datasets reference → Interpret the interaction → Open the editable benchmark →

Survival analysis

6-MP leukemia trial

χ² = 16.79 · df = 1 · P = 4.17×10⁻⁵

Absolute tolerance: χ² 1×10⁻²; P 1% relative Freireich et al. (1963) → Work through the survival curves →

ROC analysis

Hanley–McNeil rating data

AUC = 0.893 · n = 109

Absolute tolerance: AUC 1×10⁻³ Hanley & McNeil (1982) → Work through the ROC thresholds →

What this does—and does not—prove

Validation is analysis-specific.

A benchmark supports the exact statistic, options and numerical tolerance it exercises. It is not a blanket guarantee that every method is appropriate for every dataset. Researchers remain responsible for study design, assumptions, exclusions and interpretation.

When a result differs from its reference, the automated suite fails. New statistical features are expected to add an independent oracle before they ship.

Test it with data whose answer you already know.

Start with a published example or one of your lab's verified projects. Inspect the Results sheet, assumptions and generated methods text before adopting a workflow.

Start the 14-day trial Review migration scope