Plotwright

Worked method-comparison example

High correlation does not mean two methods agree.

Correlation asks whether two measurements vary together. Bland–Altman analysis asks how far apart paired measurements are—and whether that disagreement is acceptable for the intended use.

17 paired subjectsOriginal PEFR dataBias + limits of agreement9-minute read

The short answer

Association and agreement answer different questions.

A correlation can be high whenever subjects span a wide measurement range, even if the two instruments differ enough to be non-interchangeable. For agreement, inspect each paired difference, estimate the average bias and limits of agreement, and compare those limits with tolerances chosen before looking at the results.

Decision rule: use correlation for association. Use a method- comparison design and Bland–Altman analysis for agreement. Do not declare two methods interchangeable from a large r or a non-significant paired test.

The original peak-flow comparison

Bland and Altman measured peak expiratory flow rate (PEFR) on 17 people with a Wright meter and a Mini Wright meter. Each instrument was used twice; this worked example uses the first reading from each instrument, one pair per person.

Person Wright Mini Wright Pair mean Wright − Mini
1 494 512 503.0 −18
2 395 430 412.5 −35
3 516 520 518.0 −4
4 434 428 431.0 6
5 476 500 488.0 −24
6 557 600 578.5 −43
7 413 364 388.5 49
8 442 380 411.0 62
9 650 658 654.0 −8
10 433 445 439.0 −12
11 417 432 424.5 −15
12 656 626 641.0 30
13 267 260 263.5 7
14 478 477 477.5 1
15 178 259 218.5 −81
16 423 350 386.5 73
17 427 451 439.0 −24

Units are L/min. The sign convention matters: every result below is Wright minus Mini Wright. Reversing the methods reverses the bias and limits.

Why r looks reassuring

Across the 17 pairs, Pearson correlation is r = 0.9433. That is a strong linear association: people with higher readings on one meter usually have higher readings on the other. It says nothing about whether a difference of 40, 60 or 80 L/min is acceptable for an individual.

Correlation is also sensitive to the between-person range. Adding people with very low and very high PEFR can increase r without improving either meter's within-person agreement. Agreement is about the paired errors, not merely the rank order of subjects.

Build the Bland–Altman plot pair by pair

Put each pair's average on the horizontal axis and its difference on the vertical axis. The center line is the mean difference, or bias. Under the basic normal- differences model, estimated 95% limits of agreement are bias ± 1.96 × SD(differences).

Bland–Altman plot of the first PEFR readings Seventeen pair means are plotted against Wright minus Mini Wright differences. Bias is minus 2.1 litres per minute and the estimated 95 percent limits of agreement are approximately minus 78.1 and plus 73.9 litres per minute. Person 1: mean 503, difference −18 Person 2: mean 412.5, difference −35 Person 3: mean 518, difference −4 Person 4: mean 431, difference 6 Person 5: mean 488, difference −24 Person 6: mean 578.5, difference −43 Person 7: mean 388.5, difference 49 Person 8: mean 411, difference 62 Person 9: mean 654, difference −8 Person 10: mean 439, difference −12 Person 11: mean 424.5, difference −15 Person 12: mean 641, difference 30 Person 13: mean 263.5, difference 7 Person 14: mean 477.5, difference 1 Person 15: mean 218.5, difference −81 Person 16: mean 386.5, difference 73 Person 17: mean 439, difference −24 100 50 0 −50 −100 200 400 600 Upper LoA +73.9 Bias −2.1 Lower LoA −78.1 Pair mean (L/min) Wright − Mini (L/min)
The strong correlation is invisible here because this plot asks the agreement question directly. The vertical spread of paired differences is the relevant evidence.

Read bias, limits and uncertainty separately

  • Bias: −2.1176 L/min (approximate 95% CI −22.0488 to 17.8135). On average, the Wright first reading is about 2.1 L/min lower.
  • SD of paired differences: 38.7651 L/min.
  • Estimated 95% limits of agreement: −78.0959 to 73.8606 L/min.
  • Approximate CI for the lower limit: −112.8516 to −43.3403; for the upper limit: 39.1050 to 108.6163 L/min.

The original paper rounded bias to −2.1 and SD to 38.8, then illustrated limits with approximately ±2 SD. Plotwright uses the normal 1.96 multiplier, so its unrounded limits differ slightly from the historical rounded display.

Small bias is not enough: average disagreement is near zero because positive and negative differences cancel. Interchangeability depends on whether the full limits—and their uncertainty—fit a pre-specified acceptable range. This dataset alone does not supply that clinical tolerance.

Check the design and model before interpreting

  • Independent experimental units: one pair per person is used here. Repeated readings need a repeated-measures extension; they must not be treated as independent subjects.
  • Difference shape: the basic limits assume paired differences are approximately normal. Inspect the difference distribution and unusual points.
  • Constant disagreement: inspect whether spread changes with magnitude. A log/ratio or regression approach may be more appropriate when it does.
  • Measurement range: sample the range where the methods will be used; do not extrapolate agreement beyond it.
  • Acceptance bounds: define practically acceptable disagreement from the intended use before seeing the plot.

Plotwright's difference-on-average slope is 0.02869 (P = 0.7495). This sample does not show evidence of proportional bias under that linear test, but n = 17 gives limited power and “not detected” is not “proved absent.” The plot and subject- matter tolerance remain primary.

A compact reporting template

“Agreement between first Wright and Mini Wright PEFR readings was evaluated in 17 paired subjects using Wright minus Mini differences. Mean bias was −2.12 L/min (approximate 95% CI −22.05 to 17.81), with estimated 95% limits of agreement from −78.10 to 73.86 L/min. Approximate confidence intervals for the limits were −112.85 to −43.34 and 39.10 to 108.62 L/min. All pair means and differences were shown.”

Then state the pre-specified acceptable limits and whether the estimated limits and their uncertainty meet them. Report the sign convention, units, handling of repeats and missing pairs, and any transformation or proportional-bias model.

What Plotwright checks

Plotwright sends all 17 pairs to its production Bland–Altman engine. The executable benchmark checks the original paper's rounded bias and SD, reconstructs modern 1.96 limits from NumPy/SciPy, checks the difference-on-average regression, and independently confirms that Pearson correlation is 0.9433.

Those checks establish the defined calculations. They cannot define a clinically acceptable difference, repair an inadequate sampling design, justify pooling repeated readings, or prove interchangeability.

Sources and reproducibility

  1. Bland & Altman (1986)—the original method-comparison paper and PEFR example.
  2. Martin Bland's PEFR dataset—all 17 subjects and both replicate readings from each instrument.
  3. Bland & Altman (1999)—limits of agreement, model diagnostics, confidence intervals and extensions for magnitude-dependent and repeated measurements.
  4. Giavarina (2015): Understanding Bland Altman analysis—agreement interpretation and why correlation answers a different question.
  5. Plotwright statistical validation—published expected values, tolerances and evidence limits.

Keep every pair, difference and limit inspectable.

Plotwright opens all 17 subjects, a paired before–after graph and the complete Bland–Altman result in one editable local project.

Open the exact agreement project Related guide: choosing a correlation