Code
library(ivdtools)
library(readr)8 Bottle-to-Bottle Variability Analysis
8.1 Analysis Objectives
Bottle-to-bottle variability evaluates the differences in measurement results among different bottles (or different lots) of the same product. The analysis is usually done in two steps: first, use bottle_anova() to test whether the means of the bottles (or lots) differ statistically, and then use variance-component analysis to separate the between-bottle variability from the within-bottle variability, judging whether the magnitude of the difference meets the pre-specified precision limits. This document demonstrates both scenarios: the bottle-to-bottle difference of 15 bottles × 5 measurements, and the between-lot difference when different reagent lots measure the same quality-control material.
The example data are deterministic teaching data; statistical significance is not equivalent to product unacceptability; judgments can only be made against the acceptance limits pre-specified in the protocol.
8.2 Overview of Functions
| Function | Main purpose | Key input or output |
|---|---|---|
bottle_anova() |
One-way or nested ANOVA | value ~ batch or value ~ batch/vial |
tukey() |
Tukey HSD pairwise comparison and compact letters | term specifies the effect |
precision() / variance() |
Split between-bottle/within-bottle variance components | value ~ bottle one-way |
8.3 Example 1: Bottle-to-Bottle Difference of 15 Bottles × 5 Measurements
8.3.1 Reading and Validating Data
Code
bottle <- read_csv("./data/bottle-difference.csv", show_col_types = FALSE)
bottle <- as.data.frame(bottle)
bottle$bottle <- factor(bottle$bottle)
str(bottle)'data.frame': 75 obs. of 3 variables:
$ bottle : Factor w/ 15 levels "B01","B02","B03",..: 1 1 1 1 1 2 2 2 2 2 ...
$ replicate: num 1 2 3 4 5 1 2 3 4 5 ...
$ value : num 97.5 100.5 98.8 97.5 97.8 ...Code
dim(bottle)[1] 75 38.3.2 Step 1: Test for Significant Difference
The one-way model value ~ bottle tests whether the means of the 15 bottles are equal:
Code
ba <- bottle_anova(bottle, value ~ bottle)
print(ba)
Bottle ANOVA Analysis
Response: value
Group variable: bottle (15 groups)
Complete cases: 75 / 75
Confidence level: 0.95
Descriptive Statistics:
------------------------------------------------------------
bottle n Mean SD
------------------------------------------------------------
B01 5 98.4520 1.2739
B02 5 101.7140 0.9360
B03 5 99.8060 0.8153
B04 5 102.7980 0.4992
B05 5 98.7520 0.8662
B06 5 100.0960 1.4971
B07 5 100.7180 0.8466
B08 5 100.5000 1.3179
B09 5 99.8160 1.0246
B10 5 101.6400 0.8034
B11 5 99.3300 0.9360
B12 5 98.4700 1.0884
B13 5 99.7440 1.1004
B14 5 100.9040 0.4803
B15 5 99.7920 1.4477
ANOVA Table:
------------------------------------------------------------
Term Df Sum Sq Mean Sq F-value p-value Sig
------------------------------------------------------------
bottle 14 107.7233 7.6945 7.1460 0.0000 ***
Residuals 60 64.6057 1.0768
------------------------------------------------------------
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Assumption Checks:
Bartlett test: K-squared = 9.9325, df = 14, p-value = 0.7671 (OK)
Shapiro-Wilk: W = 0.9828, p-value = 0.4011 (OK)
Post-hoc: use tukey() for pairwise comparisonsThe ANOVA table gives the between-bottle F test; the assumptions section reports the Bartlett variance-homogeneity and Shapiro-Wilk normality tests.
8.3.3 Pairwise Comparison
Code
posthoc <- tukey(ba)
posthoc
Tukey HSD Post-hoc Test
Response: value
Complete cases: 75 / 75
Confidence level: 0.95
Term: bottle
------------------------------------------------------------
Comparison Diff lwr upr p-value Sig
------------------------------------------------------------
B02-B01 3.2620 0.9412 5.5828 0.0005 ***
B03-B01 1.3540 -0.9668 3.6748 0.7526
B04-B01 4.3460 2.0252 6.6668 0.0000 ***
B05-B01 0.3000 -2.0208 2.6208 1.0000
B06-B01 1.6440 -0.6768 3.9648 0.4526
B07-B01 2.2660 -0.0548 4.5868 0.0624
B08-B01 2.0480 -0.2728 4.3688 0.1408
B09-B01 1.3640 -0.9568 3.6848 0.7431
B10-B01 3.1880 0.8672 5.5088 0.0008 ***
B11-B01 0.8780 -1.4428 3.1988 0.9896
B12-B01 0.0180 -2.3028 2.3388 1.0000
B13-B01 1.2920 -1.0288 3.6128 0.8078
B14-B01 2.4520 0.1312 4.7728 0.0287 *
B15-B01 1.3400 -0.9808 3.6608 0.7656
B03-B02 -1.9080 -4.2288 0.4128 0.2229
B04-B02 1.0840 -1.2368 3.4048 0.9391
B05-B02 -2.9620 -5.2828 -0.6412 0.0026 **
B06-B02 -1.6180 -3.9388 0.7028 0.4794
B07-B02 -0.9960 -3.3168 1.3248 0.9686
B08-B02 -1.2140 -3.5348 1.1068 0.8674
B09-B02 -1.8980 -4.2188 0.4228 0.2298
B10-B02 -0.0740 -2.3948 2.2468 1.0000
B11-B02 -2.3840 -4.7048 -0.0632 0.0384 *
B12-B02 -3.2440 -5.5648 -0.9232 0.0006 ***
B13-B02 -1.9700 -4.2908 0.3508 0.1830
B14-B02 -0.8100 -3.1308 1.5108 0.9952
B15-B02 -1.9220 -4.2428 0.3988 0.2134
B04-B03 2.9920 0.6712 5.3128 0.0022 **
B05-B03 -1.0540 -3.3748 1.2668 0.9507
B06-B03 0.2900 -2.0308 2.6108 1.0000
B07-B03 0.9120 -1.4088 3.2328 0.9852
B08-B03 0.6940 -1.6268 3.0148 0.9990
B09-B03 0.0100 -2.3108 2.3308 1.0000
B10-B03 1.8340 -0.4868 4.1548 0.2778
B11-B03 -0.4760 -2.7968 1.8448 1.0000
B12-B03 -1.3360 -3.6568 0.9848 0.7693
B13-B03 -0.0620 -2.3828 2.2588 1.0000
B14-B03 1.0980 -1.2228 3.4188 0.9330
B15-B03 -0.0140 -2.3348 2.3068 1.0000
B05-B04 -4.0460 -6.3668 -1.7252 0.0000 ***
B06-B04 -2.7020 -5.0228 -0.3812 0.0092 **
B07-B04 -2.0800 -4.4008 0.2408 0.1258
B08-B04 -2.2980 -4.6188 0.0228 0.0549
B09-B04 -2.9820 -5.3028 -0.6612 0.0023 **
B10-B04 -1.1580 -3.4788 1.1628 0.9026
B11-B04 -3.4680 -5.7888 -1.1472 0.0002 ***
B12-B04 -4.3280 -6.6488 -2.0072 0.0000 ***
B13-B04 -3.0540 -5.3748 -0.7332 0.0016 **
B14-B04 -1.8940 -4.2148 0.4268 0.2326
B15-B04 -3.0060 -5.3268 -0.6852 0.0020 **
B06-B05 1.3440 -0.9768 3.6648 0.7619
B07-B05 1.9660 -0.3548 4.2868 0.1854
B08-B05 1.7480 -0.5728 4.0688 0.3515
B09-B05 1.0640 -1.2568 3.3848 0.9470
B10-B05 2.8880 0.5672 5.2088 0.0037 **
B11-B05 0.5780 -1.7428 2.8988 0.9999
B12-B05 -0.2820 -2.6028 2.0388 1.0000
B13-B05 0.9920 -1.3288 3.3128 0.9696
B14-B05 2.1520 -0.1688 4.4728 0.0968
B15-B05 1.0400 -1.2808 3.3608 0.9556
B07-B06 0.6220 -1.6988 2.9428 0.9997
B08-B06 0.4040 -1.9168 2.7248 1.0000
B09-B06 -0.2800 -2.6008 2.0408 1.0000
B10-B06 1.5440 -0.7768 3.8648 0.5576
B11-B06 -0.7660 -3.0868 1.5548 0.9972
B12-B06 -1.6260 -3.9468 0.6948 0.4711
B13-B06 -0.3520 -2.6728 1.9688 1.0000
B14-B06 0.8080 -1.5128 3.1288 0.9953
B15-B06 -0.3040 -2.6248 2.0168 1.0000
B08-B07 -0.2180 -2.5388 2.1028 1.0000
B09-B07 -0.9020 -3.2228 1.4188 0.9866
B10-B07 0.9220 -1.3988 3.2428 0.9837
B11-B07 -1.3880 -3.7088 0.9328 0.7199
B12-B07 -2.2480 -4.5688 0.0728 0.0670
B13-B07 -0.9740 -3.2948 1.3468 0.9739
B14-B07 0.1860 -2.1348 2.5068 1.0000
B15-B07 -0.9260 -3.2468 1.3948 0.9831
B09-B08 -0.6840 -3.0048 1.6368 0.9992
B10-B08 1.1400 -1.1808 3.4608 0.9125
B11-B08 -1.1700 -3.4908 1.1508 0.8956
B12-B08 -2.0300 -4.3508 0.2908 0.1498
B13-B08 -0.7560 -3.0768 1.5648 0.9976
B14-B08 0.4040 -1.9168 2.7248 1.0000
B15-B08 -0.7080 -3.0288 1.6128 0.9988
B10-B09 1.8240 -0.4968 4.1448 0.2859
B11-B09 -0.4860 -2.8068 1.8348 1.0000
B12-B09 -1.3460 -3.6668 0.9748 0.7601
B13-B09 -0.0720 -2.3928 2.2488 1.0000
B14-B09 1.0880 -1.2328 3.4088 0.9374
B15-B09 -0.0240 -2.3448 2.2968 1.0000
B11-B10 -2.3100 -4.6308 0.0108 0.0523
B12-B10 -3.1700 -5.4908 -0.8492 0.0009 ***
B13-B10 -1.8960 -4.2168 0.4248 0.2312
B14-B10 -0.7360 -3.0568 1.5848 0.9982
B15-B10 -1.8480 -4.1688 0.4728 0.2668
B12-B11 -0.8600 -3.1808 1.4608 0.9914
B13-B11 0.4140 -1.9068 2.7348 1.0000
B14-B11 1.5740 -0.7468 3.8948 0.5257
B15-B11 0.4620 -1.8588 2.7828 1.0000
B13-B12 1.2740 -1.0468 3.5948 0.8226
B14-B12 2.4340 0.1132 4.7548 0.0311 *
B15-B12 1.3220 -0.9988 3.6428 0.7819
B14-B13 1.1600 -1.1608 3.4808 0.9015
B15-B13 0.0480 -2.2728 2.3688 1.0000
B15-B14 -1.1120 -3.4328 1.2088 0.9266
------------------------------------------------------------
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Compact Letter Display:
------------------------------------------------------------
B04: a
B02: a
B10: a
B14: a
B07: ab
B08: ab
B06: b
B09: b
B03: b
B15: b
B13: b
B11: b
B05: b
B12: b
B01: bTukey HSD gives all pairwise comparisons and compact letter displays; bottles with different letters differ significantly in mean. In this example, there are many bottles, so some significant pairwise differences are normal; the key question is whether the magnitude of the difference is acceptable.
8.3.4 Step 2: Whether the Difference Magnitude Meets Expectations
Use precision(value ~ bottle) to split the total variability into the between-bottle component (bottle) and the within-bottle component (error):
Code
bp <- precision(bottle, value ~ bottle)
bp <- variance(bp)Variance components
sample component VC %Total SD CV[%]
---------------------------------------------
all bottle 1.3236 55.1 1.1505 1.1485
all error 1.0768 44.9 1.0377 1.0359 The between-bottle SD is ≈ 1.15, CV ≈ 1.15%, and the within-bottle SD is ≈ 1.04, CV ≈ 1.04%. Assuming the protocol specifies a total precision limit of 2% (CV), the between-bottle difference is within the limit. Note: The “limit” here is only an example; formal studies must use the limits specified in the protocol or standard, and should compare the total CV (including between- and within-bottle) and each component separately.
Code
bp <- ci(bp)Confidence intervals (SD)
sample component estimate lower upper
----------------------------------------
all total 1.5493 1.2449 2.0519
all bottle 1.1505 0.4254 1.5704
all error 1.0377 0.8807 1.2633
Confidence intervals (%CV)
sample component estimate lower upper
----------------------------------------
all total 1.5467 1.2428 2.0485
all bottle 1.1485 0.4246 1.5678
all error 1.0359 0.8792 1.2612 8.4 Example 2: Different Reagent Lots Measuring the Same Quality-Control Material
8.4.1 Reading Data
Three reagent lots (L1, L2, L3) each measure the same quality-control material 15 times:
Code
lot_data <- read_csv("./data/reagent-lot.csv", show_col_types = FALSE)
lot_data <- as.data.frame(lot_data)
lot_data$lot <- factor(lot_data$lot)
str(lot_data)'data.frame': 45 obs. of 3 variables:
$ lot : Factor w/ 3 levels "L1","L2","L3": 1 1 1 1 1 1 1 1 1 1 ...
$ replicate: num 1 2 3 4 5 6 7 8 9 10 ...
$ value : num 100 102 101 102 101 ...Code
lot_summary <- aggregate(value ~ lot, lot_data, function(x) c(n = length(x), mean = mean(x), sd = sd(x)))
lot_summary <- data.frame(lot = lot_summary$lot, lot_summary$value)
lot_summary8.4.2 Testing the Between-Lot Difference
Code
ba_lot <- bottle_anova(lot_data, value ~ lot)
print(ba_lot)
Bottle ANOVA Analysis
Response: value
Group variable: lot (3 groups)
Complete cases: 45 / 45
Confidence level: 0.95
Descriptive Statistics:
------------------------------------------------------------
lot n Mean SD
------------------------------------------------------------
L1 15 100.8173 1.1626
L2 15 101.2760 1.4271
L3 15 100.5673 0.7829
ANOVA Table:
------------------------------------------------------------
Term Df Sum Sq Mean Sq F-value p-value Sig
------------------------------------------------------------
lot 2 3.8754 1.9377 1.4528 0.2454
Residuals 42 56.0183 1.3338
------------------------------------------------------------
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Assumption Checks:
Bartlett test: K-squared = 4.6245, df = 2, p-value = 0.0990 (OK)
Shapiro-Wilk: W = 0.9817, p-value = 0.6870 (OK)
Post-hoc: use tukey() for pairwise comparisonsCode
tukey(ba_lot)
Tukey HSD Post-hoc Test
Response: value
Complete cases: 45 / 45
Confidence level: 0.95
Term: lot
------------------------------------------------------------
Comparison Diff lwr upr p-value Sig
------------------------------------------------------------
L2-L1 0.4587 -0.5659 1.4832 0.5269
L3-L1 -0.2500 -1.2745 0.7745 0.8247
L3-L2 -0.7087 -1.7332 0.3159 0.2245
------------------------------------------------------------
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Compact Letter Display:
------------------------------------------------------------
L2: a
L1: a
L3: aThe compact letter display shows between which lots the means differ significantly and in which direction. Even if the between-lot mean difference is statistically significant, it must still be judged against the pre-specified acceptable bias (for example, the allowable target-value range of the quality-control material) to determine whether it is within the acceptable range; do not draw conclusions from the p-value alone.
8.5 Key Points for Interpreting Results
- Set acceptance limits first: Before analysis, specify the allowable difference magnitude between lots/bottles (SD/CV or mean-bias limit).
- Significance ≠ acceptability: ANOVA/Tukey test whether there is a difference; whether the magnitude is acceptable is determined by the limits.
- Variance decomposition: Use variance components to separate between-bottle/lot and within-bottle variability, and evaluate the contribution of each component separately.
- Pairwise-comparison scale: With many bottles, the number of pairwise comparisons is large; use the compact letter display to focus on the difference structure.
- Consistent conditions: Between-lot comparisons should keep conditions such as method, instrument, operator, and time constant, with only the lot varying.
- Complete records: Report the mean, SD of each bottle/lot, the ANOVA table, Tukey results, variance components, and limit judgments.