Code
library(ivdtools)
library(readr)14 ROC Curve and Cutoff Determination
14.1 Analysis Objectives
The receiver operating characteristic (ROC) curve evaluates the ability of a quantitative marker to distinguish positive from negative outcomes, and is commonly used to establish a diagnostic cutoff. The roc() function in ivdtools provides a progressive workflow: roc() creates the object → describe() → auc() (AUC and confidence interval) → cutoff() (Youden and closest-corner optimal cutoffs) → plot() and summary().
This document demonstrates the complete ROC analysis workflow with a complete single-marker example. The reference variable must be binary; if the original reference is a continuous variable, first convert it to binary (see continuous_to_binary() in 《Qualitative Analysis》) and record the threshold and positive direction.
14.2 Overview of Functions
| Function | Main purpose | Key input or output |
|---|---|---|
roc() |
Create a ROC object | cols, reference, id, positive |
describe() |
Marker descriptive statistics | reference forced to be treated as categorical |
auc() |
AUC and confidence interval | per marker |
cutoff() |
Optimal cutoff | Youden and closest corner; requires auc() first |
plot() |
ROC curve | cutpoint marks the optimal point |
summary() |
Summary |
14.3 Example: Single-Marker ROC Analysis
14.3.1 Reading and Validating Data
Code
roc_dat <- read_csv("./data/roc-example.csv", show_col_types = FALSE)
roc_dat <- as.data.frame(roc_dat)
str(roc_dat)'data.frame': 100 obs. of 3 variables:
$ sid: num 1 2 3 4 5 6 7 8 9 10 ...
$ ref: num 0 0 0 0 0 0 0 0 0 0 ...
$ x1 : num 7.59 8.31 5.89 5.59 9.57 ...Code
table(roc_dat$ref)
0 1
50 50 The reference column ref is binary 0/1, with 50 positive and 50 negative cases.
14.3.2 Creating the Object and Description
Code
ro <- roc(roc_dat, cols = "x1", reference = "ref", id = "sid")
ro
ROC Analysis
Data class: data.frame
Total rows: 100
Complete cases: 100
Reference: ref (binary)
Positive / Neg: 50 / 50
Evaluation cols: 1 (x1)
Duplicate IDs: none
Analysis slots:
[ ] Describe
[ ] AUC
[ ] Cutoff
[ ] MLRCode
ro <- describe(ro)
Descriptive Statistics
Variable n Mean SD Min Q1 Median Q3 Max NA
---------------------------------------------------------------------------
x1 100 9.2576 4.4744 -0.3894 5.8224 8.5472 12.9591 18.7295
ref
Level Count Percent
--------------------------------------
0 50 50.0%
1 50 50.0%Parameter notes: The
colsofroc()are the quantitative marker columns;referenceis the binary reference column (either numeric 0/1, a two-level factor, or a two-value character is accepted);positivespecifies the positive level (when not given, the last sorted level is taken by default). The direction of each marker (geq/leq) is determined automatically from the mean of the positive group.
14.3.3 AUC
Code
ro <- auc(ro)
AUC Table
Column AUC 95% CI Direction
------------------------------------------------------------
x1 0.9420 [0.8940, 0.9900] geqCode
ro$auc_tableThe AUC of marker x1 is ≈ 0.942 (95% CI [0.894, 0.990]), indicating strong discrimination.
14.3.4 Optimal Cutoff
Code
ro <- cutoff(ro)
Optimal Cutoff Analysis
Column: x1
Youden / Corner: cutoff = 8.7617 (sens = 0.9000, spec = 0.9200, Youden = 0.8200)
Full cutoff table:
Cutoff Sens Spec TP FP TN FN Youden
----------------------------------------------------------------------
-Inf 1.0000 0.0000 50 50 0 0 0.0000
18.7295 0.0200 1.0000 1 0 50 49 0.0200
18.5675 0.0400 1.0000 2 0 50 48 0.0400
18.2248 0.0600 1.0000 3 0 50 47 0.0600
17.7837 0.0800 1.0000 4 0 50 46 0.0800
16.4453 0.1000 1.0000 5 0 50 45 0.1000
15.8893 0.1200 1.0000 6 0 50 44 0.1200
15.8087 0.1400 1.0000 7 0 50 43 0.1400
15.6058 0.1600 1.0000 8 0 50 42 0.1600
15.5003 0.1800 1.0000 9 0 50 41 0.1800
15.2855 0.2000 1.0000 10 0 50 40 0.2000
15.0168 0.2200 1.0000 11 0 50 39 0.2200
14.8639 0.2400 1.0000 12 0 50 38 0.2400
14.6980 0.2600 1.0000 13 0 50 37 0.2600
14.4757 0.2800 1.0000 14 0 50 36 0.2800
14.4469 0.3000 1.0000 15 0 50 35 0.3000
14.4042 0.3200 1.0000 16 0 50 34 0.3200
14.1952 0.3400 1.0000 17 0 50 33 0.3400
14.1938 0.3600 1.0000 18 0 50 32 0.3600
13.9382 0.3800 1.0000 19 0 50 31 0.3800
13.7845 0.4000 1.0000 20 0 50 30 0.4000
13.3718 0.4200 1.0000 21 0 50 29 0.4200
13.0699 0.4400 1.0000 22 0 50 28 0.4400
12.9970 0.4600 1.0000 23 0 50 27 0.4600
12.9854 0.4800 1.0000 24 0 50 26 0.4800
12.9659 0.5000 1.0000 25 0 50 25 0.5000
12.9569 0.5200 1.0000 26 0 50 24 0.5200
12.7812 0.5400 1.0000 27 0 50 23 0.5400
12.7534 0.5600 1.0000 28 0 50 22 0.5600
12.5227 0.5800 1.0000 29 0 50 21 0.5800
12.3440 0.6000 1.0000 30 0 50 20 0.6000
12.2962 0.6200 1.0000 31 0 50 19 0.6200
12.1394 0.6400 1.0000 32 0 50 18 0.6400
12.1354 0.6600 1.0000 33 0 50 17 0.6600
12.0393 0.6800 1.0000 34 0 50 16 0.6800
11.7418 0.6800 0.9800 34 1 49 16 0.6600
11.6642 0.7000 0.9800 35 1 49 15 0.6800
11.5082 0.7000 0.9600 35 2 48 15 0.6600
11.4038 0.7200 0.9600 36 2 48 14 0.6800
11.2133 0.7400 0.9600 37 2 48 13 0.7000
11.0696 0.7600 0.9600 38 2 48 12 0.7200
10.3902 0.7800 0.9600 39 2 48 11 0.7400
10.1719 0.8000 0.9600 40 2 48 10 0.7600
9.8836 0.8200 0.9600 41 2 48 9 0.7800
9.5685 0.8200 0.9400 41 3 47 9 0.7600
9.3526 0.8400 0.9400 42 3 47 8 0.7800
9.1944 0.8600 0.9400 43 3 47 7 0.8000
9.1032 0.8800 0.9400 44 3 47 6 0.8200
8.8558 0.8800 0.9200 44 4 46 6 0.8000
8.7617 0.9000 0.9200 45 4 46 5 0.8200
8.5538 0.9000 0.9000 45 5 45 5 0.8000
8.5406 0.9000 0.8800 45 6 44 5 0.7800
8.5296 0.9200 0.8800 46 6 44 4 0.8000
8.5259 0.9200 0.8600 46 7 43 4 0.7800
8.3125 0.9200 0.8400 46 8 42 4 0.7600
8.2985 0.9200 0.8200 46 9 41 4 0.7400
7.9078 0.9200 0.8000 46 10 40 4 0.7200
7.8493 0.9200 0.7800 46 11 39 4 0.7000
7.6187 0.9200 0.7600 46 12 38 4 0.6800
7.5941 0.9200 0.7400 46 13 37 4 0.6600
7.5556 0.9200 0.7200 46 14 36 4 0.6400
7.5150 0.9200 0.7000 46 15 35 4 0.6200
7.4355 0.9200 0.6800 46 16 34 4 0.6000
7.1837 0.9200 0.6600 46 17 33 4 0.5800
6.8235 0.9400 0.6600 47 17 33 3 0.6000
6.8029 0.9400 0.6400 47 18 32 3 0.5800
6.7765 0.9400 0.6200 47 19 31 3 0.5600
6.6770 0.9400 0.6000 47 20 30 3 0.5400
6.5446 0.9400 0.5800 47 21 29 3 0.5200
6.5440 0.9600 0.5800 48 21 29 2 0.5400
6.1603 0.9600 0.5600 48 22 28 2 0.5200
6.0665 0.9600 0.5400 48 23 27 2 0.5000
6.0554 0.9600 0.5200 48 24 26 2 0.4800
6.0361 0.9600 0.5000 48 25 25 2 0.4600
5.9000 0.9600 0.4800 48 26 24 2 0.4400
5.8857 0.9600 0.4600 48 27 23 2 0.4200
5.6326 0.9600 0.4400 48 28 22 2 0.4000
5.6281 0.9600 0.4200 48 29 21 2 0.3800
5.5851 0.9600 0.4000 48 30 20 2 0.3600
5.5802 0.9600 0.3800 48 31 19 2 0.3400
5.4514 0.9600 0.3600 48 32 18 2 0.3200
5.3207 0.9600 0.3400 48 33 17 2 0.3000
5.1582 0.9600 0.3200 48 34 16 2 0.2800
5.0599 0.9600 0.3000 48 35 15 2 0.2600
5.0167 0.9600 0.2800 48 36 14 2 0.2400
4.5654 0.9600 0.2600 48 37 13 2 0.2200
3.9706 0.9800 0.2600 49 37 13 1 0.2400
3.9552 0.9800 0.2400 49 38 12 1 0.2200
3.9103 1.0000 0.2400 50 38 12 0 0.2400
3.7719 1.0000 0.2200 50 39 11 0 0.2200
3.5881 1.0000 0.2000 50 40 10 0 0.2000
3.4583 1.0000 0.1800 50 41 9 0 0.1800
3.4422 1.0000 0.1600 50 42 8 0 0.1600
2.9673 1.0000 0.1400 50 43 7 0 0.1400
2.9111 1.0000 0.1200 50 44 6 0 0.1200
2.7698 1.0000 0.1000 50 45 5 0 0.1000
2.5203 1.0000 0.0800 50 46 4 0 0.0800
2.3646 1.0000 0.0600 50 47 3 0 0.0600
2.2596 1.0000 0.0400 50 48 2 0 0.0400
0.4733 1.0000 0.0200 50 49 1 0 0.0200
-0.3894 1.0000 0.0000 50 50 0 0 0.0000
Inf 0.0000 1.0000 0 0 50 50 0.0000Code
print(ro$cutoff_table)
Optimal Cutoff Analysis
Column: x1
Youden / Corner: cutoff = 8.7617 (sens = 0.9000, spec = 0.9200, Youden = 0.8200)
Full cutoff table:
Cutoff Sens Spec TP FP TN FN Youden
----------------------------------------------------------------------
-Inf 1.0000 0.0000 50 50 0 0 0.0000
18.7295 0.0200 1.0000 1 0 50 49 0.0200
18.5675 0.0400 1.0000 2 0 50 48 0.0400
18.2248 0.0600 1.0000 3 0 50 47 0.0600
17.7837 0.0800 1.0000 4 0 50 46 0.0800
16.4453 0.1000 1.0000 5 0 50 45 0.1000
15.8893 0.1200 1.0000 6 0 50 44 0.1200
15.8087 0.1400 1.0000 7 0 50 43 0.1400
15.6058 0.1600 1.0000 8 0 50 42 0.1600
15.5003 0.1800 1.0000 9 0 50 41 0.1800
15.2855 0.2000 1.0000 10 0 50 40 0.2000
15.0168 0.2200 1.0000 11 0 50 39 0.2200
14.8639 0.2400 1.0000 12 0 50 38 0.2400
14.6980 0.2600 1.0000 13 0 50 37 0.2600
14.4757 0.2800 1.0000 14 0 50 36 0.2800
14.4469 0.3000 1.0000 15 0 50 35 0.3000
14.4042 0.3200 1.0000 16 0 50 34 0.3200
14.1952 0.3400 1.0000 17 0 50 33 0.3400
14.1938 0.3600 1.0000 18 0 50 32 0.3600
13.9382 0.3800 1.0000 19 0 50 31 0.3800
13.7845 0.4000 1.0000 20 0 50 30 0.4000
13.3718 0.4200 1.0000 21 0 50 29 0.4200
13.0699 0.4400 1.0000 22 0 50 28 0.4400
12.9970 0.4600 1.0000 23 0 50 27 0.4600
12.9854 0.4800 1.0000 24 0 50 26 0.4800
12.9659 0.5000 1.0000 25 0 50 25 0.5000
12.9569 0.5200 1.0000 26 0 50 24 0.5200
12.7812 0.5400 1.0000 27 0 50 23 0.5400
12.7534 0.5600 1.0000 28 0 50 22 0.5600
12.5227 0.5800 1.0000 29 0 50 21 0.5800
12.3440 0.6000 1.0000 30 0 50 20 0.6000
12.2962 0.6200 1.0000 31 0 50 19 0.6200
12.1394 0.6400 1.0000 32 0 50 18 0.6400
12.1354 0.6600 1.0000 33 0 50 17 0.6600
12.0393 0.6800 1.0000 34 0 50 16 0.6800
11.7418 0.6800 0.9800 34 1 49 16 0.6600
11.6642 0.7000 0.9800 35 1 49 15 0.6800
11.5082 0.7000 0.9600 35 2 48 15 0.6600
11.4038 0.7200 0.9600 36 2 48 14 0.6800
11.2133 0.7400 0.9600 37 2 48 13 0.7000
11.0696 0.7600 0.9600 38 2 48 12 0.7200
10.3902 0.7800 0.9600 39 2 48 11 0.7400
10.1719 0.8000 0.9600 40 2 48 10 0.7600
9.8836 0.8200 0.9600 41 2 48 9 0.7800
9.5685 0.8200 0.9400 41 3 47 9 0.7600
9.3526 0.8400 0.9400 42 3 47 8 0.7800
9.1944 0.8600 0.9400 43 3 47 7 0.8000
9.1032 0.8800 0.9400 44 3 47 6 0.8200
8.8558 0.8800 0.9200 44 4 46 6 0.8000
8.7617 0.9000 0.9200 45 4 46 5 0.8200
8.5538 0.9000 0.9000 45 5 45 5 0.8000
8.5406 0.9000 0.8800 45 6 44 5 0.7800
8.5296 0.9200 0.8800 46 6 44 4 0.8000
8.5259 0.9200 0.8600 46 7 43 4 0.7800
8.3125 0.9200 0.8400 46 8 42 4 0.7600
8.2985 0.9200 0.8200 46 9 41 4 0.7400
7.9078 0.9200 0.8000 46 10 40 4 0.7200
7.8493 0.9200 0.7800 46 11 39 4 0.7000
7.6187 0.9200 0.7600 46 12 38 4 0.6800
7.5941 0.9200 0.7400 46 13 37 4 0.6600
7.5556 0.9200 0.7200 46 14 36 4 0.6400
7.5150 0.9200 0.7000 46 15 35 4 0.6200
7.4355 0.9200 0.6800 46 16 34 4 0.6000
7.1837 0.9200 0.6600 46 17 33 4 0.5800
6.8235 0.9400 0.6600 47 17 33 3 0.6000
6.8029 0.9400 0.6400 47 18 32 3 0.5800
6.7765 0.9400 0.6200 47 19 31 3 0.5600
6.6770 0.9400 0.6000 47 20 30 3 0.5400
6.5446 0.9400 0.5800 47 21 29 3 0.5200
6.5440 0.9600 0.5800 48 21 29 2 0.5400
6.1603 0.9600 0.5600 48 22 28 2 0.5200
6.0665 0.9600 0.5400 48 23 27 2 0.5000
6.0554 0.9600 0.5200 48 24 26 2 0.4800
6.0361 0.9600 0.5000 48 25 25 2 0.4600
5.9000 0.9600 0.4800 48 26 24 2 0.4400
5.8857 0.9600 0.4600 48 27 23 2 0.4200
5.6326 0.9600 0.4400 48 28 22 2 0.4000
5.6281 0.9600 0.4200 48 29 21 2 0.3800
5.5851 0.9600 0.4000 48 30 20 2 0.3600
5.5802 0.9600 0.3800 48 31 19 2 0.3400
5.4514 0.9600 0.3600 48 32 18 2 0.3200
5.3207 0.9600 0.3400 48 33 17 2 0.3000
5.1582 0.9600 0.3200 48 34 16 2 0.2800
5.0599 0.9600 0.3000 48 35 15 2 0.2600
5.0167 0.9600 0.2800 48 36 14 2 0.2400
4.5654 0.9600 0.2600 48 37 13 2 0.2200
3.9706 0.9800 0.2600 49 37 13 1 0.2400
3.9552 0.9800 0.2400 49 38 12 1 0.2200
3.9103 1.0000 0.2400 50 38 12 0 0.2400
3.7719 1.0000 0.2200 50 39 11 0 0.2200
3.5881 1.0000 0.2000 50 40 10 0 0.2000
3.4583 1.0000 0.1800 50 41 9 0 0.1800
3.4422 1.0000 0.1600 50 42 8 0 0.1600
2.9673 1.0000 0.1400 50 43 7 0 0.1400
2.9111 1.0000 0.1200 50 44 6 0 0.1200
2.7698 1.0000 0.1000 50 45 5 0 0.1000
2.5203 1.0000 0.0800 50 46 4 0 0.0800
2.3646 1.0000 0.0600 50 47 3 0 0.0600
2.2596 1.0000 0.0400 50 48 2 0 0.0400
0.4733 1.0000 0.0200 50 49 1 0 0.0200
-0.3894 1.0000 0.0000 50 50 0 0 0.0000
Inf 0.0000 1.0000 0 0 50 50 0.0000cutoff() gives all candidate cutoffs with their corresponding sensitivity, specificity, and Youden index, and marks the optimal one with two criteria: Youden (maximizing Se + Sp − 1) and closest corner (minimizing the distance to (0,1)). In this example, the Youden/Corner optimal cutoff is 8.76 (sensitivity 0.90, specificity 0.92, Youden 0.82).
Parameter notes:
cutoff()must be executed afterauc();plot(cutpoint=...)requirescutoff()to have been run.cutpointcan be"Youden","Corner","all", orNULL.
14.3.5 Graphics and Summary
Code
plot(ro, cutpoint = "Youden")
Code
summary(ro)
ROC Analysis -- Summary
--------------------------------------------------
ROC Analysis
Data class: data.frame
Total rows: 100
Complete cases: 100
Reference: ref (binary)
Positive / Neg: 50 / 50
Evaluation cols: 1 (x1)
Duplicate IDs: none
Analysis slots:
[x] Describe
[x] AUC
[ ] Cutoff
[ ] MLR
Descriptive Statistics
Variable n Mean SD Min Q1 Median Q3 Max NA
---------------------------------------------------------------------------
x1 100 9.2576 4.4744 -0.3894 5.8224 8.5472 12.9591 18.7295
ref
Level Count Percent
--------------------------------------
0 50 50.0%
1 50 50.0%
AUC Table
Column AUC 95% CI Direction
------------------------------------------------------------
x1 0.9420 [0.8940, 0.9900] geq
Optimal Cutoff Analysis
Column: x1
Youden / Corner: cutoff = 8.7617 (sens = 0.9000, spec = 0.9200, Youden = 0.8200)
Full cutoff table:
Cutoff Sens Spec TP FP TN FN Youden
----------------------------------------------------------------------
-Inf 1.0000 0.0000 50 50 0 0 0.0000
18.7295 0.0200 1.0000 1 0 50 49 0.0200
18.5675 0.0400 1.0000 2 0 50 48 0.0400
18.2248 0.0600 1.0000 3 0 50 47 0.0600
17.7837 0.0800 1.0000 4 0 50 46 0.0800
16.4453 0.1000 1.0000 5 0 50 45 0.1000
15.8893 0.1200 1.0000 6 0 50 44 0.1200
15.8087 0.1400 1.0000 7 0 50 43 0.1400
15.6058 0.1600 1.0000 8 0 50 42 0.1600
15.5003 0.1800 1.0000 9 0 50 41 0.1800
15.2855 0.2000 1.0000 10 0 50 40 0.2000
15.0168 0.2200 1.0000 11 0 50 39 0.2200
14.8639 0.2400 1.0000 12 0 50 38 0.2400
14.6980 0.2600 1.0000 13 0 50 37 0.2600
14.4757 0.2800 1.0000 14 0 50 36 0.2800
14.4469 0.3000 1.0000 15 0 50 35 0.3000
14.4042 0.3200 1.0000 16 0 50 34 0.3200
14.1952 0.3400 1.0000 17 0 50 33 0.3400
14.1938 0.3600 1.0000 18 0 50 32 0.3600
13.9382 0.3800 1.0000 19 0 50 31 0.3800
13.7845 0.4000 1.0000 20 0 50 30 0.4000
13.3718 0.4200 1.0000 21 0 50 29 0.4200
13.0699 0.4400 1.0000 22 0 50 28 0.4400
12.9970 0.4600 1.0000 23 0 50 27 0.4600
12.9854 0.4800 1.0000 24 0 50 26 0.4800
12.9659 0.5000 1.0000 25 0 50 25 0.5000
12.9569 0.5200 1.0000 26 0 50 24 0.5200
12.7812 0.5400 1.0000 27 0 50 23 0.5400
12.7534 0.5600 1.0000 28 0 50 22 0.5600
12.5227 0.5800 1.0000 29 0 50 21 0.5800
12.3440 0.6000 1.0000 30 0 50 20 0.6000
12.2962 0.6200 1.0000 31 0 50 19 0.6200
12.1394 0.6400 1.0000 32 0 50 18 0.6400
12.1354 0.6600 1.0000 33 0 50 17 0.6600
12.0393 0.6800 1.0000 34 0 50 16 0.6800
11.7418 0.6800 0.9800 34 1 49 16 0.6600
11.6642 0.7000 0.9800 35 1 49 15 0.6800
11.5082 0.7000 0.9600 35 2 48 15 0.6600
11.4038 0.7200 0.9600 36 2 48 14 0.6800
11.2133 0.7400 0.9600 37 2 48 13 0.7000
11.0696 0.7600 0.9600 38 2 48 12 0.7200
10.3902 0.7800 0.9600 39 2 48 11 0.7400
10.1719 0.8000 0.9600 40 2 48 10 0.7600
9.8836 0.8200 0.9600 41 2 48 9 0.7800
9.5685 0.8200 0.9400 41 3 47 9 0.7600
9.3526 0.8400 0.9400 42 3 47 8 0.7800
9.1944 0.8600 0.9400 43 3 47 7 0.8000
9.1032 0.8800 0.9400 44 3 47 6 0.8200
8.8558 0.8800 0.9200 44 4 46 6 0.8000
8.7617 0.9000 0.9200 45 4 46 5 0.8200
8.5538 0.9000 0.9000 45 5 45 5 0.8000
8.5406 0.9000 0.8800 45 6 44 5 0.7800
8.5296 0.9200 0.8800 46 6 44 4 0.8000
8.5259 0.9200 0.8600 46 7 43 4 0.7800
8.3125 0.9200 0.8400 46 8 42 4 0.7600
8.2985 0.9200 0.8200 46 9 41 4 0.7400
7.9078 0.9200 0.8000 46 10 40 4 0.7200
7.8493 0.9200 0.7800 46 11 39 4 0.7000
7.6187 0.9200 0.7600 46 12 38 4 0.6800
7.5941 0.9200 0.7400 46 13 37 4 0.6600
7.5556 0.9200 0.7200 46 14 36 4 0.6400
7.5150 0.9200 0.7000 46 15 35 4 0.6200
7.4355 0.9200 0.6800 46 16 34 4 0.6000
7.1837 0.9200 0.6600 46 17 33 4 0.5800
6.8235 0.9400 0.6600 47 17 33 3 0.6000
6.8029 0.9400 0.6400 47 18 32 3 0.5800
6.7765 0.9400 0.6200 47 19 31 3 0.5600
6.6770 0.9400 0.6000 47 20 30 3 0.5400
6.5446 0.9400 0.5800 47 21 29 3 0.5200
6.5440 0.9600 0.5800 48 21 29 2 0.5400
6.1603 0.9600 0.5600 48 22 28 2 0.5200
6.0665 0.9600 0.5400 48 23 27 2 0.5000
6.0554 0.9600 0.5200 48 24 26 2 0.4800
6.0361 0.9600 0.5000 48 25 25 2 0.4600
5.9000 0.9600 0.4800 48 26 24 2 0.4400
5.8857 0.9600 0.4600 48 27 23 2 0.4200
5.6326 0.9600 0.4400 48 28 22 2 0.4000
5.6281 0.9600 0.4200 48 29 21 2 0.3800
5.5851 0.9600 0.4000 48 30 20 2 0.3600
5.5802 0.9600 0.3800 48 31 19 2 0.3400
5.4514 0.9600 0.3600 48 32 18 2 0.3200
5.3207 0.9600 0.3400 48 33 17 2 0.3000
5.1582 0.9600 0.3200 48 34 16 2 0.2800
5.0599 0.9600 0.3000 48 35 15 2 0.2600
5.0167 0.9600 0.2800 48 36 14 2 0.2400
4.5654 0.9600 0.2600 48 37 13 2 0.2200
3.9706 0.9800 0.2600 49 37 13 1 0.2400
3.9552 0.9800 0.2400 49 38 12 1 0.2200
3.9103 1.0000 0.2400 50 38 12 0 0.2400
3.7719 1.0000 0.2200 50 39 11 0 0.2200
3.5881 1.0000 0.2000 50 40 10 0 0.2000
3.4583 1.0000 0.1800 50 41 9 0 0.1800
3.4422 1.0000 0.1600 50 42 8 0 0.1600
2.9673 1.0000 0.1400 50 43 7 0 0.1400
2.9111 1.0000 0.1200 50 44 6 0 0.1200
2.7698 1.0000 0.1000 50 45 5 0 0.1000
2.5203 1.0000 0.0800 50 46 4 0 0.0800
2.3646 1.0000 0.0600 50 47 3 0 0.0600
2.2596 1.0000 0.0400 50 48 2 0 0.0400
0.4733 1.0000 0.0200 50 49 1 0 0.0200
-0.3894 1.0000 0.0000 50 50 0 0 0.0000
Inf 0.0000 1.0000 0 0 50 50 0.000014.4 Key Points for Interpreting Results
- Reference reliable: ROC conclusions depend on the reference standard; errors in the reference propagate to the AUC and cutoff.
- Direction confirmed: Confirm that the marker’s judgment direction (geq/leq) is clinically meaningful.
- Cutoff selection: Youden and closest corner are two criteria; the final cutoff should be determined in light of clinical costs (false-negative/false-positive weights) and the protocol, not only statistical optimality.
- Sample size: The AUC confidence interval varies with sample size and event count; interpret cautiously with small samples or too few events.
- Method limitations: This workflow is an empirical ROC/trapezoidal AUC; paired multi-curve comparison, partial AUC, etc., require separate designs.
- Independent validation: The AUC on the training data is the apparent performance; it should be validated on independent data before generalization.