Table I.
Performance comparison in different cross-validation and test sets.
Test | AUC | SEa | SPa | MCC | Sw | QL | QS |
---|---|---|---|---|---|---|---|
DM3000b | 0.858 | 0.70 | 0.87 | 0.45 | 0.58 | 0.66 | 0.74 |
DM1229c | 0.860 | 0.70 | 0.87 | 0.43 | 0.57 | 0.65 | 0.73 |
SL329d | 0.886 | 0.78 | 0.85 | 0.63 | 0.63 | 0.81 | 0.64 |
SL477-Te | 0.870 | 0.77 | 0.85 | 0.63 | 0.62 | 0.80 | 0.57 |
SL477-Xf | 0.882 | 0.77 | 0.85 | 0.63 | 0.62 | 0.79 | 0.63 |
SE: Sensitivity, SP: Specificity.
Ten-fold cross validation. For the binary prediction, we adjusted threshold (0.08) of probability prediction to achieve the highest Sw.
The threshold was adjusted so that comparison was done at same specificity as DM3000 (87%).
Trained by DM4229 and independently tested on SL329.
Ten-fold cross validation on SL477 only. Threshold was adjusted to yield the same specificity as SL329 (85%).
Trained by X-ray structures only (DX4080) and independently tested on SL477. Threshold was adjusted to yield the same specificity as SL329 (85%).