Accuracy and misinference errors of uni- and multivariate evaluations of binary ground-truth models of brain function of single source region and double source regions
Inference method . | Accuracy % . | Accuracy weighted % . | Sensitivity % . | Specificity % . | Misinference error % . |
---|---|---|---|---|---|
Single source binary | |||||
LSC (Fig. 3A) | 33 | 5 | 100 | 31 | 92 |
LSM (Fig. 3B) LSM_vc | 48 71 | 8 54 | 100 100 | 47 70 | 88 26 |
MAPP (Fig. 3C) | 100 | 100 | 100 | 100 | 0 |
MSA (Fig. 3D) | 100 | 100 | 100 | 100 | 0 |
Single source binary and 10% noise | |||||
LSC | 77 | 11 | 100 | 78 | 88 |
LSM | 74 | 11 | 100 | 73 | 88 |
MAPP | 98 | 46 | 100 | 98 | 6 |
MSA | 98 | 73 | 100 | 98 | 4 |
Double source binary synergistic | |||||
LSC | 22 | 22 | 100 | 18 | 90 |
LSM | 35 | 35 | 100 | 32 | 86 |
MAPP | 100 | 100 | 100 | 100 | 0 |
MSA | 100 | 100 | 100 | 100 | 0 |
Double source binary redundant | |||||
LSC | 39 | 39 | 88 | 37 | 91 |
LSM | 47 | 47 | 100 | 45 | 86 |
MAPP | 98 | 98 | 99 | 98 | 1 |
MSA | 93 | 92 | 100 | 92 | 12 |
Double source binary redundant with 10% noise | |||||
LSC | 80 | 79 | 100 | 79 | 89 |
LSM | 77 | 76 | 99 | 76 | 86 |
MAPP | 100 | 100 | 100 | 100 | 95 |
MSA | 98 | 98 | 100 | 98 | 23 |
Double source binary redundant with 50% noise | |||||
LSC | 79 | 78 | 99 | 78 | 90 |
LSM | 74 | 74 | 99 | 73 | 86 |
MAPP | 100 | 100 | 100 | 100 | 96 |
MSA | 97 | 96 | 97 | 97 | 53 |
Double source binary mutual inhibition | |||||
LSC | 41 | 41 | 90 | 40 | 92 |
LSM | 47 | 46 | 98 | 44 | 85 |
MAPP | 95 | 95 | 98 | 95 | 1 |
MSA | 95 | 95 | 96 | 95 | 7 |
Inference method . | Accuracy % . | Accuracy weighted % . | Sensitivity % . | Specificity % . | Misinference error % . |
---|---|---|---|---|---|
Single source binary | |||||
LSC (Fig. 3A) | 33 | 5 | 100 | 31 | 92 |
LSM (Fig. 3B) LSM_vc | 48 71 | 8 54 | 100 100 | 47 70 | 88 26 |
MAPP (Fig. 3C) | 100 | 100 | 100 | 100 | 0 |
MSA (Fig. 3D) | 100 | 100 | 100 | 100 | 0 |
Single source binary and 10% noise | |||||
LSC | 77 | 11 | 100 | 78 | 88 |
LSM | 74 | 11 | 100 | 73 | 88 |
MAPP | 98 | 46 | 100 | 98 | 6 |
MSA | 98 | 73 | 100 | 98 | 4 |
Double source binary synergistic | |||||
LSC | 22 | 22 | 100 | 18 | 90 |
LSM | 35 | 35 | 100 | 32 | 86 |
MAPP | 100 | 100 | 100 | 100 | 0 |
MSA | 100 | 100 | 100 | 100 | 0 |
Double source binary redundant | |||||
LSC | 39 | 39 | 88 | 37 | 91 |
LSM | 47 | 47 | 100 | 45 | 86 |
MAPP | 98 | 98 | 99 | 98 | 1 |
MSA | 93 | 92 | 100 | 92 | 12 |
Double source binary redundant with 10% noise | |||||
LSC | 80 | 79 | 100 | 79 | 89 |
LSM | 77 | 76 | 99 | 76 | 86 |
MAPP | 100 | 100 | 100 | 100 | 95 |
MSA | 98 | 98 | 100 | 98 | 23 |
Double source binary redundant with 50% noise | |||||
LSC | 79 | 78 | 99 | 78 | 90 |
LSM | 74 | 74 | 99 | 73 | 86 |
MAPP | 100 | 100 | 100 | 100 | 96 |
MSA | 97 | 96 | 97 | 97 | 53 |
Double source binary mutual inhibition | |||||
LSC | 41 | 41 | 90 | 40 | 92 |
LSM | 47 | 46 | 98 | 44 | 85 |
MAPP | 95 | 95 | 98 | 95 | 1 |
MSA | 95 | 95 | 96 | 95 | 7 |
LSM_vc: LSM volume corrected; TP: true positive; TN: true negative; FP: false positive; FN: false negative. Accuracy = (TP + TN)/(TP + TN + FP + FN); Accuracy_weighted = (TP*(1-var(diag))+TN*(1-sum(off_diag)))/(TP + TN + FP + FN). Sensitivity = TP/(TP + FN); Specificity = TN/(TN + FP); Mis-inference error = sum of all inferred regional contributions beyond the ones of the target/s. All the quantities are computed on the normalized significant values in the matrices (figures instead show all the values in the matrices, without excluding the nonsignificant values). For the single source binary simulation with noise we used a SVM predictor with linear kernel and cost function = 1, while for all other simulations, we used a regression tree predictor (default parameters). For all simulations without noise, we considered all the values for the regions not expected to be found by the method (FP and TN), whereas for the simulations with noise, we only considered the regions with a contribution higher than 0.001 to avoid all kinds of spurious results.
Accuracy and misinference errors of uni- and multivariate evaluations of binary ground-truth models of brain function of single source region and double source regions
Inference method . | Accuracy % . | Accuracy weighted % . | Sensitivity % . | Specificity % . | Misinference error % . |
---|---|---|---|---|---|
Single source binary | |||||
LSC (Fig. 3A) | 33 | 5 | 100 | 31 | 92 |
LSM (Fig. 3B) LSM_vc | 48 71 | 8 54 | 100 100 | 47 70 | 88 26 |
MAPP (Fig. 3C) | 100 | 100 | 100 | 100 | 0 |
MSA (Fig. 3D) | 100 | 100 | 100 | 100 | 0 |
Single source binary and 10% noise | |||||
LSC | 77 | 11 | 100 | 78 | 88 |
LSM | 74 | 11 | 100 | 73 | 88 |
MAPP | 98 | 46 | 100 | 98 | 6 |
MSA | 98 | 73 | 100 | 98 | 4 |
Double source binary synergistic | |||||
LSC | 22 | 22 | 100 | 18 | 90 |
LSM | 35 | 35 | 100 | 32 | 86 |
MAPP | 100 | 100 | 100 | 100 | 0 |
MSA | 100 | 100 | 100 | 100 | 0 |
Double source binary redundant | |||||
LSC | 39 | 39 | 88 | 37 | 91 |
LSM | 47 | 47 | 100 | 45 | 86 |
MAPP | 98 | 98 | 99 | 98 | 1 |
MSA | 93 | 92 | 100 | 92 | 12 |
Double source binary redundant with 10% noise | |||||
LSC | 80 | 79 | 100 | 79 | 89 |
LSM | 77 | 76 | 99 | 76 | 86 |
MAPP | 100 | 100 | 100 | 100 | 95 |
MSA | 98 | 98 | 100 | 98 | 23 |
Double source binary redundant with 50% noise | |||||
LSC | 79 | 78 | 99 | 78 | 90 |
LSM | 74 | 74 | 99 | 73 | 86 |
MAPP | 100 | 100 | 100 | 100 | 96 |
MSA | 97 | 96 | 97 | 97 | 53 |
Double source binary mutual inhibition | |||||
LSC | 41 | 41 | 90 | 40 | 92 |
LSM | 47 | 46 | 98 | 44 | 85 |
MAPP | 95 | 95 | 98 | 95 | 1 |
MSA | 95 | 95 | 96 | 95 | 7 |
Inference method . | Accuracy % . | Accuracy weighted % . | Sensitivity % . | Specificity % . | Misinference error % . |
---|---|---|---|---|---|
Single source binary | |||||
LSC (Fig. 3A) | 33 | 5 | 100 | 31 | 92 |
LSM (Fig. 3B) LSM_vc | 48 71 | 8 54 | 100 100 | 47 70 | 88 26 |
MAPP (Fig. 3C) | 100 | 100 | 100 | 100 | 0 |
MSA (Fig. 3D) | 100 | 100 | 100 | 100 | 0 |
Single source binary and 10% noise | |||||
LSC | 77 | 11 | 100 | 78 | 88 |
LSM | 74 | 11 | 100 | 73 | 88 |
MAPP | 98 | 46 | 100 | 98 | 6 |
MSA | 98 | 73 | 100 | 98 | 4 |
Double source binary synergistic | |||||
LSC | 22 | 22 | 100 | 18 | 90 |
LSM | 35 | 35 | 100 | 32 | 86 |
MAPP | 100 | 100 | 100 | 100 | 0 |
MSA | 100 | 100 | 100 | 100 | 0 |
Double source binary redundant | |||||
LSC | 39 | 39 | 88 | 37 | 91 |
LSM | 47 | 47 | 100 | 45 | 86 |
MAPP | 98 | 98 | 99 | 98 | 1 |
MSA | 93 | 92 | 100 | 92 | 12 |
Double source binary redundant with 10% noise | |||||
LSC | 80 | 79 | 100 | 79 | 89 |
LSM | 77 | 76 | 99 | 76 | 86 |
MAPP | 100 | 100 | 100 | 100 | 95 |
MSA | 98 | 98 | 100 | 98 | 23 |
Double source binary redundant with 50% noise | |||||
LSC | 79 | 78 | 99 | 78 | 90 |
LSM | 74 | 74 | 99 | 73 | 86 |
MAPP | 100 | 100 | 100 | 100 | 96 |
MSA | 97 | 96 | 97 | 97 | 53 |
Double source binary mutual inhibition | |||||
LSC | 41 | 41 | 90 | 40 | 92 |
LSM | 47 | 46 | 98 | 44 | 85 |
MAPP | 95 | 95 | 98 | 95 | 1 |
MSA | 95 | 95 | 96 | 95 | 7 |
LSM_vc: LSM volume corrected; TP: true positive; TN: true negative; FP: false positive; FN: false negative. Accuracy = (TP + TN)/(TP + TN + FP + FN); Accuracy_weighted = (TP*(1-var(diag))+TN*(1-sum(off_diag)))/(TP + TN + FP + FN). Sensitivity = TP/(TP + FN); Specificity = TN/(TN + FP); Mis-inference error = sum of all inferred regional contributions beyond the ones of the target/s. All the quantities are computed on the normalized significant values in the matrices (figures instead show all the values in the matrices, without excluding the nonsignificant values). For the single source binary simulation with noise we used a SVM predictor with linear kernel and cost function = 1, while for all other simulations, we used a regression tree predictor (default parameters). For all simulations without noise, we considered all the values for the regions not expected to be found by the method (FP and TN), whereas for the simulations with noise, we only considered the regions with a contribution higher than 0.001 to avoid all kinds of spurious results.
This PDF is available to Subscribers Only
View Article Abstract & Purchase OptionsFor full access to this pdf, sign in to an existing account, or purchase an annual subscription.