Comparative performance metrics (for the basic model design) of the meta-learner in the stacked SVM and RF models, set with a labelling threshold of 0.5 and 0.65. Results suggest clear overfitting, and generalization scores are weaker and more erratic than those of our basic model presented in Table 10.
| Data set . | . | Accuracy . | Precision . | Recall . | F1 score . |
|---|---|---|---|---|---|
| Threshold set to 0.5 | |||||
| 1st Level training set (%) | SVM | 97.54 | 98.46 | 90.75 | 94.45 |
| RF | 100.0 | 100.0 | 100.0 | 100.0 | |
| Meta-learner training set (%) | SVM | 89.07 | 79.15 | 71.37 | 75.06 |
| RF | 96.75 | 98.09 | 87.61 | 92.55 | |
| Validation set (%) | SVM | 88.24 | 78.72 | 67.27 | 72.55 |
| RF | 89.71 | 81.44 | 71.82 | 76.33 | |
| Test set (%) | SVM | 88.85 | 81.71 | 66.34 | 73.22 |
| RF | 89.08 | 86.81 | 61.88 | 72.25 | |
| Threshold set to 0.65 | |||||
| 1st level training set (%) | SVM | 97.54 | 98.46 | 90.75 | 94.45 |
| RF | 100.0 | 100.0 | 100.0 | 100.0 | |
| Meta-learner training set (%) | SVM | 89.07 | 79.15 | 71.37 | 75.06 |
| RF | 94.69 | 100.0 | 76.92 | 86.96 | |
| Validation set (%) | SVM | 88.24 | 78.72 | 67.27 | 72.55 |
| RF | 90.34 | 91.03 | 64.55 | 75.53 | |
| Test set (%) | SVM | 88.85 | 81.71 | 66.34 | 73.22 |
| RF | 88.85 | 95.61 | 53.96 | 68.99 | |
| Data set . | . | Accuracy . | Precision . | Recall . | F1 score . |
|---|---|---|---|---|---|
| Threshold set to 0.5 | |||||
| 1st Level training set (%) | SVM | 97.54 | 98.46 | 90.75 | 94.45 |
| RF | 100.0 | 100.0 | 100.0 | 100.0 | |
| Meta-learner training set (%) | SVM | 89.07 | 79.15 | 71.37 | 75.06 |
| RF | 96.75 | 98.09 | 87.61 | 92.55 | |
| Validation set (%) | SVM | 88.24 | 78.72 | 67.27 | 72.55 |
| RF | 89.71 | 81.44 | 71.82 | 76.33 | |
| Test set (%) | SVM | 88.85 | 81.71 | 66.34 | 73.22 |
| RF | 89.08 | 86.81 | 61.88 | 72.25 | |
| Threshold set to 0.65 | |||||
| 1st level training set (%) | SVM | 97.54 | 98.46 | 90.75 | 94.45 |
| RF | 100.0 | 100.0 | 100.0 | 100.0 | |
| Meta-learner training set (%) | SVM | 89.07 | 79.15 | 71.37 | 75.06 |
| RF | 94.69 | 100.0 | 76.92 | 86.96 | |
| Validation set (%) | SVM | 88.24 | 78.72 | 67.27 | 72.55 |
| RF | 90.34 | 91.03 | 64.55 | 75.53 | |
| Test set (%) | SVM | 88.85 | 81.71 | 66.34 | 73.22 |
| RF | 88.85 | 95.61 | 53.96 | 68.99 | |
Comparative performance metrics (for the basic model design) of the meta-learner in the stacked SVM and RF models, set with a labelling threshold of 0.5 and 0.65. Results suggest clear overfitting, and generalization scores are weaker and more erratic than those of our basic model presented in Table 10.
| Data set . | . | Accuracy . | Precision . | Recall . | F1 score . |
|---|---|---|---|---|---|
| Threshold set to 0.5 | |||||
| 1st Level training set (%) | SVM | 97.54 | 98.46 | 90.75 | 94.45 |
| RF | 100.0 | 100.0 | 100.0 | 100.0 | |
| Meta-learner training set (%) | SVM | 89.07 | 79.15 | 71.37 | 75.06 |
| RF | 96.75 | 98.09 | 87.61 | 92.55 | |
| Validation set (%) | SVM | 88.24 | 78.72 | 67.27 | 72.55 |
| RF | 89.71 | 81.44 | 71.82 | 76.33 | |
| Test set (%) | SVM | 88.85 | 81.71 | 66.34 | 73.22 |
| RF | 89.08 | 86.81 | 61.88 | 72.25 | |
| Threshold set to 0.65 | |||||
| 1st level training set (%) | SVM | 97.54 | 98.46 | 90.75 | 94.45 |
| RF | 100.0 | 100.0 | 100.0 | 100.0 | |
| Meta-learner training set (%) | SVM | 89.07 | 79.15 | 71.37 | 75.06 |
| RF | 94.69 | 100.0 | 76.92 | 86.96 | |
| Validation set (%) | SVM | 88.24 | 78.72 | 67.27 | 72.55 |
| RF | 90.34 | 91.03 | 64.55 | 75.53 | |
| Test set (%) | SVM | 88.85 | 81.71 | 66.34 | 73.22 |
| RF | 88.85 | 95.61 | 53.96 | 68.99 | |
| Data set . | . | Accuracy . | Precision . | Recall . | F1 score . |
|---|---|---|---|---|---|
| Threshold set to 0.5 | |||||
| 1st Level training set (%) | SVM | 97.54 | 98.46 | 90.75 | 94.45 |
| RF | 100.0 | 100.0 | 100.0 | 100.0 | |
| Meta-learner training set (%) | SVM | 89.07 | 79.15 | 71.37 | 75.06 |
| RF | 96.75 | 98.09 | 87.61 | 92.55 | |
| Validation set (%) | SVM | 88.24 | 78.72 | 67.27 | 72.55 |
| RF | 89.71 | 81.44 | 71.82 | 76.33 | |
| Test set (%) | SVM | 88.85 | 81.71 | 66.34 | 73.22 |
| RF | 89.08 | 86.81 | 61.88 | 72.25 | |
| Threshold set to 0.65 | |||||
| 1st level training set (%) | SVM | 97.54 | 98.46 | 90.75 | 94.45 |
| RF | 100.0 | 100.0 | 100.0 | 100.0 | |
| Meta-learner training set (%) | SVM | 89.07 | 79.15 | 71.37 | 75.06 |
| RF | 94.69 | 100.0 | 76.92 | 86.96 | |
| Validation set (%) | SVM | 88.24 | 78.72 | 67.27 | 72.55 |
| RF | 90.34 | 91.03 | 64.55 | 75.53 | |
| Test set (%) | SVM | 88.85 | 81.71 | 66.34 | 73.22 |
| RF | 88.85 | 95.61 | 53.96 | 68.99 | |
This PDF is available to Subscribers Only
View Article Abstract & Purchase OptionsFor full access to this pdf, sign in to an existing account, or purchase an annual subscription.
