In this section, the experimental result analysis of the ODCNN-CIHAD model is tested using two datasets. The first UNSW-NB15 [24] is created by an Australian security laboratory using IXIA Perfect Strom tool and the details are given in Table 1. It has 42 features (excluding the labels) and ten classes (9 attacks and 1 normal). The number of samples in the dataset before class imblancing handling is 82332 and it becomes 82300 after class balanced data as illustrated in Table 1.
Table 1
UNSW-NB15 dataset details
Label | Class Name | Before | After |
0 | Normal | 37000 | 8230 |
1 | Generic | 18871 | 8230 |
2 | Exploits | 11132 | 8230 |
3 | Fuzzers | 6062 | 8230 |
4 | DoS | 4089 | 8230 |
5 | Reconnaissance | 3496 | 8230 |
6 | Analysis | 677 | 8230 |
7 | Backdoor | 583 | 8230 |
8 | Shellcode | 378 | 8230 |
9 | Worms | 44 | 8230 |
Total Number of Samples | 82332 | 82300 |
Next, the results are inspected using Kaggle dataset [25], comprising samples under two classes. Before class imbalance handling, the number of instances in the dataset is 1567 and it becomes 2800 samples after class balance as depicted in Table 2.
Table 2
Class Name | Before | After |
Pass | 1463 | 1400 |
Fail | 104 | 1400 |
Total | 1567 | 2800 |
Table 3
Result analysis of ODCNN-CIHAD approach with various measures under UNSW-NB15 dataset
Class Labels | Accuracy | Precision | Sensitivity | Specificity | F-Score | MCC |
Training Set (70%) |
Normal | 99.30 | 95.26 | 97.89 | 99.46 | 96.56 | 96.18 |
Generic | 99.47 | 97.64 | 97.08 | 99.74 | 97.36 | 97.06 |
Exploits | 99.24 | 96.18 | 96.19 | 99.58 | 96.18 | 95.76 |
Fuzzers | 99.27 | 96.31 | 96.38 | 99.59 | 96.35 | 95.94 |
DoS | 99.25 | 95.16 | 97.47 | 99.45 | 96.30 | 95.90 |
Reconnaissance | 99.26 | 98.68 | 93.84 | 99.86 | 96.20 | 95.83 |
Analysis | 99.29 | 95.60 | 97.16 | 99.52 | 96.37 | 95.98 |
Backdoor | 99.45 | 96.98 | 97.59 | 99.66 | 97.28 | 96.98 |
Shellcode | 99.57 | 97.02 | 98.80 | 99.66 | 97.90 | 97.67 |
Worms | 99.33 | 98.54 | 94.70 | 99.84 | 96.58 | 96.23 |
Average | 99.34 | 96.74 | 96.71 | 99.63 | 96.71 | 96.35 |
Testing Set (30%) |
Normal | 99.38 | 95.68 | 98.15 | 99.51 | 96.90 | 96.57 |
Generic | 99.42 | 97.28 | 96.79 | 99.71 | 97.03 | 96.71 |
Exploits | 99.13 | 95.79 | 95.48 | 99.53 | 95.64 | 95.15 |
Fuzzers | 99.38 | 96.64 | 97.19 | 99.63 | 96.91 | 96.57 |
DoS | 99.33 | 95.55 | 97.84 | 99.50 | 96.68 | 96.32 |
Reconnaissance | 99.19 | 98.59 | 93.31 | 99.85 | 95.88 | 95.48 |
Analysis | 99.34 | 96.86 | 97.00 | 99.62 | 96.93 | 96.56 |
Backdoor | 99.33 | 95.93 | 97.41 | 99.54 | 96.67 | 96.29 |
Shellcode | 99.40 | 95.86 | 98.13 | 99.54 | 96.98 | 96.66 |
Worms | 99.42 | 98.70 | 95.43 | 99.86 | 97.04 | 96.73 |
Average | 99.33 | 96.69 | 96.67 | 99.63 | 96.67 | 96.30 |
Figure 3 demonstrates the confusion matrices accomplished by the ODCNN-CIHAD model on UNSW-NB15 dataset. The figure shows that the ODCNN-CIHAD model has proficiently categorized all the ten classes on 70% of TR data and 30% of TS data.
Table 3 provides an overall anomaly classification performance of the ODCNN-CIHAD system on 70% of TR data and 30% of TS data under UNSW-NB15 dataset. The results highlighted that the ODCNN-CIHAD model has shown enhanced performance in both aspects. For instance, with 70% of TR data, the ODCNN-CIHAD model has attained maximum average \(acc{u}_{y}\) of 99.34%, \(pre{c}_{n}\) of 96.74%, \(sen{s}_{y}\) of 96.71%, \(spe{c}_{y}\) of 99.63%, \({F}_{score}\) of 96.71%, and MCC of 96.35%. Additionally, with 30% of TS data, the ODCNN-CIHAD approach has obtained maximal average \(acc{u}_{y}\) of 99.33%, \(pre{c}_{n}\) of 96.69%, \(sen{s}_{y}\) of 96.67%, \(spe{c}_{y}\) of 99.63%, \({F}_{score}\) of 96.67%, and MCC of 96.30%.
The training accuracy (TA) and validation accuracy (VA) obtained by the ODCNN-CIHAD system on UNSW-NB15 dataset is depicted in Fig. 4. The experimental outcome revealed that the ODCNN-CIHAD technique has achieved maximal values of TA and VA. Particularly the VA appeared superior to TA.
The training loss (TL) and validation loss (VL) gained by the ODCNN-CIHAD approach on UNSW-NB15 dataset are established in Fig. 5. The experimental outcome exposed that the ODCNN-CIHAD algorithm has been able to minimal values of TL and VL. In specific, the VL is lesser than TL.
Figure 5. TL and VL analysis of ODCNN-CIHAD approach under UNSW-NB15 dataset
A clear precision-recall examination of the ODCNN-CIHAD method on UNSW-NB15 dataset is represented in Fig. 6. The figure indicated that the ODCNN-CIHAD method has resulted in enhanced values of precision-recall values under all classes.
A brief ROC analysis of the ODCNN-CIHAD method on UNSW-NB15 dataset is portrayed in Fig. 7. The outcomes exposed the ODCNN-CIHAD approach has demonstrated its ability in categorizing distinct classes on the UNSW-NB15 dataset.
Figure 6. Precision-recall analysis of ODCNN-CIHAD approach under UNSW-NB15 dataset
Table 4 reports a detailed comparative study of the ODCNN-CIHAD methodology with existing models on the test UNSW-NB15 dataset [26–28].
Figure 8 highlights the comparative study of the ODCNN-CIHAD system with existing techniques interms of \(pre{c}_{n}\), \(sen{s}_{y}\), and \({F}_{score}\) on the UNSW-NB15 dataset. The figure implied that the ODCNN-CIHAD approach has gained maximal values of \(pre{c}_{n}\), \(sen{s}_{y}\), and \({F}_{score}\)over other models. For instance, based on \(pre{c}_{n}\), the ODCNN-CIHAD model has obtained higher \(pre{c}_{n}\) of 96.69% whereas the ANN, LR, KNN, SVM, DT, SSAE, NB, VLSTM, and SSA-CRNN models have attained lower \(pre{c}_{n}\) values of 57.41%, 66.93%, 62.94%, 50.52%, 63.85%, 59.30%, 64.76%, 66.75%, and 67.01% respectively. In addition, according to \({F}_{score}\), the ODCNN-CIHAD approach has reached superior \({F}_{score}\) of 96.30% whereas the ANN, LR, KNN, SVM, DT, SSAE, NB, VLSTM, and SSA-CRNN systems have attained lesser \({F}_{score}\) values of 55.12%, 57.33%, 53.07%, 53.37%, 48.86%, 51.41%, 52.49%, 58.55%, and 59.73% correspondingly.
Figure 9 demonstrates the comparison analysis of the ODCNN-CIHAD technique with existing approaches with respect to \(acc{u}_{y}\) on the UNSW-NB15 dataset. The figure revealed that the ODCNN-CIHAD approach has reached maximal values of \(acc{u}_{y}\) over other models. For sample, based on \(pre{c}_{n}\), the ODCNN-CIHAD system has obtained higher \(acc{u}_{y}\) of 99.33% whereas the ANN, LR, KNN, SVM, DT, SSAE, NB, VLSTM, and SSA-CRNN methodologies
Table 4
Comparative analysis of ODCNN-CIHAD approach with existing methodologies under UNSW-NB15 dataset
Methods | Accuracy | Precision | Sensitivity | F-Score |
ANN Algorithm | 79.33 | 57.41 | 58.45 | 55.12 |
LR Algorithm | 63.03 | 66.93 | 55.08 | 57.33 |
kNN Algorithm | 71.09 | 62.94 | 53.38 | 53.07 |
SVM Algorithm | 64.58 | 50.52 | 53.20 | 53.37 |
DT Algorithm | 70.53 | 63.85 | 53.19 | 48.86 |
SSAE Algorithm | 88.96 | 59.30 | 58.99 | 51.41 |
NB Algorithm | 93.94 | 64.76 | 57.73 | 52.49 |
VLSTM Algorithm | 95.91 | 66.75 | 52.90 | 58.55 |
SSA-CRNN | 98.39 | 67.01 | 59.14 | 59.73 |
ODCNN-CIHAD | 99.33 | 96.69 | 96.67 | 96.30 |
have obtained reduced \(acc{u}_{y}\) values of 79.33%, 63.03%, 71.09%, 64.58%, 70.53%, 88.96%, 93.94%, 95.91%, and 98.39% correspondingly.
Figure 10 depicts the confusion matrices accomplished by the ODCNN-CIHAD methodology on Kaggle dataset. The figure demonstrated that the ODCNN-CIHAD system has proficiently categorized all the ten classes on 70% of TR data and 30% of TS data.
Table 5 offers an overall anomaly classification performance of the ODCNN-CIHAD system on 70% of TR data and 30% of TS data under Kaggle dataset. The outcomes displayed that the ODCNN-CIHAD system has outperformed higher performance in both aspects. For instance, with 70% of TR data, the ODCNN-CIHAD approach has reached higher average \(acc{u}_{y}\) of 97.70%, \(pre{c}_{n}\) of 97.70%, \(sen{s}_{y}\) of 97.74%, \(spe{c}_{y}\) of 97.74%, and \({F}_{score}\) of 97.70%. Moreover, with 30% of TS data, the ODCNN-CIHAD algorithm has attained maximal average \(acc{u}_{y}\) of 98.21%, \(pre{c}_{n}\) of 98.28%, \(sen{s}_{y}\) of 98.14%, \(spe{c}_{y}\) of 98.14%, and \({F}_{score}\) of 98.20%.
Table 5
Result analysis of ODCNN-CIHAD approach with various measures under Kaggle dataset
UCI SECOM Dataset |
Class Labels | Accuracy | Precision | Sensitivity | Specificity | F-Score |
Training Set (70%) |
Pass | 97.70 | 98.99 | 96.54 | 98.95 | 97.75 |
Fail | 97.70 | 96.41 | 98.95 | 96.54 | 97.66 |
Average | 97.70 | 97.70 | 97.74 | 97.74 | 97.70 |
Testing Set (30%) |
Pass | 98.21 | 98.95 | 97.17 | 99.11 | 98.05 |
Fail | 98.21 | 97.60 | 99.11 | 97.17 | 98.35 |
Average | 98.21 | 98.28 | 98.14 | 98.14 | 98.20 |
The TA and VA gained by the ODCNN-CIHAD methodology on Kaggle dataset are illustrated in Fig. 11. The experimental outcome exposed that the ODCNN-CIHAD technique has achieved maximal values of TA and VA. Particularly the VA performed superior to TA.
The TL and VL achieved by the ODCNN-CIHAD approach on Kaggle dataset are depicted in Fig. 12. The experimental outcome revealed that the ODCNN-CIHAD algorithm has been able to minimal values of TL and VL. In specific, the VL is lesser than TL.
A clear precision-recall examination of the ODCNN-CIHAD approach on Kaggle dataset is represented in Fig. 13. The figure exposed that the ODCNN-CIHAD methodology has resulted in higher values of precision-recall values under all classes.
A brief ROC investigation of the ODCNN-CIHAD method on Kaggle dataset is displayed in Fig. 14. The outcomes referred to as the ODCNN-CIHAD approach have outperformed its
capability in categorizing distinct classes on the Kaggle dataset.
Table 6 demonstrates detailed comparative analysis of the ODCNN-CIHAD approach with existing methodologies on the test Kaggle dataset [29]. Comparative analysis of the ODCNN-CIHAD approach with existing methodologies taken from metaheuristic feature selection with deep learning enabled cascaded recurrent neural network for anomaly detection in Industrial Internet of Things environment [30].
Figure 15 depicts the comparison examination of the ODCNN-CIHAD method with existing methodologies with respect to \(pre{c}_{n}\), \(sen{s}_{y}\), and \({F}_{score}\) on the Kaggle dataset. The figure implied that the ODCNN-CIHAD system has reached maximal values of \(pre{c}_{n}\), \(sen{s}_{y}\), and \({F}_{score}\)over other techniques. For sample, according to \(pre{c}_{n}\), the ODCNN-CIHAD approach has obtained improved \(pre{c}_{n}\) of 98.28% whereas the DNN layer1, DNN layer2, DNN layer3, DNN layer4, ensemble, PSO ensemble, and SSA-CRNN techniques have attained lesser \(pre{c}_{n}\) values of 90.36%, 89.72%, 90.48%, 90.87%, 90.08%, 91.10%, and 92.03% correspondingly. Besides, based on \({F}_{score}\), the ODCNN-CIHAD algorithm has achieved maximal \({F}_{score}\) of 98.20% whereas the DNN layer1, DNN layer2, DNN layer3, DNN layer4, ensemble, PSO ensemble, and SSA-CRNN systems have obtained minimal \({F}_{score}\) values of 90.04%, 90.65%, 86.36%, 90.60%, 90.79%, 90.61%, and 91.08% correspondingly.
Table 6
Comparative analysis of ODCNN-CIHAD approach with existing methodologies under Kaggle dataset
Methods | Accuracy | Precision | Sensitivity | F-Score |
DNN Layer-1 | 92.04 | 90.36 | 83.83 | 90.04 |
DNN Layer-2 | 92.85 | 89.72 | 86.61 | 90.65 |
DNN Layer-3 | 83.29 | 90.48 | 87.47 | 86.36 |
DNN Layer-4 | 93.27 | 90.87 | 87.98 | 90.60 |
Ensemble Model | 91.15 | 90.08 | 88.23 | 90.79 |
PSO Ensemble Model | 93.65 | 91.10 | 86.58 | 90.61 |
SSA-CRNN | 96.84 | 92.03 | 89.38 | 91.08 |
ODCNN-CIHAD | 98.21 | 98.28 | 98.14 | 98.20 |
Figure 16 examines the comparison investigation of the ODCNN-CIHAD approach with existing systems in terms of \(acc{u}_{y}\) on the Kaggle dataset. The figure referred that the ODCNN-CIHAD system has attained superior values of \(acc{u}_{y}\) over other models. For instance, based on \(acc{u}_{y}\), the ODCNN-CIHAD method has obtained maximal \(acc{u}_{y}\) of 98.21% whereas the DNN layer1, DNN layer2, DNN layer3, DNN layer4, ensemble, PSO ensemble, and SSA-CRNN approaches have gained reduced \(acc{u}_{y}\) values of 92.04%, 92.85%, 83.29%, 93.27%, 91.15%, 93.65%, and 96.84% correspondingly.
From the detailed results and discussion, it is assumed that the ODCNN-CIHAD model has gained effectual performance over other models on both datasets.