The increasing complexity of cyber threats requires the development of effective methods for detecting and classifying attacks in network traffic. This study analyzes the effectiveness of three popular machine learning algorithms: Random Forest, which is used for anomaly detection, Support Vector Machines (SVM), which performs cyber threat classification, and autoencoders, which are used for data preprocessing and deep traffic analysis. Considering each method's advantages, a combined model is proposed that combines the capabilities of these algorithms, increasing the efficiency of threat detection and optimizing the attack classification process. The experiments demonstrate that the proposed approach increases attack detection accuracy by 3–7% and reduces response time compared to using individual machine learning algorithms. In addition, the combined model contributes to resource optimization, which is a critical factor for deploying scalable solutions in real-world conditions. Special attention is paid to the hybrid model that combines autoencoders and the Random Forest algorithm. Its effectiveness has been confirmed in test environments, where better results in detecting network anomalies were demonstrated compared to the use of only one algorithm. The proposed approach allows not only to increase the level of protection of information systems, but also to provide flexibility in configuration, which makes the model suitable for use in a wide range of cybersecurity tasks. The results obtained can be used to improve existing cyber protection methods, in particular in the field of critical infrastructure protection, where timely detection of threats is a key security factor. The high efficiency of the combined approach confirms its feasibility for implementation in network traffic monitoring systems and cybersecurity in general.
[1] U. Islam, A. Muhammad, R. Mansoor, M. S. Hossain, I. Ahmad, E. T. Eldin, J. A. Khan, A. U. Rehman, and M. Shafiq, “Detection of Distributed Denial of Service (DDoS) Attacks in IoT Based Monitoring System of Banking Sector Using Machine Learning Models,” Sustainability, vol. 14, no. 14, p. 8374, Jul. 2022, doi: 10.3390/su14148374
[2] C. Liu, J. Yang, and J. Wu, “Web intrusion detection system combined with feature analysis and SVM optimization,” EURASIP Journal on Wireless Communications and Networking, vol. 2020, no. 1, pp. 1–14, doi: 10.1186/s13638-019-1591-1
[3] W. Song, M. Beshley, K. Przystupa, H. Beshley, O. Kochan, A. Pryslupskyi, D. Pieniak, and J. Su, “A Software Deep Packet Inspection System for Network Traffic Analysis and Anomaly Detection,” Sensors, vol. 20, no. 6, p. 1637, Mar. 2020, doi: 10.3390/s20061637
[4] M. M. Klymash and M. M. Panchenko, “Packet Delay Monitoring System in Software-Configured Telecommunication Networks,” in Proc. of the International Scientific and Technical Conference “Telecommunications Perspectives,” 2016. (In Ukrainian)
[5] E. Pantelidis, G. Bendiab, S. Shiaeles, and N. Kolokotronis, “Insider Detection Using Deep Autoencoder and Variational Autoencoder Neural Networks,” in Proc. 2021 IEEE Int. Conf. on Cyber Security and Resilience (CSR), Rhodes, Greece, Jul. 2021, pp. 155–160, doi: 10.1109/CSR51186.2021.9527954
[6] Z. S. Mahdi, R. M. Zaki, and L. Alzubaidi, “Advanced hybrid techniques for cyberattack detection and prevention,” Security and Privacy, vol. 7, no. 2, pp. e471, 2024, doi: 10.1002/spy2.471
[7] M. Injadat, A. Moubayed, A. B. Nassif, and A. Shami, “Multi-Stage Optimized Machine Learning Framework for Network Intrusion Detection,” IEEE Transactions on Network and Service Management, Jun. 2021, vol. 18, no. 2, pp. 1803–1816, doi: 10.1109/TNSM.2020.3014929
[8] CICIDS 2017 Dataset, Canadian Institute for Cybersecurity, University of New Brunswick. [Online]. Available: https://www.unb.ca/cic/datasets/ids-2017.html (accessed: 10.03.2025).
[9] P. Prakriti, “Cyber threat detection using machine learning,” Int. J. Sci. Res. Eng. Manag. (IJSREM), Apr. 2024, vol. 4, no. 4, pp. 1–6, doi: 10.55041/IJSREM36799
[10] I. Khlevna and B. Koval, “Parallel and distributed machine learning techniques for anomaly detection systems,” in Proc. of the Int. Workshop on Cybersecurity Providing in Information and Telecommunication Systems (CPITS 2023), CEUR Workshop Proc., vol. 3624, pp. 131–138. [https://ceur-ws.org/Vol-3624/Paper_16.pdf]
[11] H. Kamal and M. Mashaly, “Enhanced hybrid deep learning models-based anomaly detection method for two-stage binary and multi-class classification of attacks in intrusion detection systems,” Algorithms, vol. 18, no. 2, p. 69, Feb. 2025, doi: 10.3390/a18020069
[12] H. Torabi, “Practical autoencoder-based anomaly detection by using vector reconstruction error in cloud computing networks,” Cybersecurity, vol. 5, no. 1, p. 9, 2022, doi: 10.1186/s42400-022-00134-9
[13] E. E. Abdallah, W. Eleisah, and A. F. Otoom, “Intrusion detection systems using supervised machine learning techniques: A survey,” Procedia Computer Science, vol. 201, pp. 125–132, 2022, doi: 10.1016/j.procs.2022.03.029