Adaptive and Resilient Machine Learning Model for Anomaly and Intrusion Detection in Modern Networks

Abstract

Due to increased complexity and frequency of attacks, we should leave behind the oldsignature-based systems used to detect networks and adopt newer and dynamic detectionsystems of anomalies. The existing static machine learning models are poor at adapting to the network environment; they are not well able to generalize to new threats and performance decays when network patterns evolve. Our study addresses these issues through designing and thoroughly developing a new Hybrid Multi-Layered Stacking Ensemble Model that is robust and resilient in anomaly detection in the network. The approach uses the combination of five various classifiers, K-nearest neighbors, Gradient Boosting, support vectors machine, random forest and logistic regression, to enhance variety as well as minimize errors. To test the model with the recent real world CICIDS 2017 data that provides numerous older and recent attacks such as DDoS, PortScan, and Botnet and the NFS-2023-TE data that concentrates on the new IoT/ Edge environments, we tested the model on both. A careful testing in terms of accuracy, precision, recall, and F1-score indicates that the stacking method is superior. More specifically, the hybrid model achieved 98.79% accuracy on the CICIDS 2017 dataset which is understandably higher than that of individual learners. Despite the perfect detection of large attacks such as DDoS, PortScan (F1 -score 1.00), the model has a significant limitation in identifying small, low-impact attacks such as Bot attacks (F1 -score 0.55, recall 0.38) which, we took a closer look at. These findings validate the fact that the combination of various ensembles increases the strength of the system. Future research will also include the deep learning models to enhance feature extraction, develop real-time drift adaptation and investigate blockchain based federated learning to provide secure and collaborative resilience in distributed networks.