Abstract

Host-based intrusion detection systems (HIDS) represent a critical layer of defense against cyberattacks that bypass perimeter controls or originate from within a host environment. Despite significant progress, existing solutions continue to suffer from high false-positive rates, susceptibility to adversarial evasion, and an inherent inability to detect zero-day threats. This research investigates artificial intelligence and machine learning techniques to advance host-based intrusion detection through two complementary experiments. First, a Random Forest classifier is trained and evaluated on the ADFA-WD benchmark dataset using a TF-IDF and n-gram preprocessing pipeline, establishing a reproducible performance baseline (ROC-AUC: 0.74, F1-Score: 0.12). Second, a novel eBPF-based collection framework is designed for Linux systems, pairing kernel-level system call telemetry with an LSTM Autoencoder trained exclusively on normal behavioral sequences. Evaluated on a controlled synthetic dataset of 8,417 behavioral sessions simulating realistic Linux web server attack scenarios, the LSTM Autoencoder achieves an F1-Score of 0.66 and ROC-AUC of 0.81, demonstrating the architectural superiority of sequential, context-aware modeling over traditional ensemble approaches.

References

• S. Forrest, S. A. Hofmeyr, A. Somayaji, and T. A. Longstaff, “A sense of self for Unix processes,” Proc. IEEE Symp. Security and Privacy, pp. 120–128, 1996.
• R. Sekar, M. Bendre, D. Dhurjati, and P. Bollineni, “A fast automaton-based method for detecting anomalous program behaviors,” Proc. IEEE Symp. Security and Privacy, pp. 144–155, 2001.
• D. Kim, H. Lee, S. Cho, and B. Noh, “A recurrent neural network based approach for intrusion detection,” Proc. IEEE Intl. Conf. Big Data, pp. 2828–2836, 2016.
• J. Choi, H. Kim, and C. Choi, “Intrusion detection system combined with CNN and RNN for system call sequences,” Proc. PlatCon, pp. 1–5, 2017.
• A. Javaid, Q. Niyaz, W. Sun, and M. Alam, “A deep learning approach for network intrusion detection system,” Proc. 9th EAI BICT Conf., pp. 21–26, 2016.
• A. Goyal, X. Han, G. Wang, and A. Bates, “Sometimes, you aren’t what you do: Mimicry attacks against provenance graph host intrusion detection systems,” Proc. NDSS Symp., 2023.
• J. Glass-Vanderlan, M. K. Reiter, and A. Bates, “Provenance-based intrusion detection: Opportunities and challenges,” ACM Computing Surveys, vol. 55, no. 1, pp. 1–36, 2023.
• X. Han, T. F. J. Pasquier, A. Bates, J. Mickens, and M. Seltzer, “UNICORN: Runtime provenance-based detector for advanced persistent threats,” Proc. NDSS Symp., 2022.
• C. Kruegel, E. Kirda, D. Mutz, W. Robertson, and G. Vigna, “Automating mimicry attacks using static binary analysis,” Proc. USENIX Security Symp., pp. 161–176, 2005.
• N. K. Niemann and R. G. Blockmon, “Using machine learning to predict the insider threat in a network environment,” M.S. thesis, Naval Postgraduate School, Monterey, CA, USA, 2021.
• B. Bin Sarhan and N. Altwaijry, “Insider threat detection using machine learning approach,” Applied Sciences, vol. 13, no. 1, p. 259, 2022.
• A. Z. Ahmad, R. Abdullah, and M. F. Abdollah, “Transformer-based anomaly detection for host intrusion detection systems,” IEEE Access, vol. 12, pp. 14221–14235, 2024.
• G. Creech and J. Hu, “A semantic approach to host-based intrusion detection systems,” IEEE Trans. Computers, vol. 63, no. 4, pp. 807–819, 2014.
• M. Almseidin, M. Alzubi, S. Kovacs, and M. Alkasassbeh, “Evaluation of machine learning algorithms for intrusion detection system,” Proc. IEEE SISY, pp. 277–282, 2017.
• University of New South Wales, “ADFA Intrusion Detection Datasets,” 2013. [Online]. Available: https://www.unsw.adfa.edu.au
• B. Gregg, BPF Performance Tools: Linux System and Application Observability, 1st ed., Addison-Wesley, 2019.