Institutional Repository [SANDBOX]
Technical University of Crete
EN  |  EL

Search

Browse

My Space

Distributed real-time network intrusion detection system on apache spark

Kalosynakis Minas-Diomfeas

Full record


URI: http://purl.tuc.gr/dl/dias/E8666D35-46BE-4E3D-8B7C-BE8D7D1CD6F8
Year 2021
Type of Item Diploma Work
License
Details
Bibliographic Citation Minas-Diomfeas Kalosynakis, "Distributed real-time network intrusion detection system on apache spark", Diploma Work, School of Electrical and Computer Engineering, Technical University of Crete, Chania, Greece, 2021 https://doi.org/10.26233/heallink.tuc.90442
Appears in Collections

Summary

In recent years, the rapid increase of internet based services raises signif-icant information security concerns. A large amount of network trafficdata is generated on a daily basis with high speed while security threatsbecome increasingly more complex. Fast and efficient detection of in-trusive activities in such conditions is a challenging task. In order toaddress this issue, we propose a distributed intrusion detection systemthat utilizes machine learning classifiers to identify malicious networkactivity in real-time. Specifically, we use the Chi-Squared algorithm toselect important features, based on which we build Decision Tree, Ran-dom Forest, and Extreme Gradient Boosting classification models onApache Spark Big Data platform. The proposed system supports scala-bility in all of its different layers and provides a user-friendly graphicalinterface to visualize network activity. Experimental results againstthe NSL-KDD dataset demonstrate that the system can perform bi-nary classification with an area under ROC curve of 97% using theRandom Forest machine learning model.

Available Files

Services

Statistics