Datasets
Open Access
MQTT-IoT-IDS2020: MQTT Internet of Things Intrusion Detection Dataset
- Citation Author(s):
-
HananHindy
Abertay UniversityChristosTachtatzisUniversity of StrathclydeRobertAtkinsonUniversity of StrathclydeEthanBayneAbertay UniversityXavierBellekensUniversity of Strathclyde - Submitted by:
- Hanan Hindy
- Last updated:
- Mon, 08/31/2020 - 12:43
- DOI:
- 10.21227/bhxy-ep04
- Data Format:
- License:
22455 Views- Categories:
- Keywords:
Abstract
Message Queuing Telemetry Transport (MQTT) protocol is one of the most used standards used in Internet of Things (IoT) machine to machine communication. The increase in the number of available IoT devices and used protocols reinforce the need for new and robust Intrusion Detection Systems (IDS). However, building IoT IDS requires the availability of datasets to process, train and evaluate these models. The dataset presented in this paper is the first to simulate an MQTT-based network. The dataset is generated using a simulated MQTT network architecture. The network comprises twelve sensors, a broker, a simulated camera, and an attacker. Five scenarios are recorded: (1) normal operation, (2) aggressive scan, (3) UDP scan, (4) Sparta SSH brute-force, and (5) MQTT brute-force attack. The raw pcap files are saved, then features are extracted. Three abstraction levels of features are extracted from the raw pcap files: (a) packet features, (b) Unidirectional flow features and (c) Bidirectional flow features. The csv feature files in the dataset are suited for Machine Learning (ML) usage. Also, the raw pcap files are suitable for the deeper analysis of MQTT IoT networks communication and the associated attacks.
The dataset consists of 5 pcap files, namely, normal.pcap, sparta.pcap, scan_A.pcap, mqtt_bruteforce.pcap and scan_sU.pcap. Each file represents a recording of one scenario; normal operation, Sparta SSH brute-force, aggressive scan, MQTT brute-force and UDP scan respectively. The attack pcap files contain background normal operations. The attacker IP address is “192.168.2.5”. Basic packet features are extracted from the pcap files into CSV files with the same pcap file names. The features include flags, length, MQTT message parameters, etc. Later, unidirectional and bidirectional features are extracted. It is important to note that for the bidirectional flows, some features (pointed as *) have two values—one for forward flow and one for the backward flow. The two features are recorded and distinguished by a prefix “fwd_” for forward and “bwd_” for backward.
Dataset Files
pcap_files.zip (1.35 GB)- packet_features.zip (261.27 MB)
- uniflow_features.zip (16.08 MB)
- biflow_features.zip (14.56 MB)
Open Access dataset files are accessible to all logged in users. Don't have a login? Create a free IEEE account. IEEE Membership is not required.
Comments
really very interesting, thank very much
really useful ，thank you very much
Great work!
Thank you very much for sharing.
how i can download csv file of MQTT dataset?
Please find the packet_features.zip on the same page, it contains CSV files.
how i can download csv file of MQTT dataset?
Thank you for your interest in the dataset. The features can be found in the packet_features.zip (in the dataset files section). Also, you can find more details of the experiments in our published paper "https://link.springer.com/chapter/10.1007/978-3-030-64758-2_6" and the code is on GitHub "https://github.com/AbertayMachineLearningGroup/MQTT_ML"
Hello!
I'm working on Anomaly Detection on my Master's Thesis, but with an emphasis on physical signals (more Fault Detection, less Intrusion Detection), which would involve, in this case, the signals from the sensors themselves. I would, therefore, ignore all data regarding packet metadata (source, origin, message size, etc.). Would you consider that this dataset has enough information on what the sensors are picking?
Thank you!
Hello Carlos,
Thank you for your interest in the dataset. Unfortunately, the MQTT-IoT-IDS2020's focuses more on the data transferred between the sensors (specifically in protocol-based attacks vs generic attacks), not the sensors data/messages themselves. Sensor messages are randomly generated.
You can find more details of the experiments in our published paper "https://link.springer.com/chapter/10.1007/978-3-030-64758-2_6" and the code is on GitHub "https://github.com/AbertayMachineLearningGroup/MQTT_ML".
Finally, I would suggest that you reuse the simulated network architecture to generate data for your experiments.
I hope this helps!
Hello
Thanks for this data set! It is definitely needed. For bidirectional flow generation, did you use a specific tool to identify TCP/UDP flows and calculate the features or did you manually convert the raw packet data?
Hello James,
Thank you for your interest in the dataset. I manually extracted the features from the raw packets using python, specifically dpkt package.
The code is available on GitHub.
I hope this helps!
Salut super job *smiley pouce*
Thank you San!
Hello
I want to see what the difference is between three and three datasets.
I want to know on what basis you categorized the features within these three datasets.
Thank you for your answer
Thank you for your interest in the dataset. The features are grouped based on 3 categories (packet features, unidirectional flow features and bidirectional flow features).
You can read more details in our paper: Machine Learning Based IoT Intrusion Detection System: An MQTT Case Study (MQTT-IoT-IDS2020 Dataset)
https://link.springer.com/chapter/10.1007/978-3-030-64758-2_6
.
Thank you so much