Datasets
Standard Dataset
Cloud Telescope Internet Background Radiation - October 2023 - February 2024
- Citation Author(s):
- Submitted by:
- Fabricio Bortoluzzi
- Last updated:
- Mon, 09/30/2024 - 12:23
- DOI:
- 10.21227/zkyy-gk56
- Data Format:
- License:
- Categories:
- Keywords:
Abstract
This dataset results from a 5-month-long Cloud Telescope Internet Background Radiation collection experiment conducted during the months of October 2023 until February 2024.
A total amount of 130 EC2 instances (sensors) were deployed across all the 26 commercially available AWS regions at the time, 5 sensors per region.
A Cloud Telescope sensor does not serve information. All traffic arriving to the sensor is unsolicited, and potentially malicious. Sensors were configured to allow all unsolicited traffic.
This dataset contains 530,914,922 network events recorded in the PCAP format. 425,674,635 (80%) of the datagrams fit the TCP transport-layer protocol, while 86,139,655 (16%) belong to ICMP messages, and 18,982,971 (4%) are UDP datagrams.
The architecture is reproducible. Terraform Infrastructure-As-Code is available at: https://github.com/lucasbeiler/ibr-iac
We keep a set of basic bash and python scripts that could help you to get started with your own ETL (extract-transform-load) workflow. This is available at: https://github.com/bortoluzzi/ibr-etl
We will attach the paper with a full analysis of this dataset once it gets published and indexed on IEEE Xplore.
The files in this dataset are too large to be handled by Wireshark.
Command-line dataset exploration can be done with the use of Tshark. Tshark can be installed on Ubuntu-like systems with "sudo apt-get install tshark". On Windows, Tshark is installed with Wireshark.
The command to read an uncompressed dataset file is "tshark -r af-south-1.pcap" assuming you want to browse south africa's sub-dataset named "af-south-1.pcap".
Reading straight from the compressed is also possible with: "tshark -r af-south-1.pcap.gz".
We have launched a video explaining how to get started with Internet Background Radiation using the first Cloud Telescope dataset. Please watch https://bit.ly/ibr_workshop
Dataset Files
- e3_af-south-1.pcap.gz (565.02 MB)
- e3_ap-east-1.pcap.gz (670.61 MB)
- e3_ap-northeast-1.pcap.gz (655.36 MB)
- e3_ap-northeast-2_.pcap.gz (729.04 MB)
- e3_ap-northeast-3.pcap.gz (536.22 MB)
- e3_ap-south-1.pcap.gz (604.74 MB)
- e3_ap-south-2.pcap.gz (497.55 MB)
- e3_ap-southeast-1.pcap.gz (1,002.64 MB)
- e3_ap-southeast-2.pcap.gz (580.56 MB)
- e3_ap-southeast-3.pcap.gz (1.30 GB)
- e3_ca-central-1.pcap.gz (550.95 MB)
- e3_eu-central-1.pcap.gz (604.28 MB)
- e3_eu-central-2.pcap.gz (602.04 MB)
- e3_eu-north-1.pcap.gz (636.46 MB)
- e3_eu-south-1.pcap.gz (448.08 MB)
- e3_eu-south-2.pcap.gz (571.73 MB)
- e3_eu-west-1.pcap.gz (574.31 MB)
- e3_eu-west-2.pcap.gz (626.35 MB)
- e3_eu-west-3.pcap.gz (700.51 MB)
- e3_me-central-1.pcap.gz (480.73 MB)
- e3_me-south-1.pcap.gz (1.61 GB)
- e3_sa-east-1.pcap.gz (447.81 MB)
- e3_us-east-1.pcap.gz (592.48 MB)
- e3_us-east-2.pcap.gz (590.10 MB)
- e3_us-west-1.pcap.gz (1.13 GB)
- e3_us-west-2.pcap.gz (611.19 MB)