A new generation of computer vision, namely event-based or neuromorphic vision, provides a new paradigm for capturing visual data and the way such data is processed.
Event-based vision is a state-of-art technology of robot vision. It is particularly promising for use in both mobile robots and drones for visual navigation tasks.
Due to a highly novel type of visual sensors used in event-based vision, only a few datasets aimed at visual navigation tasks are publicly available. Such datasets provide an opportunity to evaluate visual odometry and visual SLAM methods by imitating data readout from real sensors.
This dataset is intended to cover visual navigation tasks for mobile robots navigating in different types of agricultural environment. The dataset might open new opportunities for the evaluation of existing and creation of new event-based visual navigation methods for use in agricultural scenes that contain a lot of vegetation, animals, and patterned objects.
The new dataset was created using our own custom-designed Sensor Bundle, which was installed on a mobile robot platform. During data acquisition sessions, the platform was manually controlled in such environments as forests, plantations, cattle farm, etc.
The Sensor Bundle consists of the dynamic vision sensor, a LIDAR, an RGB-D camera, and environmental sensors (temperature, humidity, and air pressure).
The provided data sequences are accompanied by calibration data. The dynamic visual sensor, the LIDAR, and environmental sensors were time-synchronized with a precision of 1 us and time-aligned with an accuracy of +/- 1 ms. Ground-truth was generated by Lidar-SLAM methods.
In total, there are 21 data sequences in 12 different scenarios for the autumn season. Each data sequence is accompanied by a video demonstrating its content and a detailed description, including known issues.
The reported common issues include relatively small missing fragments of data and the RGB-D sensor's frame number sequence issues.
The new dataset is mostly designed for Visual Odometry tasks, however, it also includes loop-closures for applying event-based visual SLAM methods.
A.Zujevs is supported by the European Regional Development Fund within the Activity 22.214.171.124 “Post-doctoral Research Aid” of the Specific Aid Objective 1.1.1 (No.126.96.36.199/VIAA/2/18/334), while the others are supported by the Latvian Council of Science (lzp-2018/1-0482).
The dataset consists of 21 data sequences recorded in 12 agricultural scenarios in the autumn season. The following
<..sequence_id..>'s are available:
01_forest-- Closed loop, Forest trail, No wind, Daytime
02_forest-- Closed loop, Forest trail, No wind, Daytime
03_green_meadow-- Closed loop, Meadow, grass up to 30 cm, No wind, Daytime
04_green_meadow-- Closed loop, Meadow, grass up to 30 cm, Mild wind, Daytime
05_road_asphalt-- Closed loop, Asphalt road, No wind, Nighttime
06_plantation-- Closed loop, Shrubland, Mild wind, Daytime
07_plantation-- Closed loop, Asphalt road, No wind, Nighttime
08_plantation_water-- Random movement, Sprinklers (water drops on camera lens), No wind, Nighttime
09_cattle_farm-- Closed loop, Cattle farm, Mild wind, Daytime
10_cattle_farm-- Closed loop, Cattle farm, Mild wind, Daytime
11_cattle_farm_feed_table-- Closed loop, Cattle farm feed table, Mild wind, Daytime
12_cattle_farm_feed_table-- Closed loop, Cattle farm feed table, Mild wind, Daytime
13_ditch-- Closed loop, Sandy surface, Edge of ditch or drainage channel, No wind, Daytime
14_ditch-- Closed loop, Sandy surface, Shore or bank, Strong wind, Daytime
15_young_pines-- Closed loop, Sandy surface, Pine coppice, No wind, Daytime
16_winter_cereal_field-- Closed loop, Winter wheat sowing, Mild wind, Daytime
17_winter_cereal_field-- Closed loop, Winter wheat sowing, Mild wind, Daytime
18_winter_rapeseed_field-- Closed loop, Winter rapeseed, Mild wind, Daytime
19_winter_rapeseed_field-- Closed loop, Winter rapeseed, Mild wind, Daytime
20_field_with_a_cow-- Closed loop, Cows tethered in pasture, Mild wind, Daytime
21_field_with_a_cow-- Closed loop, Cows tethered in pasture, Mild wind, Daytime
Each sequence contains the following separately downloadable files:
<..sequence_id..>_data.tar.gz-- entire date sequence in raw data format (
AEDAT2.0- DVS, images - RGB-D, point clouds in pcd files - LIDAR, and IMU
csvfiles with original sensor timestamps). Timestamp conversion formulas are available.
<..sequence_id..>_rosdata.tar.gz-- main sequence in
ROS bagformat. All sensors timestamps are aligned with DVS with an accuracy of less than 1 ms.
<..sequence_id..>_rawcalib_data.tar.gz-- recorded fragments that can be used to perform the calibration independently (intrinsic, extrinsic and time alignment).
<..sequence_id..>_video.mp4-- provides an overview of the sequence data (for the DVS and RGB-D sensors).
The contents of each archive are described below...
Raw format data
<..sequence_id..>_data.tar.gz contains the following files and folders:
+ meta-data/ - all the useful information about the sequence
| + meta-data.md - detailed information about the sequence,
| | sensors, files, and data formats
| + cad_model.pdf - sensors placement
| + <...>_timeconvs.json - timestamp conversion formulas
| + ground-truth/ - movement ground-truth data,
| | calculated using 3 different Lidar-SLAM algorithms
| | (Cartographer, HDL-Graph, LeGo-LOAM)
| + calib-params/ - intrinsic and extrinsic calibration parameters
+ recording/ - main sequence
+ dvs/ - DVS events and IMU data
+ lidar/ - Lidar point clouds and IMU data
+ realsense/ - Realsense camera RGB, Depth frames, and IMU data
+ sensorboard/ - environmental sensors data
(temperature, humidity, air pressure)
ROS Bag format data
Will be published later...
<..sequence_id..>_rawcalib_data.tar.gz archive contains the following files and folders:
+ imu_alignments/ - IMU recordings of the platform lifting/releasing before
| and after the main sequence
| (can be used for custom timestamp alignment)
+ solenoids/ - IMU recordings of the solenoid vibrations before
| and after the main sequence
| (can be used for custom timestamp alignment)
+ lidar_rs/ - Lidar vs Realsense camera extrinsic calibration
| by placing a spherical object (ball) in the field of view of both sensors
+ dvs_rs/ - DVS and Realsense camera intrinsic and extrinsic
calibration frames (checkerboard pattern)
- 01_forest_data.tar.gz (5.36 GB)
- 01_forest_rawcalib_data.tar.gz (1.65 GB)
- 01_forest_video.mp4 (89.61 MB)
- 02_forest_data.tar.gz (5.54 GB)
- 02_forest_rawcalib_data.tar.gz (1.33 GB)
- 02_forest_video.mp4 (96.21 MB)
- 03_green_meadow_data.tar.gz (2.53 GB)
- 03_green_meadow_rawcalib_data.tar.gz (1.50 GB)
- 03_green_meadow_video.mp4 (52.97 MB)
- 04_green_meadow_data.tar.gz (2.58 GB)
- 04_green_meadow_rawcalib_data.tar.gz (1.65 GB)
- 04_green_meadow_video.mp4 (53.43 MB)
- 05_road_asphalt_data.tar.gz (1.26 GB)
- 05_road_asphalt_rawcalib_data.tar.gz (1.01 GB)
- 05_road_asphalt_video.mp4 (8.85 MB)
- 06_plantation_data.tar.gz (3.90 GB)
- 06_plantation_rawcalib_data.tar.gz (1.90 GB)
- 06_plantation_video.mp4 (48.18 MB)
- 07_plantation_data.tar.gz (3.93 GB)
- 07_plantation_rawcalib_data.tar.gz (1.85 GB)
- 07_plantation_video.mp4 (49.24 MB)
- 08_plantation_water_data.tar.gz (7.22 GB)
- 08_plantation_water_rawcalib_data.tar.gz (1.95 GB)
- 08_plantation_water_video.mp4 (85.47 MB)
- 09_cattle_farm_data.tar.gz (2.81 GB)
- 09_cattle_farm_rawcalib_data.tar.gz (1,007.75 MB)
- 09_cattle_farm_video.mp4 (36.01 MB)
- 10_cattle_farm_data.tar.gz (2.74 GB)
- 10_cattle_farm_rawcalib_data.tar.gz (966.74 MB)
- 10_cattle_farm_video.mp4 (36.97 MB)
- 11_cattle_farm_feed_table_data.tar.gz (2.10 GB)
- 11_cattle_farm_feed_table_rawcalib_data.tar.gz (855.12 MB)
- 11_cattle_farm_feed_table_video.mp4 (19.74 MB)
- 12_cattle_farm_feed_table_data.tar.gz (2.00 GB)
- 12_cattle_farm_feed_table_rawcalib_data.tar.gz (919.29 MB)
- 12_cattle_farm_feed_table_video.mp4 (19.61 MB)
- 13_ditch_data.tar.gz (1.38 GB)
- 13_ditch_rawcalib_data.tar.gz (1.20 GB)
- 13_ditch_video.mp4 (19.18 MB)
- 14_ditch_data.tar.gz (3.53 GB)
- 14_ditch_rawcalib_data.tar.gz (1,005.13 MB)
- 14_ditch_video.mp4 (80.89 MB)
- 15_young_pines_data.tar.gz (2.56 GB)
- 15_young_pines_rawcalib_data.tar.gz (982.49 MB)
- 15_young_pines_video.mp4 (38.60 MB)
- 16_winter_cereal_field_data.tar.gz (2.05 GB)
- 16_winter_cereal_field_rawcalib_data.tar.gz (916.56 MB)
- 17_winter_cereal_field_data.tar.gz (1.84 GB)
- 17_winter_cereal_field_rawcalib_data.tar.gz (883.68 MB)
- 17_winter_cereal_field_video.mp4 (32.77 MB)
- 18_winter_rapeseed_field_data.tar.gz (2.15 GB)
- 18_winter_rapeseed_field_rawcalib_data.tar.gz (789.78 MB)
- 18_winter_rape_field_video.mp4 (52.88 MB)
- 19_winter_rapeseed_field_data.tar.gz (1.90 GB)
- 19_winter_rapeseed_field_rawcalib_data.tar.gz (714.77 MB)
- 19_winter_rape_field_video.mp4 (49.73 MB)
- 20_field_with_a_cow_data.tar.gz (3.57 GB)
- 20_field_with_a_cow_rawcalib_data.tar.gz (943.47 MB)
- 20_field_with_a_cow_video.mp4 (90.37 MB)
- 21_field_with_a_cow_data.tar.gz (3.97 GB)
- 21_field_with_a_cow_rawcalib_data.tar.gz (1.00 GB)
- 21_field_with_a_cow_video.mp4 (120.80 MB)