Agri-EBV-winter-summer

Citation Author(s):
Andrejs
Zujevs
Riga Technical University, Latvia
Mihails
Pudzs
Riga Technical University, Latvia
Vitalijs
Osadcuks
Latvia University of Life Sciences and Technologies, Latvia
Arturs
Ardavs
Riga Technical University, Latvia
Maris
Galauskis
Riga Technical University, Latvia
Janis
Grundspenkis
Riga Technical University, Latvia
Submitted by:
Andrejs Zujevs
Last updated:
Fri, 06/21/2024 - 13:34
DOI:
10.21227/64s9-1828
Data Format:
tar.gz
mp4
License:
Creative Commons Attribution
1092 Views
Categories:
Agriculture
Artificial Intelligence
Sensors
Computer Vision
Keywords:
Event-based vision, visual navigation, agriculture
0
0 ratings - Please login to submit your rating.
CITE SHARE/EMBED

Abstract 

A new generation of computer vision, namely event-based or neuromorphic vision, provides a new paradigm for capturing visual data and the way such data is processed. Event-based vision is a state-of-art technology of robot vision. It is particularly promising for use in both mobile robots and drones for visual navigation tasks. Due to a highly novel type of visual sensors used in event-based vision, only a few datasets aimed at visual navigation tasks are publicly available. Such datasets provide an opportunity to evaluate visual odometry and visual SLAM methods by imitating data readout from real sensors. This dataset is intended to cover visual navigation tasks for mobile robots navigating in different types of agricultural environment. The dataset might open new opportunities for the evaluation of existing and creation of new event-based visual navigation methods for use in agricultural scenes that contain a lot of vegetation, animals, and patterned objects. The new dataset was created using our own custom-designed Sensor Bundle, which was installed on a mobile robot platform. During data acquisition sessions, the platform was manually controlled in such environments as forests, plantations, farms, etc. The Sensor Bundle consists of the dynamic vision sensor, a LIDAR, an RGB-D camera, and environmental sensors (temperature, humidity, and air pressure). The provided data sequences are accompanied by calibration data. The dynamic visual sensor, the LIDAR, and environmental sensors were time-synchronized with a precision of 1 us and time-aligned with an accuracy of +/- 1 ms. Ground-truth was generated by Lidar-SLAM methods. In total, there are 11 data sequences in 6 different scenarios for the winter season and 31 data sequences in 14 different scenarios for the spring/summer season. Each data sequence is accompanied by a video demonstrating its content and a detailed description, including known issues. The reported common issues include relatively small missing fragments of data and the RGB-D sensor's frame number sequence issues. The new dataset is mostly designed for Visual Odometry tasks, however, it also includes loop-closures for applying event-based visual SLAM methods. A.Zujevs is supported by the European Regional Development Fund within the Activity 1.1.1.2 “Post-doctoral Research Aid” of the Specific Aid Objective 1.1.1 (No.1.1.1.2/VIAA/2/18/334), while the others are supported by the Latvian Council of Science (lzp-2018/1-0482).

BibTex citation: @inproceedings{Zujevs2021, author = {Zujevs, Andrejs and Pudzs, Mihails and Osadcuks, Vitalijs and Ardavs, Arturs and Galauskis, Maris and Grundspenkis, Janis}, booktitle = {IEEE International Conference on Robotics and Automation}, title = {{A Neuromorphic Vision Dataset for Visual Navigation Tasks in Agriculture}}, month = {June}, year = {2021}}

Instructions: 

The dataset includes the following sequences:

  • 01_winter_forest – Daytime, No wind, Clear weather, Snowy scenery, Closed loop, Forest trail
  • 02_winter_forest - Daytime, No wind, Clear weather, Snowy scenery, Closed loop, Forest trail
  • 03_winter_parking_lot - Daytime, No wind, Clear weather, Snowy scenery, Closed loop, Asphalt road
  • 04_winter_bush_rows - Daytime, No wind, Snowy scenery, Closed loop, Shrubland
  • 05_winter_bush_rows - Daytime, No wind, Snowy scenery, Closed loop, Shrubland
  • 06_winter_greenhouse_complex - Daytime, No wind, Snowy scenery, Closed loop, Cattle farm feed table
  • 07_winter_greenhouse_complex - Daytime, No wind, Snowy scenery, Closed loop, Cattle farm feed table
  • 08_winter_orchard - Daytime, No wind, Snowy scenery, Closed loop, Orchard
  • 09_winter_orchard - Daytime, No wind, Snowy scenery, Closed loop, Orchard
  • 10_winter_farm - Daytime, No wind, Snowy scenery, Closed loop, Cattle farm feed table
  • 11_winter_farm - Daytime, No wind, Snowy scenery, Closed loop, Cattle farm feed table
  • 12_summer_bush_rows - Daytime, Mild wind, Closed loop, Shrubland
  • 13_summer_bush_rows - Daytime, Mild wind, Closed loop, Shrubland
  • 14_summer_farm - Daytime, Mild wind, Closed loop, Shrubland, Tilled field
  • 15_summer_farm - Daytime, Mild wind, Closed loop, Shrubland, Tilled field
  • 16_summer_orchard - Daytime, Mild wind, Closed loop, Shrubland, Orchard
  • 17_summer_orchard - Daytime, Mild wind, Closed loop, Shrubland, Orchard
  • 18_summer_garden - Daytime, Mild wind, Closed loop, Pine coppice, Winter wheat sowing, Winter rapeseed
  • 19_summer_garden - Daytime, Mild wind, Closed loop, Pine coppice, Winter wheat sowing, Winter rapeseed
  • 20_summer_farm - Daytime, Mild wind, Closed loop, Orchard, Tilled field, Cows tethered in pasture
  • 21_summer_farm - Daytime, Mild wind, Closed loop, Orchard, Tilled field, Cows tethered in pasture
  • 22_summer_hangar - Daytime, No wind, Closed loop
  • 23_summer_hangar - Daytime, No wind, Closed loop
  • 24_summer_hangar - Daytime, No wind, Closed loop
  • 25_summer_puddles - Daytime, No wind, Closed loop, Meadow, grass up to 30 cm
  • 26_summer_green_meadow - Daytime, No wind, Closed loop, Meadow, grass up to 30 cm
  • 27_summer_green_meadow - Daytime, No wind, Closed loop, Meadow, grass up to 30 cm
  • 28_summer_grooved_field - Daytime, No wind, Closed loop, Meadow, grass up to 100 cm, Furrows (longitudinally and transversely)
  • 29_summer_grooved_field - Daytime, No wind, Closed loop, Meadow, grass up to 100 cm, Furrows (longitudinally and transversely)
  • 30_summer_grooved_field - Daytime, No wind, Closed loop, Furrows (longitudinally and transversely)
  • 31_summer_grooved_field - Daytime, No wind, Closed loop, Furrows (longitudinally and transversely)
  • 32_summer_cereal_field - Daytime, No wind, Closed loop, Meadow, grass up to 100 cm
  • 33_summer_cereal_field - Daytime, No wind, Closed loop, Meadow, grass up to 100 cm
  • 34_summer_forest - Daytime, No wind, Closed loop, Forest trail
  • 35_summer_forest - Daytime, No wind, Closed loop, Forest trail
  • 36_summer_forest - Daytime, No wind, Closed loop, Forest trail, Forest surface - moss, branches, stumps
  • 37_summer_forest - Daytime, No wind, Closed loop, Forest trail, Forest surface - moss, branches, stumps
  • 38_summer_dark_parking_lot - Twilight, No wind, Closed loop, Asphalt road, Lawn
  • 39_summer_dark_parking_lot - Twilight, No wind, Closed loop, Asphalt road, Lawn
  • 40_summer_parking_lot - Daytime, Mild wind, Closed loop, Asphalt road, Lawn
  • 41_summer_greenhouse - Daytime, Closed loop, Greenhouse
  • 42_summer_greenhouse - Daytime, Closed loop, Greenhouse

Each sequence contains the following separately downloadable files:

  • <..sequence_id..>_video.mp4 – provides an overview of the sequence data (for the DVS and RGB-D sensors).
  • <..sequence_id..>_data.tar.gz – entire date sequence in raw data format (AEDAT2.0 - DVS, images - RGB-D, point clouds in pcd files - LIDAR, and IMU csv files with original sensor timestamps). Timestamp conversion formulas are available.
  • <..sequence_id..>_rawcalib_data.tar.gz – recorded fragments that can be used to perform the calibration independently (intrinsic, extrinsic and time alignment).
  • <..sequence_id..>_rosbags.tar.gz – main sequence in ROS bag format. All sensors timestamps are aligned with DVS with an accuracy of less than 1 ms.

The contents of each archive are described below..

Raw format data

The archive <..sequence_id..>_data.tar.gz contains the following files and folders:

  • ./meta-data/ - all the useful information about the sequence
  • ./meta-data/meta-data.md - detailed information about the sequence, sensors, files, and data formats
  • ./meta-data/cad_model.pdf - sensors placement
  • ./meta-data/<...>_timeconvs.json - coefficients for timestamp conversion formulas
  • ./meta-data/ground-truth/ - movement ground-truth data, calculated using 3 different Lidar-SLAM algorithms (Cartographer, HDL-Graph, LeGo-LOAM)
  • ./meta-data/calib-params/ - intrinsic and extrinsic calibration parameters
  • ./recording/ - main sequence
  • ./recording/dvs/ - DVS events and IMU data
  • ./recording/lidar/ - Lidar point clouds and IMU data
  • ./recording/realsense/ - Realsense camera RGB, Depth frames, and IMU data
  • ./recording/sensorboard/ - environmental sensors data (temperature, humidity, air pressure)

Calibration data

The <..sequence_id..>_rawcalib_data.tar.gz archive contains the following files and folders:

  • ./imu_alignments/ - IMU recordings of the platform lifting before and after the main sequence (can be used for custom timestamp alignment)
  • ./solenoids/ - IMU recordings of the solenoid vibrations before and after the main sequence (can be used for custom timestamp alignment)
  • ./lidar_rs/ - Lidar vs Realsense camera extrinsic calibration by showing both sensors a spherical object (ball)
  • ./dvs_rs/ - DVS and Realsense camera intrinsic and extrinsic calibration frames (checkerboard pattern)

ROS Bag format data

There are six rosbag files for each scene, their contents are as follows:

  • <..sequence_id..>_dvs.bag (topics: /dvs/camera_info, /dvs/events, /dvs/imu, and accordingly message types: sensor_msgs/CameraInfo, dvs_msgs/EventArray, sensor_msgs/Imu).
  • <..sequence_id..>_lidar.bag (topics: /lidar/imu/acc, /lidar/imu/gyro, /lidar/pointcloud, and accordingly message types: sensor_msgs/Imu, sensor_msgs/Imu, sensor_msgs/PointCloud2).
  • <..sequence_id..>_realsense.bag (topics: /realsense/camera_info, /realsense/depth, /realsense/imu/acc, /realsense/imu/gyro, /realsense/rgb, /tf, and accordingly message types: sensor_msgs/CameraInfo, sensor_msgs/Image, sensor_msgs/Imu, sensor_msgs/Imu, sensor_msgs/Image, tf2_msgs/TFMessage).
  • <..sequence_id..>_sensorboard.bag (topics: /sensorboard/air_pressure, /sensorboard/relative_humidity, /sensorboard/temperature, and accordingly message types: sensor_msgs/FluidPressure, sensor_msgs/RelativeHumidity, sensor_msgs/Temperature).
  • <..sequence_id..>_trajectories.bag (topics: /cartographer, /hdl, /lego_loam, and accordingly message types: geometry_msgs/PoseStamped, geometry_msgs/PoseStamped, geometry_msgs/PoseStamped).
  • <..sequence_id..>_data_for_realsense_lidar_calibration.bag (topics: /lidar/pointcloud, /realsense/camera_info, /realsense/depth, /realsense/rgb, /tf, and accordingly message types: sensor_msgs/PointCloud2, sensor_msgs/CameraInfo, sensor_msgs/Image, sensor_msgs/Image, tf2_msgs/TFMessage).

Comments

There was an issue with the upload of 13_summer_bush_rows_rawcalib_data_.tar.gz, and it had to be renamed (underscore symbol added at the end of the file name).

Submitted by Andrejs Zujevs on Tue, 09/07/2021 - 11:32

Dataset Files

LOGIN TO ACCESS DATASET FILES
Open Access dataset files are accessible to all logged in  users. Don't have a login?  Create a free IEEE account.  IEEE Membership is not required.

QUESTIONS?

Login to Send Author a Private Message

Report a problem with this Dataset