Thermal Optical Annotated Multi Class Image Dataset

We manually annotate all images using the VGG Image Annotator (VIA) [Dutta, Abhishek, Ankush Gupta, and Andrew Zissermann. "VGG image annotator (VIA)." URL: http://www.robots.ox.ac.uk/~vgg/software/via (2016).] for the creation of the box.

We use the standard annotation format provided. 

'sonel_annotation.csv' uses the image present in the folder named 'sonel'.

Similarly, the files 'flir_annotation.csv' and 'flir_old_annotation.csv' are based on the images present in the fodlers 'flir' and 'flir_old'

The images can be found as a part of our older work which is presented as an open database [Suranjan Goswami, Nand Kumar Yadav, Satish Kumar Singh. "Thermal Visual Paired Dataset." doi: 10.21227/jjba-6220]

The data is classified into 5 different classes

Class:Abbreviation:Key 

modern infrastructure: inf:5

crowd: cro:4

human:hum:3

animal:ani:2

nature:nat:1

In each file, which is presented as an excel file, the data columns are as follows:

filename, file size, file attribute, region count, region id, region shape attributes and region attributes.

region count shows the number of regions present in each image, region attribute presents the details of the rectangle which contains the said attribute and the region attributes presents the attribute name.

These can be directly input into VIA after loading the corresponding database images to see the outlined annotations.

Since the annotation presented by VIA might not be easily usable by all data readers, we have modified the same to be easily processed as the numbers files

These are 'sonel_annotation-numbers.csv', 'flir_annotation-numbers.csv' and 'flir_old_annotation-numbers.csv' .

Here, the class abbreviations are replaced by their corresponding number key as provided above.

Please note that the database we have used contains both registered and unregistered images as a part of the database. 

All registered thermal images that have been annotated only, not the unregistered ones as our work required registered thermal images.

This is a one way registration: that is, the annotation done on the thermal images should reflect on the optical images. 

We have not included the optical annotation method here, wherein we use DETR to annotate the registered optical images and use the corresponding mapping to create the 2 way annotation.

We also include 3 ZIP files with the images and their corresponding annotations both manually and done with DETR.

All annotations are labelled as NAME, X_START coordinate, Y_START coordinate, WIDTH, HEIGHT, CLASS for the individual manual annotations.

FOr the DETR annotations, they correspond to NAME, X_START coordinate, Y_START coordinate, X_END coordinate, Y_END coordinate, CLASS.

This database is presented as a part of our work "Novel Deep Learning Method for Thermal to Annotated Thermal-Optical Fused Images"