far-IR images for semantic segmentation

Citation Author(s):
Submitted by:
Greg Walker
Last updated:
Fri, 06/09/2023 - 14:23
Data Format:
0 ratings - Please login to submit your rating.


A long-standing problem in thermal imaging is the inherent assumption of a uniform and known emissivity across an entire image. Semantic segmentation of the materials in a thermal image can identify the pixel-wise emissivity, thus rectifying the spatially uniform emissivity assumption with no human intervention. We have created a multispectral thermal image dataset consisting of nine materials (acrylic, aluminum, bakelite, ceramic, cork, EVA, granite, maple, and silicone) at six different temperatures. Four unique neural networks (U-Net, modified-HybridSN, 3D-2D U-Net, and modified-SMFFNet) were tested to identify spectral-spatial and spatial features to semantically segment the test scenes of the dataset by material. The modified-SMFFNet performed the best with an overall accuracy of 84.5\%, an average accuracy of 82.1\%, and a Cohen's kappa score (x100) of 82.1 when evaluated against the ground truth. The resulting semantic segmentation of materials from the modified-SMFFNet was leveraged to allow unique emissivities for each pixel. Applying the surface radiative property correction decreased the average of all materials' root mean square error (RMSE) between the average surface temperatures and the corresponding thermocouple temperature from 8.2\textsuperscript{o}C to 4.3\textsuperscript{o}C. The range of the RMSE between the materials' average surface temperature and the thermocouple temperature decreased from 9.8\textsuperscript{o}C to 2.8\textsuperscript{o}C. The correction poorly models highly reflective materials in nonuniformly heated surroundings and composite materials with nonuniform surface emissivity.


The image data (value from 0 to 255 for each pixel) is in CSV format.  The images are organized in the RawData subdirectory by temperature in degreees C.  Masks for each image is provided in the Masks subdirectory.  The name of the file specifies the filter used for the scene (FS is full spectrum, LP and SP are longpass and short pass respectively).  The ordinal number at the beginning of the filename indicates the orientation of the blocks.  The colors used in the ground truth images are referenced in the readme.txt.