Holoscopic 3D Micro-Gesture Database

Citation Author(s):
Yi
Liu
Brunel University London
Mohammad R.
Swash
Brunel University London
Hongying
Meng
Brunel University London
Submitted by:
Hongying Meng
Last updated:
Tue, 01/12/2021 - 12:48
DOI:
10.21227/5byd-gx52
Data Format:
Links:
License:
0
0 ratings - Please login to submit your rating.

Abstract 

Holoscopic micro-gesture recognition (HoMG) database was recorded using a holoscopic 3D camera, which have 3 conventional gestures from 40 participants under different settings and conditions. The principle of holoscopic 3D (H3D) imaging mimics fly’s eye technique that captures a true 3D optical model of the scene using a microlens array. For the purpose of H3D micro-gesture recognition. HoMG database has two subsets. The video subset has 960 videos and the image subset has 30635 images, while both have three type of microgestures (classes). Each subset has been divided into three partitions: training set, development set and testing set where there is not overlap between them in term of the subjects. The database has been used for Holoscopic Micro-Gesture Recognition Challenge 2018 (HoMGR 2018) that was held at IEEE Face & Gesture 2018 (FG2018) - Xi'an, China, 15-19th May 2018 (https://fg2018.cse.sc.edu/Challenges.html). The database is now publicly available for wider research communities in the research areas of holoscopic 3D image processing, machine learning for gesture recognition and its application in AR and VR.

Instructions: 

Holoscopic micro-gesture recognition (HoMG) database consists of 3 hand gestures: Button, Dial and Slider from 40 subjects with various ages and settings, which includes the right and left hand, two of record distance.

For video subset: There are 40 subjects, and each subject has 24 videos due to the different setting and three gestures. For each video, the frame rate is 25 frames per second and length of videos are from few seconds to 20 seconds and not equally. The whole dataset was divided into 3 parts. 20 subjects for the training set, 10 subjects for development set and another 10 subjects for testing set.

For image subset: Video can capture the motion information of the micro-gesture and it is a good way for micro-gesture recognition. From each video recording, the different number of frames were selected as the still micro-gesture images. The image resolution 1920 by 1080. In total, there are 30635 images selected. The whole dataset was split into three partitions: A Training, Development, and Testing partition. There are 15237 images in the training subsets of 20 participants with 8364 in close distance and 6853 in the far distance. There are 6956 images in the development subsets of 10 participants with 3077 in close distance and 3879 in far distance. There are 8442 images in the testing subsets of 10 participants with 3930 in close distance and 4512 in far distance.