Radar-based hand, arm, and body gestures

Citation Author(s):
Arthur
Sluÿters
Université catholique de Louvain
Sébastien
Lambot
Université catholique de Louvain
Jean
Vanderdonckt
Université catholique de Louvain
Submitted by:
Arthur Sluyters
Last updated:
Mon, 04/08/2024 - 08:53
DOI:
10.21227/5n08-v130
License:
389 Views
Categories:
Keywords:
0
0 ratings - Please login to submit your rating.

Abstract 

For gesture recognition, radar sensors provide a unique alternative to other input devices, such as cameras or motion sensors. They combine a low sensitivity to lighting conditions, an ability to see through surfaces, and user privacy preservation, with a small form factor and low power usage. However, radar signals can be noisy, complex to analyze, and do not transpose from one radar to another.

To overcome these limitations, we introduce an electromagnetic model and inversion-based approach for radar signal processing that satisfies three properties: radar system invariance (the output signal is normalized to become independent from the radar), background scene invariance (the output signal is no longer affected by the background scene), and one-shot calibration (the radar needs to be calibrated only once).

Two experiments are conducted to evaluate the efficacy of our approach on a set of 20 hand gesture classes and to identify differentiable subsets of gestures, reaching up to 90.58%, 97.78%, and 98.54% accuracy on the full gesture set,  on a subset of 12 and 8 gesture classes in a user-dependent scenario, respectively. Our approach also shows promising results in a mixed scenario, for customized gestures.

This dataset contains the raw radar signal of the 20 gestures, as well as the signal at two stages of our signal processing dataflow. It also contains the results of our two experiments.

Instructions: 

Introduction

 The dataset is composed of raw and preprocessed gesture recordings and the results of an evaluation.

 

IEEE2023 Dataset (raw, users 1-3) and IEEE2023 Dataset (raw, users 4-6)

Directory structure

  • USR_ID_1

    • antennapairs.out: list of antenna pairs used for the recording
    • WalabotSignal_User_USER_ID_1_Gesture_GESTURE_ID_1-SAMPLE_ID_1.out
    • ...
    • WalabotSignal_User_USER_ID_6_Gesture_GESTURE_ID_20-SAMPLE_ID_10.out
  • ...
  • USER_ID_6

Out files structure

Time domain data. 1024 fast-time samples per frame, 12 pairs of antennas, variable number of frames per gesture. Gestures ar recorded at about 40 frames per second.

Each file consists of a set of lines, where each lines has 2 columns. The first column is the fast time and the scond column is the amplitude of the signal measured at the time.

Lines are ordered as follows:

 

  • FRAME_1_ANTENNA_PAIR_1_FT_1
  • ...
  • FRAME_1_ANTENNA_PAIR_1_FT_1024
  • ...
  • ...
  • FRAME_1_ANTENNA_PAIR_12_FT_1
  • ...
  • FRAME_1_ANTENNA_PAIR_12_FT_1024
  • ...
  • ...
  • FRAME_N_ANTENNA_PAIR_1_FT_1
  • ...
  • FRAME_N_ANTENNA_PAIR_1_FT_1024
  • ...
  • ...
  • FRAME_N_ANTENNA_PAIR_12_FT_1
  • ...
  • FRAME_N_ANTENNA_PAIR_12_FT_1024

 

IEEEE2023 Dataset (time gating, users 1-6)

Directory structure

  • GESTURE_ID_1

    • USER_ID_1
      • GESTURE_ID_1-SAMPLE_ID_1.json
      • ...
      • GESTURE_ID_1-SAMPLE_ID_10.json
    • ...
    • USER_ID_6
  • ...
  • GESTURE_ID_20

Json files structure

  • "name": "GESTURE_ID",
  • "subject": "USER_ID",
  • "paths": a list of path elements, one per pair of antennas
    • "label": "ANTENNA_PAIR_NAME"
    • "strokes": a list containing one stroke element
      • "id": STROKE_ID
      • "points": a list of point elements, one per frame
        • "coordinates": frequency-domain signal, as a vector. The first half of the vector represents the real part of the complex signal. The second half represents the imaginary values of the complex signal.
        • "t": "FRAME_ID"

 

IEEEE2023 Dataset (filtering, users 1-6)

Directory structure

  • GESTURE_ID_1

    • USER_ID_1
      • GESTURE_ID_1-SAMPLE_ID_1.json
      • ...
      • GESTURE_ID_1-SAMPLE_ID_10.json
    • ...
    • USER_ID_6
  • ...
  • GESTURE_ID_20

Json files structure

  • "name": "GESTURE_ID",
  • "subject": "USER_ID",
  • "paths": a list of path elements, one per pair of antennas
    • "label": "ANTENNA_PAIR_NAME"
    • "strokes": a list containing one stroke element
      • "id": STROKE_ID
      • "points": a list of point elements, one per frame
        • "x": distance [m]
        • "y": relative permittivity
        • "t": "FRAME_ID"
Funding Agency: 
Fonds de la Recherche Scientifique - FNRS
Grant Number: 
40001931, 40011629