The name format of the video files are as follows: “sequenceType_sequenceNumber_challengeSourceType_challengeType_challengeLevel.mp4”

·         sequenceType: 01 – Real data 02 – Unreal data

·         sequenceNumber: A number in between [01 – 49]

·         challengeSourceType: 00 – No challenge source (which means no challenge) 01 – After affect

·         challengeType: 00 – No challenge 01 – Decolorization 02 – Lens blur 03 – Codec error 04 – Darkening 05 – Dirty lens 06 – Exposure 07 – Gaussian blur 08 – Noise 09 – Rain 10 – Shadow 11 – Snow 12 – Haze

·         challengeLevel: A number in between [01-05] where 01 is the least severe and 05 is the most severe challenge.

Test Sequences

We split the video sequences into 70% training set and 30% test set. The sequence numbers corresponding to test set are given below:

[01_04_x_x_x, 01_05_x_x_x, 01_06_x_x_x, 01_07_x_x_x, 01_08_x_x_x, 01_18_x_x_x, 01_19_x_x_x, 01_21_x_x_x, 01_24_x_x_x, 01_26_x_x_x, 01_31_x_x_x, 01_38_x_x_x, 01_39_x_x_x, 01_41_x_x_x, 01_47_x_x_x, 02_02_x_x_x, 02_04_x_x_x, 02_06_x_x_x, 02_09_x_x_x, 02_12_x_x_x, 02_13_x_x_x, 02_16_x_x_x, 02_17_x_x_x, 02_18_x_x_x, 02_20_x_x_x, 02_22_x_x_x, 02_28_x_x_x, 02_31_x_x_x, 02_32_x_x_x, 02_36_x_x_x]

The videos with all other sequence numbers are in the training set. Note that “x” above refers to the variations listed earlier.

The name format of the annotation files are as follows: “sequenceType_sequenceNumber.txt“

Challenge source type, challenge type, and challenge level do not affect the annotations. Therefore, the video sequences that start with the same sequence type and the sequence number have the same annotations.

·         sequenceType: 01 – Real data 02 – Unreal data

·         sequenceNumber: A number in between [01 – 49]

The format of each line in the annotation file (txt) should be: “frameNumber_signType_llx_lly_lrx_lry_ulx_uly_urx_ury”. You can see a visual coordinate system example in our GitHub page.

·         frameNumber: A number in between [001-300]

·         signType: 01 – speed_limit 02 – goods_vehicles 03 – no_overtaking 04 – no_stopping 05 – no_parking 06 – stop 07 – bicycle 08 – hump 09 – no_left 10 – no_right 11 – priority_to 12 – no_entry 13 – yield 14 – parking

The name format of the provided images are as follows: "sequenceType_signType_challengeType_challengeLevel_Index.bmp"

• sequenceType: 01 - Real data 02 - Unreal data

• signType: 01 - speed_limit 02 - goods_vehicles 03 - no_overtaking 04 - no_stopping 05 - no_parking 06 - stop 07 - bicycle 08 - hump 09 - no_left 10 - no_right 11 - priority_to 12 - no_entry 13 - yield 14 - parking

• challengeType: 00 - No challenge 01 - Decolorization 02 - Lens blur 03 - Codec error 04 - Darkening 05 - Dirty lens 06 - Exposure 07 - Gaussian blur 08 - Noise 09 - Rain 10 - Shadow 11 - Snow 12 - Haze

• challengeLevel: A number in between [01-05] where 01 is the least severe and 05 is the most severe challenge.

• Index: A number shows different instances of traffic signs in the same conditions.

The MEx Multi-modal Exercise dataset contains data of 7 different physiotherapy exercises, performed by 30 subjects recorded with 2 accelerometers, a pressure mat and a depth camera.

Application

The dataset can be used for exercise recognition, exercise quality assessment and exercise counting, by developing algorithms for pre-processing, feature extraction, multi-modal sensor fusion, segmentation and classification.

Data collection method

Each subject was given a sheet of 7 exercises with instructions to perform the exercise at the beginning of the session. At the beginning of each exercise the researcher demonstrated the exercise to the subject, then the subject performed the exercise for maximum 60 seconds while being recorded with four sensors. During the recording, the researcher did not give any advice or kept count or time to enforce a rhythm.

Sensors

Obbrec Astra Depth Camera 

-       sampling frequency – 15Hz 

-       frame size – 240x320

Sensing Tex Pressure Mat

-       sampling frequency – 15Hz

-       frame size – 32*16

Axivity AX3 3-Axis Logging Accelerometer

-       sampling frequency – 100Hz

-       range – 8g

Sensor Placement

All the exercises were performed lying down on the mat while the subject wearing two accelerometers on the wrist and the thigh. The depth camera was placed above the subject facing down-words recording an aerial view. Top of the depth camera frame was aligned with the top of the pressure mat frame and the subject’s shoulders such that the face will not be included in the depth camera video.

Data folder

MEx folder has four folders, one for each sensor. Inside each sensor folder,

30 folders can be found, one for each subject. In each subject folder, 8 files can be found for each exercise with 2 files for exercise 4 as it is performed on two sides. (The user 22 will only have 7 files as they performed the exercise 4 on only one side.)  One line in the data files correspond to one timestamped and sensory data.

Attribute Information

The 4 columns in the act and acw files is organized as follows:

1 – timestamp

2 – x value

3 – y value

4 – z value

Min value = -8

Max value = +8

The 513 columns in the pm file is organized as follows:

1 - timestamp

2-513 – pressure mat data frame (32x16)

Min value – 0

Max value – 1

The 193 columns in the dc file is organized as follows:

1 - timestamp

2-193 – depth camera data frame (12x16)

dc data frame is scaled down from 240x320 to 12x16 using the OpenCV resize algorithm

Min value – 0

Max value – 1

* GSR signals of each participant:The files whose names begin with letter A correspond to the GSR registers extracted from the participants. These files have a single column which correspond to the values of the GSR signal sampled at Fs=1Hz.* Labels of each signal:The files whose names begin with LABEL correspond to the labels of the RResp of each subject.These files have two columns. The first column corresponds to the label of the register and the second column corresponds to the timestamp for that given label. The registers have been labeled using 20s windows (sliding every 5s) and being the labels positioned in the center of the window. For example:-1 12.5  --> In the time window going from 2.5s to 22.5s, the RResp label corresponds to RResp=-1, being the  center of the window at 12.5s.There are four RResp intensity levels: 0 stands for the absence of any RResp, -1 for a Low intensity RResp, -2 for a Medium intensity RResp and -3 for a High intensity RResp.

The training dataset consists of 7175 fingerprints collected from 489 different locations. Each fingerprint is stored as a JSON object corresponding to an unique scan with the following values:

• _id: contains an unique identifier for the fingerprint, uses to differentiate one fingerprint from another.

• avgMagneticMagnitude: average magnetic magnitude during scanning with the mobile phone sensor, although this value is not used is provided in case it was useful.

• location: object with the coordinates of the real world in which the sample was captured.

  • floor: number indicating the floor in which the sample was captured.

  • lat: latitude as part of the coordinate at which the sample was captured.

  • lon: longitude as part of the coordinate at which the sample was captured.

• timestamp: UNIX timestamp in which the sample was captured.

• userId: identifier of the user who captured the sample, this value will be anonymized so that it is not directly identifiable but remains unique.

• wifiDevices: list of APs appearing in the sample.

  • bssid: unique AP identifier, this value will be anonymized so that it is not directly identifiable but remains unique.

  • frequency: AP WiFi frequency.

  • level: AP WiFi signal strength (RSSI).

  • ssid: AP name, this value will be anonymized so that it is not directly identifiable but can be used to compare APs with the same name.

The training dataset was compiled by taking samples at every 3 meters on average with 15 samples per location. The time at each location was approximately 40 seconds performing consecutive scans with a bq Aquaris E5 4G device using Android stock 6.0.1 without making any movements during the process. The following is an example of a fingerprint, the list of WiFi devices has been shortened to two APs, as it was too long.

{
"_id":"5cc81e8ac28d6d2533709425",
"avgMagneticMagnitude":40.615368,
"location":{
"floor":1,
"lat": 37.357746,
"lon": -5.9878354
},
"timestamp":1556618890,
"userId":"USER-0",
"wifiDevices":[
{
"bssid":"AP-BSSID-0",
"frequency":2457,
"level":-75,
"ssid":"AP-SSID-0"
},
...
{
"bssid":"AP-BSSID-23",
"frequency":2437,
"level":-64,
"ssid":"AP-SSID-6"
}
]
}

The testing dataset consists of two tests with a total of 390 samples in random locations yet in areas captured by the training dataset and with different devices. This dataset is grouped by tests and within it are the captured samples, so both the individual error and the average error can be obtained, besides recalculating this error to test different algorithms. Each test is stored as a JSON object corresponding to an unique scan with the following values:

• _id: contains an unique identifier for the test, uses to differentiate one test from another.

• userId: identifier of the user who performed the test, this value will be anonymized so that it is not directly identifiable but remains unique.

• startTimestamp: UNIX timestamp that indicates when the test was started.

• endTimestamp: UNIX timestamp that indicates when the test was ended.

• samples: list of samples taken during testing.

  • timestamp: UNIX timestamp that indicates when the sample was collected.

  • real: object with the coordinates of the real world in which the sample was captured.

    • floor: number indicating the floor in which the sample was captured.

    • lat: latitude as part of the coordinate at which the sample was captured.

    • lon: longitude as part of the coordinate at which the sample was captured.

  • predicted: object with the predicted coordinates of the real world.

    • floor: number indicating the floor predicted.

    • lat: latitude as part of the predicted coordinate.

    • lon: longitude as part of the predicted coordinate.

  • wifiDevices: list of APs appearing in the sample.

    • bssid: unique AP identifier, this value will be anonymized so that it is not directly identifiable but remains unique.

    • frequency: AP WiFi frequency.

    • level: AP WiFi signal strength (RSSI).

    • ssid: AP name, this value will be anonymized so that it is not directly identifiable but can be used to compare APs with the same name.

  • error: approximate distance between the actual location and the predicted location.

• error: average distance between the actual locations and the predicted locations.

The testing dataset was compiled two days after the training phase by taking samples at random locations with an average of 3 meters, performing a single scan per location. The samples were taken with two devices, which represent each of the tests individually, a bq Aquaris E5 4G device using Android stock 6.0.1 and a Xiaomi Redmi 4X using Android 7.1.2 with MIUI 10 Global 9.5.16. Before taking the sample, 5 seconds were waited without making any movements. The following is an example of a test entry, the list of samples has been shortened to one sample and wifi devices has been shortened to two APs, as it was too long.

{
"_id":"5d13245e279a550b548e3bfe",
"userId":"USER-0",
"startTimestamp": 1557212799.6555429,
"endTimestamp": 1557222705.0710876,
"samples":[
{
"timestamp":1557212799.6552203,
"real":{
"floor":0,
"lat":37.358547,
"lon":-5.9867215
},
"predicted":{
"floor":0,
"lat":37.358547,
"lon":-5.9868493
},
"wifiDevices":[
{
"bssid":"AP-BSSID-156",
"frequency":2412,
"level":-80,
"ssid":"AP-SSID-5"
},
...
{
"bssid":"AP-BSSID-146",
"frequency":2462,
"level":-36,
"ssid":"AP-SSID-6"
}
],
"error":5.233510868645419
},
...
],
"error":3.975672826048607
}

In order to provide more information about the device used in each fingerprint of the dataset, the following relationship between users and devices is given:

USER-0: Xiaomi Redmi 4X (Android 7.1.2 with MIUI 10 Global 9.5.16)

USER-1: BQ Aquaris E5 4G (Android stock 6.0.1)

The dataset file (cap_sensor_data.zip) contains capacitance measurements and images. CSV data is provided in the "capData_csv" folder. Images are provided in the "images" folder. The data in MATLAB format is found in "capData.mat". The MATLAB script file, "script_plot_data.m", contains code to parse and plot the data. It can be used as an example to perform data analysis. The spatial locations of the 16 channels can be found in "channel_numbers.jpg".

Please see the attached documentation file for more details.