** Please note that this is under construction, and all data and code is still being uploaded whilst this notice is present. Thank-you. Tom **

All code is hosted as a GIT repository (below), as well as instructions, which can be found by clicking on the link/file called README.md in that repository.

https://github.com/thomasmhall-newcastle/IEEE-TNSRE-2016-lfLFPs

You are free to clone/pull this repository and use it under MIT license, on the understanding that any use of this code will be acknowledged by citing the original paper, DOI: 10.1109/TNSRE.2016.2612001, which is Open Access and can be found here: http://ieeexplore.ieee.org/document/7742994/

Datasets used in the publication:

1) List of all the dataset used to generate figure 2 and 3:

• 20210614_RunHeadandTailRailswithDelays_AML470_0-20-40-60-80intensity

• 20210615_RunHeadandTailRailswithDelays_AML470_0-20-40-60-80intensity

• 20210616_RunHeadandTailRailswithDelays_AML470_0-20-40-60-80intensity

• 20210617_RunHeadandTailRailswithDelays_AML470_0-20-40-60-80intensity

2) List of all the dataset used to generate figure S3:

• 20210618_RunHeadandTailRailswithDelays_AML67_0-20-40-60-80intensity

3) List of all the dataset folders used to generate figure 4 and table 1:

a) AML67 dataset, open loop: 

• 20210624_RunFullWormRails_Sandeep_AML67_10ulRet_red,

• 20210614_RunFullWormRails_Sandeep_AML67_10ulRet_red,

• 20200902_RunFullWormRails_Sandeep_AML67_10ulret.

b) AML67 dataset, closed loop: 

• 20210624_RunRailsTriggeredByTurning_Sandeep_AML67_10ulRet_red,

• 20210614_RunRailsTriggeredByTurning_Sandeep_AML67_10ulRet_red,

• 20200902_RunRailsTriggeredByTurning_Sandeep_AML67_10ulRet.

4) List of all the dataset folders used to generate figure 4:

a) AML470 dataset, open loop: 

• 20210723_RunFullWormRails_Sandeep_AKS_483.7.e_mec4_Chrimson_10ulRet_red,

• 20210721_RunFullWormRails_Sandeep_AKS_483.7.e_mec4_Chrimson_10ulRet_red,

• 20210720_RunFullWormRails_Sandeep_AKS_483.7.e_mec4_Chrimson_10ulRet_red.

b) AML470 dataset, closed loop: 

• 20210723_RunRailsTriggeredByTurning_Sandeep_AKS_483.7.e_mec4_Chrimson_10ulRet_red,

• 20210721_RunRailsTriggeredByTurning_Sandeep_AKS_483.7.e_mec4_Chrimson_10ulRet_red, 

• 20210720_RunRailsTriggeredByTurning_Sandeep_AKS_483.7.e_mec4_Chrimson_10ulRet_red/AML470_20210720_turns_0uW_3s_new3.mat')

Instructions for accessing files: 

1) Here are the details of the naming convention used in the filenames of the datasets used to generate figure 2, 3, S3.

Let us use the folder name “20210614_RunHeadandTailRailswithDelays_AML470_0-20-40-60-80intensity” as an example. The different components of the names are:

a) Date: In the above example, this dataset was collected on 20210614 (YYYYMMDD).

b) Experiment type: "RunHeadandTailRailswithDelays" represents the data collected by stimulating head, tail, or both.

c) Name of the strain: The above example shows the dataset collected from "AML470" strain. 

d) Experiment specific information: The tag “0-20-40-60-80intensity” says that 0, 20, 40, 60, and 80uW intensity was used during this dataset.

Moreover, once you go inside each folder, you will see many subfolders with the date and time stamp. For e.g. a subfolder in the above folder is Data20210614_141921_BoxA-PC which basically says the time stamp at which this dataset was recorded (DataYYYYMMDD_HHMMSS) followed by the name of the experimental box (BoxA-PC or BoxB-PC or BoxC-PC or BoxD-PC).

2) Here are the details of the naming convention used in the filenames of the datasets used to generate figure 4 and table 1.

Let us use the folder name “20210624_RunRailsTriggeredByTurning_Sandeep_AML67_10ulRet_red” as an example. The different components of the names are:

a) Date: In the above example, this dataset was collected on 20210624 (YYYYMMDD).

b) Experiment type: "RunRailsTriggeredByTurning" represents the data collected in closed loop protocol whereas "RunFullWormRails" represents the data collected in open loop protocol. Thus, the above example represents a dataset collected using closed loop protocol.

c) Name of the user: In the above example, the user is "Sandeep".

d) Name of the strain: The above example shows the dataset collected from "AML67" strain. Folders containing the datasets collected from AML470 will say "AKS_483.7.e_mec4_Chrimson".

e) ATR information: All the datsets used retinal on the OP50 plates to grow the worms, and hence have the tag "10ulRet".

f) Color of the stim: "red" tag says that in this protocol red stimulus was delivered to the worms.

Moreover, once you go inside each folder, you will see many subfolders with the date and time stamp. For e.g. a subfolder in the above folder is Data20210624_105852 which basically says the time stamp at which this dataset was recorded (DataYYYYMMDD_HHMMSS).

Details of files inside each experimental folder:

The dataset is in the form of the output of the real-time LabVIEW instrument for maximum compression. It still needs to go through post-processing before further analysis.

Post-processing can be done by running the /ProcessDateDirectory.m MATLAB script from the code repository.

It is organized into date directories, which aggregate all the experiments collected on the same day. 

Each experiment is it's own time stamped folder within a date directory, and it contains the following files:

- camera_distortion.png contains camera spatial calibration information in the image metadata

- CameraFrames.mkv is the raw camera images compressed with H.265

- labview_parameters.csv is the settings used by the instrument in the real-time study

- labview_tracks.mat contains the real-time tracking data in a MATLAB readable HDF5 format

- projector_to_camera_distortion.png contains the spatial calibration information that maps projector pixel space into camera pixel space

- tags.txt contains tagged information for the experiment and is used to organize and select experiments for analysis

- timestamps.mat contains timing information saved during the real-time experiments, including closed-loop lag.

- ConvertedProjectorFrames folder contains png compressed stimulus images converted to the camera's frame of reference.

Visit https://github.com/EIDOSlab/UC3-UNITOBrain to have a full companion code where a U-Net model is trained over the dataset.

Procedure :

1. The tool used for preprocessing is Anaconda-Jupyter Notebook on Intel 8th gen i5 processor with 8GB RAM
2. The dataset is prepared by extracting datapoints from '.edf' by using mne package in python. Equal amount of preictal and ictal data are extracted.
3. A period of 4096 seconds (68 minutes) each of preictal and ictal data is extracted from the '.edf' files. All ictal periods for 24 patients annotated have been included in the dataset.
4. Datapoints are loaded and preprocessed as dataframes by using pandas package in python.
5. System RAM size should be available to the maximum possible extent as dataframes are large.
6. The file chbmit_preprocessed_data.csv can be used as is for machine learning and deep learning models.

Data Availability :

The datset contains following files.

• chbmit_ictal_raw_data.csv : This file contains only ictal data from all 24 patients. The channels vary largely and amount to 96 columns in this file.
• chbmit_preictal_raw_data.csv : This file contains only preictal data from all 24 patients. The channels vary largely and amount to 96 columns in this file.
• chbmit_preictal_23channels_data.csv :This file contains only preictal data from all 24 patients. Only 23 channels are retained and amount to 23 columns in this file.
• chbmit_ictal_23channels_data.csv :This file contains only ictal data from all 24 patients. Only 23 channels are retained and amount to 23 columns in this file.
• chbmit_preprocessed_data.csv :This file contains balanced preictal and ictal data from all 24 patients. Only 23 channels are retained, outcome column is added and amount to 24 columns in this file. In outcome column '0' indicates preictal and '1' indicates ictal.

This dataset is prepared with data reduction techniques. Data cleaning and data transformation need to be done as suitable for the application or model under development. 

Original Data:

The original raw dataset in '.edf' is available at https://physionet.org/content/chbmit/1.0.0/  and to be cited as 

Goldberger, A., Amaral, L., Glass, L., Hausdorff, J., Ivanov, P. C., Mark, R., ... & Stanley, H. E. (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation [Online]. 101 (23), pp. e215–e220

For each patient NCCT data is marked "2" and CTA is marked "1".

See the paper "Object classification from randomized EEG trials" on IEEE Xplore.

Code for analyzing the dataset is included in the online supplementary materials for the paper.

The code from the online supplementary materials is also included here.

If you use this code or data, please cite the above paper.