Sintel 4D Light Field Video Dataset

Citation Author(s):
Takahiro
Kinoshita
Media Engineering Laboratory, Department of Information Science and Biomedical Engineering, Graduate School of Science and Engineering, Kagoshima University, Japan.
Satoshi
Ono
Media Engineering Laboratory, Department of Information Science and Biomedical Engineering, Graduate School of Science and Engineering, Kagoshima University, Japan.
Submitted by:
Takahiro Kinoshita
Last updated:
Fri, 03/26/2021 - 14:10
DOI:
10.1117/12.2591012
Data Format:
*.png *.npy
Links:
Project page
GitHub
Paper
License:
Creative Commons Attribution
2312 Views
Categories:
Artificial Intelligence
Image Processing
Computer Vision
Machine Learning
Keywords:
Light field, Light field video, synthetic, Sintel, Depth estimation, Disparity estimation
0
0 ratings - Please login to submit your rating.
CITE SHARE/EMBED

Abstract 

A medium-scale synthetic 4D Light Field video dataset for depth (disparity) estimation. From the open-source movie Sintel. The dataset consists of 24 synthetic 4D LFVs with 1,204x436 pixels, 9x9 views, and 20–50 frames, and has ground-truth disparity values, so that can be used for training deep learning-based methods. Each scene was rendered with a clean pass after modifying the production file of Sintel with reference to the MPI Sintel dataset.

Click here for image samples. (Our project page)

 

Updates

  • 11th FEBRUARY 2021

    The datasets containing 9x9 views are now available! Please check Sintel_LFV_9x9_with_all_disp.zip and Sintel_LFV_9x9.zip.

  • 26th NOVEMBER 2020

    Welcome to the Sintel 4D Light Field Video Dataset. The dataset used in our paper is available! Please check Sintel_LFV_cross-hair.zip.

 

Citation

If you find this useful for your research, please use the following.

==========================================

@inproceedings{kinoshita2021depth,
  title={Depth estimation from 4D light field videos},
  author={Kinoshita, Takahiro and Ono, Satoshi},
  booktitle={International Workshop on Advanced Imaging Technology (IWAIT) 2021},
  volume={11766},
  pages={117660A},
  year={2021},
  organization={International Society for Optics and Photonics}
}

==========================================

 

Instructions: 

Overview

Light Field videos:
  • 24 synthetic scenes
  • 1,204x436 pixels
  • 9x9 views
  • 20--50 frames
Ground-truth disparity values:
  • Provides disparity values for all scenes, all views, all frames, and all pixels.
  • The disparity value was obtained by transforming the depth value obtained in Blender.
    • The unit of disparity is [mm], so if the unit of [px] is needed, it needs to be multiplied by 32 to convert. (Mentioned in this issue)
Light Field setup:
  • Rendering with a “clean” pass using Blender (render25 branch).
  • The Light Field was captured by moving the camera to 9x9 viewpoints with a baseline of 0.01[m] towards a common focal plane while keeping the optical axes parallel. 

 

Datasets

Three types of datasets are provided on this page.

The reason for the three types is to eliminate the need to download extra data.

All types include all scene, all frames, and differ only in the RGB and disparity views.

1. Sintel_LFV_9x9_with_all_disp.zip
  • Includes RGB sequences for 9x9 views and disparity sequences for 9x9 views.
  • The unzipped file has 190GiB.
  • It can be used for a variety of depth estimations, e.g. not only light field but also (multi) stereo, as it includes the disparity for all views.
2. Sintel_LFV_9x9.zip
  • Includes RGB sequences for 9x9 views and disparity sequences for center view.
  • The unzipped file has 51.4GiB.
  • It can be used for light field-based depth estimations using 9x9 views.
3. Sintel_LFV_cross-hair.zip
  • Includes RGB sequences for cross-hair views and disparity sequences for center view.
  • The unzipped file has 12.1GiB.
  • It can be used for light field-based depth estimations using cross-hair views.
    • This is the data we used in our paper. (Note: We didn't use the scene named shaman_b_2 because it was not completed at that time.)

* The datasets contain RGB in .png and disparity in .npy.

 

File structure.

The following is the case of Sintel_LFV_9x9_with_all_disp.

In other cases, there is no view directory or no disparity file.

The naming convention for the view directory is {viewpoint_y:02}_{viewpoint_x:02} with 00_00 being the upper left viewpoint.

==========================================

Sintel_LFV_9x9_with_all_disp/
  ┣━━ ambushfight_1/    ...    scene directory
  ┃          ┣━━ 00_00/ ...    view directory
  ┃          ┃         ┣━━ 000.png ...    RGB of frame 0
  ┃          ┃         ┣━━ 000.npy ...    disparity of frame 0
  ┃          ┃         ┣━━ 001.png ...    RGB of frame 1
  ┃          ┃         ┣━━ 001.npy ...    disparity of frame 1
  ┃          ┣━━ 04_04/ ...    center view directory 
  ┃          ┃         ┣━━ 000.png ...    RGB of frame 0
  ┃          ┃         ┣━━ 000.npy ...    disparity of frame 0
  ┃          ┗━━ .../
  ┣━━ ambushfight_2/
  ┣━━ ambushfight_3/
  ┗━━ .../

==========================================

Dataset Files

LOGIN TO ACCESS DATASET FILES
Open Access dataset files are accessible to all logged in  users. Don't have a login?  Create a free IEEE account.  IEEE Membership is not required.

QUESTIONS?

Login to Send Author a Private Message

Report a problem with this Dataset