Realistic Multi-view Synthetic Dataset with RGB and Polarization Images Pairs

In the current field of multi-view polarization 3D reconstruction, existing datasets primarily fall into two categories: real-world dataset and synthetic dataset. Real-world datasets provide accurate acquisition of illumination from real-world environments, but requires complex setups and specialized equipment such as polarization cameras. Furthermore, specific environmental conditions, such as special lighting conditions or reflective properties, may be difficult to replicate precisely in reality. In contrast, synthetic datasets are more convenient, allowing for the generation of data from any scene and viewpoint as needed. This convenience greatly facilitates research, allowing researchers to quickly generate and test different datasets without waiting for the collection and processing of actual data.

However, most existing synthetic datasets are generated by rendering with empirical method, such as the Blinn-Phong rendering model, and obtaining polarization images by performing linear operations directly on the rendered images according to the normals of the model. This approach cannot accurately simulate the physical properties in the real world, thereby reducing the authenticity and reliability of the generated images. Therefore, it is meaningful to create a dataset based on the polarization Bidirectional Reflectance Distribution Function (pBRDF) material and path tracing technology. It can provide more realistic images that adhere to the physical properties of the real world, which will help to more accurately verify and improve our method.

Here we use Mitsuba 2 to render these images. For each view we rendered a RGB and four polarized images with different AoP, and we rendered 36 views for each models. We utilize four representative CG models (Armadillo, Stanford bunny, Dragon, and Buddha) available from Stanford 3D Scanning Repository for synthesis. The generated polarized images are all under real-world lighting conditions. The lighting condition is obtained by sampling the light intensity at 360-degree directions within a specific environment using a light probe. The resolution of images in dataset are 400 x 400. We also provide the camera intrinsic and extrinsic matrix in the json file.