Citation Author(s):
Submitted by:
Xiongkuo Min
Last updated:
Sat, 11/30/2019 - 00:37
Data Format:
Dataset Views:
0 ratings - Please login to submit your rating.
Share / Embed Cite

Smartphone has been one of the most popular digital devices in the past decades, with more than 300 million smartphones sold every quarter in the world wide. Most of the smartphone vendors, such as Apple, Huawei, Samsung, launch their new flagship smartphones every year. People use smartphone cameras to shoot selfie photos, film scenery or events, and record videos of family and friends. The specifications of smartphone camera and the quality of taken pictures are major criteria for consumer to select and buy smartphones. Many smartphone manufacturers also introduce and advertise their smartphones by introducing the strengths and advantages of their smartphone cameras. However, how to evaluate the quality of smartphone cameras and the taken pictures remains a problem for both smartphone manufacturers and confumers. Currently in the market, there are several teams and companies who evaluate the quality of smartphone cameras and announce the ranking and scores of the quality of smartphone cameras, and the scores of smartphone cameras are subjectively graded by several photographers and experts from different aspects, such as exposure, color, noise and texture. However, subjective assessment is not easy to reproduce, and it is not easy to deploy in practical image processing systems.

In the last two decades, objective image quality assessment (IQA) has been widely researched, and a large amount of objective IQA algorithms have been designed to automatically and accurately estimate the quality of images. However, most of objective IQA methods are designed to assess the overall perceived quality of the image degraded by various simulated distortions, which rarely exist in pictures taken by the modern smartphone cameras. Thus these methods are not suitable for the task of smartphone camera quality assessment, while objective evaluation methods specifically designed for the purpose of smartphone camera quality assessment are relatively rare.

The purpose of this Grand Challenge is to drive efforts of image quality assessment towards smartphone camera quality assessment. With this Grand challenge, it is expected to develop objective smartphone camera quality assessment models from four different aspects, including exposure, color, noise and texture, by using the datasets released by the organizers. The goal is to provide reference quality rankings or scores for smartphone cameras and to both smartphone manufacturers and consumers.

Participants are asked to submit four computational models to cacluate the rankings of smartphone camera from four aspects: exposure, color, noise and texture.


A training dataset will be available. It is composed of 1500 pictures taken from 100 scenes using 15 smartphones. The 15 smartphones cover a wide price range. The dataset includes various challenge scenes, e.g. high dynamic scenes, backlight scenes, night scenes, colorful scenes, portraits and distant scenes. For the 15 pictures of the same scene, four rankings of the quality of 15 smartphone cameras in four aspects: exposure, color, noise and texture will be provided. The 15 smartphones, are named and provided as Device A, Device B, and so on, and the realistic details of these smartphones will not be provided.

In our training dataset, 1 in rankings means the best performance and 15 means the worst. When several smartphones have same performance, we will provide their average rankings. For example, Device A, B, C, and their positions in the whole ranking are 3, 4, 5, we define them as ranking 4.

Please do not hesitate to contact us via email: if you have queries about the dataset.



Submitted by Wenhan Zhu on Fri, 11/08/2019 - 07:07

Dataset Files

You must be an IEEE Dataport Subscriber to access these files. Login or subscribe now. Sign up to be a Beta Tester and receive a coupon code for a free subscription to IEEE DataPort!

Thank you for rating this dataset!

Please share additional details of your rating with the IEEE DataPort community by adding a comment.

Embed this dataset on another website

Copy and paste the HTML code below to embed your dataset:

Share via email or social media

Click the buttons below:

[1] Xiongkuo Min, "QA4Camera", IEEE Dataport, 2019. [Online]. Available: Accessed: Jul. 07, 2020.
doi = {10.21227/875n-8q28},
url = {},
author = {Xiongkuo Min },
publisher = {IEEE Dataport},
title = {QA4Camera},
year = {2019} }
T1 - QA4Camera
AU - Xiongkuo Min
PY - 2019
PB - IEEE Dataport
UR - 10.21227/875n-8q28
ER -
Xiongkuo Min. (2019). QA4Camera. IEEE Dataport.
Xiongkuo Min, 2019. QA4Camera. Available at:
Xiongkuo Min. (2019). "QA4Camera." Web.
1. Xiongkuo Min. QA4Camera [Internet]. IEEE Dataport; 2019. Available from :
Xiongkuo Min. "QA4Camera." doi: 10.21227/875n-8q28