The Waterloo Streaming Quality-of-Experience Database-III
The dynamic adaptive streaming over HTTP provides an inter-operable solution to overcome the volatile network conditions, but its complex characteristic brings new challenges to objective video quality-of-experience (QoE) measurement. To test the generalizability and to facilitate the wide usage of QoE measurement techniques in real-world applications, we establish a new database named Waterloo Streaming QoE Database III (SQoE-III). Unlike existing databases constructed with hand-crafted test sequences, the SQoE-III database, so far the largest and most realistic of its kind, consists of a total of 450 streaming videos created from diverse source content and diverse distortion patterns, with six adaptation algorithms of diverse characteristics under 13 representative network conditions. All streaming videos are assessed by 34 subjects, and a comprehensive evaluation is conducted on the performance of 15 objective QoE models from four categories with regards to their efficacy in predicting subjective QoE. Detailed correlation analysis and statistical hypothesis testing are carried out. The results of this paper shed light on the future development of adaptive bitrate streaming algorithm and video QoE monitoring system.
The Waterloo Streaming Quality-of-Experience Database-III consists of 20 RAW HD reference videos and 450 simulated streaming videos, and of an average duration of 13 seconds. In order to generate meaningful and representative test videos, we conducted a set of DASH video streaming experiments, recorded the relevant streaming activities, and reconstructed the streaming session using video processing tools. The streaming sessions generated by 6 adaptive streaming algorithms including rate-based, BBA, AIMD, Elastic, QDASH, and FESTIVE under 13 wide-ranging bandwidth conditions are recorded and evaluated by 34 subjects. Subjects score the quality of each video sequence according to the 0-100 numerical quality scale.