Datasets
Open Access
Subjective and Objective Quality Assessment of the SoftCast Video Transmission Scheme
- Citation Author(s):
-
AnthonyTRIOUX
UMR 8520 - IEMN, DOAE, Univ. Polytechnique Hauts-de-France, CNRS, Univ. Lille, YNCREA, Centrale Lille - Submitted by:
- Anthony TRIOUX
- Last updated:
- Sat, 08/14/2021 - 08:28
- DOI:
- 10.21227/w3gy-tx35
- Data Format:
- License:
434 Views- Categories:
- Keywords:
Abstract
SoftCast-based linear video coding and transmission (LVCT) schemes have been proposed as a promising alternative to traditional video coding and transmission schemes in wireless environments. Currently, the performance of LVCT schemes is evaluated by means of traditional objective scores such as PSNR or SSIM. Nevertheless, since the compression is performed in a very different way from traditional coding schemes such as HEVC, visual artifacts are also quite different and deserve to be subjectively assessed.In this paper, we propose a subjective quality assessment of SoftCast, pioneer and standard of the LVCT schemes. This study aims to better understand the tradeoffs between the LVCT parameters that can be tuned to improve the quality. These parameters, including different GoP-sizes, Compression Ratios (CR) and Channel Signal-to-Noise Ratio (CSNR), are used to generate a dataset of 85 videos. A Double Stimulus Impairment Scale (DSIS) test is performed on the received videos to assess the perceived quality. Results show that the key characteristic of SoftCast, the linear relation between CSNR and PSNR, is also observed with the Mean-Opinion Scores (MOS), except at high CSNR where the quality saturates. In addition, Bjøntegaard model is used to quantify the trade-offs between CR, GoP-size and CSNR, depending on the intended application. Finally, the performance of objective metrics compared to the obtained MOS is evaluated. Results show that Multi-Scale SSIM (MS-SSIM), SSIM and Video Multimethod Assessment Fusion (VMAF) metrics offer the best correlation with the MOS values.
For more information, please refer to the following paper:
Anthony Trioux, Giuseppe Valenzise, Marco Cagnazzo, Michel Kieffer, François-Xavier Coudoux, et al., Subjective and Objective Quality Assessment of the SoftCast Video Transmission Scheme. 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP), Dec 2020, Macau, China.
The SoftCast Quality-of-Experience Database consists of 5 RAW HD reference videos and 85 videos transmitted and received through the SoftCastlinear video coding and transmission scheme, each with a duration of 5 seconds. Note that only the luminance is considered in this database. Furthermore, the number of frames depends on the framerate of the video (125 frames for 25fps and 150frames for 30fps).
In order to generate several scenario of transmission, three parameters were considered: Two GoP-sizes (8 and 32 frames), two Compression Ratios (CR=1 and 0.25) and Channel Signal-to-Noise Ratio (CSNR varying from 0 to 30dB by 3dB step). This database was evaluated by 21 participants (8 women and 13 men). They were asked to score the quality of each received video sequence according to the original one on a numerical impairment scale [0-100]. A training session including 10 stimuli was organized for each observer prior to the test in order to familiarize them with the procedure, the specific artifacts of SoftCast as well as the impairment scale.
Video files are named using the following structure:
Video_filename_y_only_GoP_X_CR_Y_CSNR_ZdB.yuv where X equals either 8 or 32 frames, Y equals either 1 or 0.25 and Z either equals 0,3,6,9,12,15,18,21,24,27,30dB.
The original video files are denoted: Video_filename_y_only_ori.yuv.
Each video file is in *.yuv format (4:2:0) where the chrominance plans are all set to 128. (This process allows to perform the VMAF computation as VMAF requires either a yuv420p, yuv422p, yuv444p, yuv420p10le, yuv422p10le or yuv444p10le video format).
The subjective scores are available in the MOS_scores.xls file.
The objective scores (frame by frame) for each videos are available in the objective_scores.zip file.
Dataset Files
MOS_scores.xlsx (12.05 kB)
training_sequences.zip (2.53 GB)- SoftCast_test_database.zip (16.35 GB)
- objective_scores.zip (461.60 kB)
Open Access dataset files are accessible to all logged in users. Don't have a login? Create a free IEEE account. IEEE Membership is not required.
Documentation
| Attachment | Size |
|---|---|
 Related_paper.pdf | 2.3 MB |
| READ_ME.pdf | 262.08 KB |
Comments
If you are interested in our work, please visit the following link about a related dataset:
https://ieee-dataport.org/open-access/perceptual-study-decoding-process-...