Active Learning for Crowdsourced QoE Modeling

Quality of experience (QoE) models predict the subjective quality of multimedia based on the relevant quality of service (QoS) factors. Due to the large space of QoS factors and the high costs of conducting subjective tests, efficient sampling strategies are required to determine which QoS configura...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on multimedia Ročník 20; číslo 12; s. 3337 - 3352
Hlavní autoři: Chang, Haw-Shiuan, Hsu, Chih-Fan, Hosfeld, Tobias, Chen, Kuan-Ta
Médium: Journal Article
Jazyk:angličtina
Vydáno: Piscataway IEEE 01.12.2018
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Active learning
Algorithms
Computer simulation
Crowdsourcing
Habituation (learning)
IQX and MIQX model
Machine learning
maximin sampling
multidimensional QoS-QoE model
Multimedia
Multimedia communication
Quality assessment
Quality of experience
Quality of service
Random sampling
Sampling methods
Streaming media
video quality assessment
ISSN:1520-9210, 1941-0077
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Quality of experience (QoE) models predict the subjective quality of multimedia based on the relevant quality of service (QoS) factors. Due to the large space of QoS factors and the high costs of conducting subjective tests, efficient sampling strategies are required to determine which QoS configurations are to be queried, that is, evaluated by subjects. In this study, we extend the IQX model proposed by [M. Fiedler, T. Hoßfeld, and P. Tran-Gia, "A generic quantitative relationship between quality of experience and quality of service," IEEE Netw. , vol. 24, no. 2, pp. 36-41, Mar./Apr.2010.] toward a multidimensional QoS-QoE model (MIQX). To explore the complicated interaction between QoS factors more efficiently, we develop active learning algorithms for the multidimensional QoE model. Then, we conduct comprehensive experiments to compare the effectiveness of applying different sampling methods to crowdsourced video quality assessment tasks. In offline experiments that assume annotators give the same scores after changing the querying order, we demonstrate that active learning performs best and that a space-filling algorithm performs significantly better than random sampling. However, when we analyze the performance of the active sampling approaches more deeply using a novel field experiment, we observe that the active learning algorithms, which have been shown to be effective in the offline setting, can fail due to the habituation effect and individual differences of annotators. The active learning methods can also succeed when these issues are mitigated. These findings suggest that simply simulating the sample acquisition order, which is widely adopted in previous active learning literature  <xref ref-type="bibr" rid="ref2 ref3 ref4">[2] -<xref ref-type="bibr" rid="ref5">[5] , is not sufficient for multimedia quality assessment tasks.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1520-9210
1941-0077
DOI:10.1109/TMM.2018.2831639