On Designing Paired Comparison Experiments for Subjective Multimedia Quality Assessment

This paper investigates the issue of designing paired comparison-based subjective quality assessment experiments for reliable results. In particular, the convergence behavior of the quality scores estimated from paired comparison results is considered. Via an extensive computer simulation experiment...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on multimedia Ročník 16; číslo 2; s. 564 - 571
Hlavní autor: Lee, Jong-Seok
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York, NY IEEE 01.02.2014
Institute of Electrical and Electronics Engineers
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Applied sciences
Bradley-Terry model
Computer science; control theory; systems
Computer simulation
Convergence
Correlation coefficients
Encoding
Errors
Exact sciences and technology
Mathematical models
mean opinion score (MOS)
Multimedia
Multimedia communication
paired comparison
Quality assessment
Quality of experience (QoE)
Roots
Simulation
Software
Streaming media
subjective quality assessment
Video coding
Correlation coefficient
Performance evaluation
Multimedia
Paired comparison
Computer simulation
Bradley-Terry model
Case based reasoning
Modeling
Subjective evaluation
Mean square error
Human error
subjective quality assessment
mean opinion score (MOS)
Quality of experience (QoE)
Service quality
ISSN:1520-9210, 1941-0077
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:This paper investigates the issue of designing paired comparison-based subjective quality assessment experiments for reliable results. In particular, the convergence behavior of the quality scores estimated from paired comparison results is considered. Via an extensive computer simulation experiment, the estimation performance in terms of the root mean squared error, the rank order correlation coefficient, and the change of the estimated scores with respect to the number of subjects are mathematically modeled. Furthermore, it is confirmed that the models coincide with the theoretical convergence behavior. Issues such as the effect of human errors and the underlying distribution of the true quality scores are also examined.
Bibliografie:ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
content type line 23
ISSN:1520-9210
1941-0077
DOI:10.1109/TMM.2013.2292590