On Designing Paired Comparison Experiments for Subjective Multimedia Quality Assessment

This paper investigates the issue of designing paired comparison-based subjective quality assessment experiments for reliable results. In particular, the convergence behavior of the quality scores estimated from paired comparison results is considered. Via an extensive computer simulation experiment...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on multimedia Jg. 16; H. 2; S. 564 - 571
1. Verfasser: Lee, Jong-Seok
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York, NY IEEE 01.02.2014
Institute of Electrical and Electronics Engineers
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Applied sciences
Bradley-Terry model
Computer science; control theory; systems
Computer simulation
Convergence
Correlation coefficients
Encoding
Errors
Exact sciences and technology
Mathematical models
mean opinion score (MOS)
Multimedia
Multimedia communication
paired comparison
Quality assessment
Quality of experience (QoE)
Roots
Simulation
Software
Streaming media
subjective quality assessment
Video coding
Correlation coefficient
Performance evaluation
Multimedia
Paired comparison
Computer simulation
Bradley-Terry model
Case based reasoning
Modeling
Subjective evaluation
Mean square error
Human error
subjective quality assessment
mean opinion score (MOS)
Quality of experience (QoE)
Service quality
ISSN:1520-9210, 1941-0077
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:This paper investigates the issue of designing paired comparison-based subjective quality assessment experiments for reliable results. In particular, the convergence behavior of the quality scores estimated from paired comparison results is considered. Via an extensive computer simulation experiment, the estimation performance in terms of the root mean squared error, the rank order correlation coefficient, and the change of the estimated scores with respect to the number of subjects are mathematically modeled. Furthermore, it is confirmed that the models coincide with the theoretical convergence behavior. Issues such as the effect of human errors and the underlying distribution of the true quality scores are also examined.
Bibliographie:ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
content type line 23
ISSN:1520-9210
1941-0077
DOI:10.1109/TMM.2013.2292590