A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form

•The User Engagement Scale (UES) measures self-reported user engagement.•We refined the factor structure of the UES using mirt for the R statistics program.•We developed a new short form of the UES (UES-SF).•The full UES and UES-SF were validated with a new data set.•We offer guidance for adopting t...

Full description

Saved in:
Bibliographic Details
Published in:International journal of human-computer studies Vol. 112; pp. 28 - 39
Main Authors: O’Brien, Heather L., Cairns, Paul, Hall, Mark
Format: Journal Article
Language:English
Published: Elsevier Ltd 01.04.2018
Subjects:
Measurement
Questionnaires
Reliability
User engagement
Validity
Measurement
Validity
Reliability
Questionnaires
User engagement
ISSN:1071-5819, 1095-9300
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:•The User Engagement Scale (UES) measures self-reported user engagement.•We refined the factor structure of the UES using mirt for the R statistics program.•We developed a new short form of the UES (UES-SF).•The full UES and UES-SF were validated with a new data set.•We offer guidance for adopting the UES and UES-SF in other studies. User engagement (UE) and its measurement have been of increasing interest in human-computer interaction (HCI). The User Engagement Scale (UES) is one tool developed to measure UE, and has been used in a variety of digital domains. The original UES consisted of 31-items and purported to measure six dimensions of engagement: aesthetic appeal, focused attention, novelty, perceived usability, felt involvement, and endurability. A recent synthesis of the literature questioned the original six-factors. Further, the ways in which the UES has been implemented in studies suggests there may be a need for a briefer version of the questionnaire and more effective documentation to guide its use and analysis. This research investigated and verified a four-factor structure of the UES and proposed a Short Form (SF). We employed contemporary statistical tools that were unavailable during the UES’ development to re-analyze the original data, consisting of 427 and 779 valid responses across two studies, and examined new data (N=344) gathered as part of a three-year digital library project. In this paper we detail our analyses, present a revised long and short form (SF) version of the UES, and offer guidance for researchers interested in adopting the UES and UES-SF in their own studies.
ISSN:1071-5819
1095-9300
DOI:10.1016/j.ijhcs.2018.01.004