What Doesn’t Get Measured Does Exist: Improving the Accuracy of Computer-Aided Text Analysis

Computer-aided text analysis (CATA) is a form of content analysis that enables the measurement of constructs by processing text into quantitative data based on the frequency of words. CATA has been proposed as a useful measurement approach with the potential to lead to important theoretical advancem...

Full description

Saved in:
Bibliographic Details
Published in:Journal of management Vol. 44; no. 7; pp. 2909 - 2933
Main Authors: McKenny, Aaron F., Aguinis, Herman, Short, Jeremy C., Anglin, Aaron H.
Format: Journal Article
Language:English
Published: Los Angeles, CA SAGE Publications 01.09.2018
Sage Publications Ltd
Subjects:
Accuracy
Content analysis
Data
Data processing
Handedness
Measurement
Measurement errors
Text analysis
Word frequency
ambidexterity
measurement error
content analysis
entrepreneurial orientation
reliability
market orientation
ISSN:0149-2063, 1557-1211
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Computer-aided text analysis (CATA) is a form of content analysis that enables the measurement of constructs by processing text into quantitative data based on the frequency of words. CATA has been proposed as a useful measurement approach with the potential to lead to important theoretical advancements. Ironically, while CATA has been offered to overcome some of the known deficiencies in existing measurement approaches, we have lagged behind in regard to assessing the technique’s measurement rigor. Our article addresses this knowledge gap and describes important implications for past as well as future research using CATA. First, we describe three sources of measurement error variance that are particularly relevant to studies using CATA: transient error, specific factor error, and algorithm error. Second, we describe and demonstrate how to calculate measurement error variance with the entrepreneurial orientation, market orientation, and organizational ambidexterity constructs, offering evidence that past substantive conclusions have been underestimated. Third, we offer best-practice recommendations and demonstrate how to reduce measurement error variance by refining existing CATA measures. In short, we demonstrate that although measurement error variance in CATA has not been measured thus far, it does exist and it affects substantive conclusions. Consequently, our article has implications for theory and practice, as well as how to assess and minimize measurement error in future CATA research with the goal of improving the accuracy of substantive conclusions.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0149-2063
1557-1211
DOI:10.1177/0149206316657594