Reliability in content analysis: The case of semantic feature norms classification

Semantic feature norms (e.g., STIMULUS: car → RESPONSE: <has four wheels>) are commonly used in cognitive psychology to look into salient aspects of given concepts. Semantic features are typically collected in experimental settings and then manually annotated by the researchers into feature ty...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Behavior research methods Ročník 49; číslo 6; s. 1984 - 2001
Hlavní autoři: Bolognesi, Marianna, Pilgram, Roosmaryn, van den Heerik, Romy
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York Springer US 01.12.2017
Springer Nature B.V
Témata:
Behavioral Science and Psychology
Classification
Cognitive ability
Cognitive Psychology
Content analysis
Humans
Methodological problems
Norms
Perceptions
Psycholinguistics - methods
Psycholinguistics - standards
Psychology
Qualitative Research
Reliability
Reproducibility of Results
Research methodology
Semantic analysis
Semantic features
Semantics
Stimulus
Sustainability
Taxonomy
Intercoder reliability
Semantic feature norms
Content analysis
ISSN:1554-3528, 1554-3528
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Semantic feature norms (e.g., STIMULUS: car → RESPONSE: <has four wheels>) are commonly used in cognitive psychology to look into salient aspects of given concepts. Semantic features are typically collected in experimental settings and then manually annotated by the researchers into feature types (e.g., perceptual features, taxonomic features, etc.) by means of content analyses—that is, by using taxonomies of feature types and having independent coders perform the annotation task. However, the ways in which such content analyses are typically performed and reported are not consistent across the literature. This constitutes a serious methodological problem that might undermine the theoretical claims based on such annotations. In this study, we first offer a review of some of the released datasets of annotated semantic feature norms and the related taxonomies used for content analysis. We then provide theoretical and methodological insights in relation to the content analysis methodology. Finally, we apply content analysis to a new dataset of semantic features and show how the method should be applied in order to deliver reliable annotations and replicable coding schemes. We tackle the following issues: (1) taxonomy structure, (2) the description of categories, (3) coder training, and (4) sustainability of the coding scheme—that is, comparison of the annotations provided by trained versus novice coders. The outcomes of the project are threefold: We provide methodological guidelines for semantic feature classification; we provide a revised and adapted taxonomy that can (arguably) be applied to both concrete and abstract concepts; and we provide a dataset of annotated semantic feature norms.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:1554-3528
1554-3528
DOI:10.3758/s13428-016-0838-6