Towards an automated evaluation of design processes—an algorithm to predict critical situations during concept synthesis

In design research, there is a need for research methods that allow for larger numbers of participants in empirical studies, as small sample sizes lead to less statistically reliable results. The number of participants is limited by current research methods, such as protocol analysis, interviews, or...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Research in engineering design Ročník 36; číslo 2; s. 3
Hlavní autoři: Zimmerer, Christoph, Wittig, Christoph, Matthiesen, Sven
Médium: Journal Article
Jazyk:angličtina
Vydáno: London Springer London 01.04.2025
Springer Nature B.V
Témata:
Algorithms
Automation
CAE) and Design
Cognition
Computer-Aided Engineering (CAD
Data analysis
Datasets
Design
Design analysis
Engineering
Engineering Design
Eye movements
Machine learning
Original Paper
Research methodology
Statistical methods
Synthesis
Tracking
Data-driven evaluation
Design research
Conceptual design
Eye tracking
Machine learning
ISSN:0934-9839, 1435-6066
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In design research, there is a need for research methods that allow for larger numbers of participants in empirical studies, as small sample sizes lead to less statistically reliable results. The number of participants is limited by current research methods, such as protocol analysis, interviews, or analysis of eye tracking videos, because they require a lot of manual work during the evaluation. This is particularly noticeable in studies that analyze the cognitive processes of designers, e.g., during concept synthesis. Therefore, the goal of this paper is to develop an algorithm that automates the analysis of eye tracking data to predict designer perceived difficulties—critical situations in which methodical support could be beneficial—during design processes. By linking eye tracking data with retrospective think aloud data, a dataset was created for training different machine learning algorithms. The dataset was further processed, and three algorithms were evaluated regarding their suitability for automated detection of difficulties. The best algorithm, a cascading one-against-all classifier, achieved an accuracy of 62% and a false-negative rate of 26.6%. Depending on the type of difficulty, different eye tracking features were relevant for the decisions of the algorithms, highlighting the importance of tailored feature selection for each type of difficulty. The findings suggest that the automated analysis of eye tracking data using machine learning potentially facilitates larger studies and statistically more reliable findings, representing a significant step toward more efficient and insightful analysis of design cognition.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0934-9839
1435-6066
DOI:10.1007/s00163-025-00444-2