Developing an Item Bank for Progress Tests and Application of Computerized Adaptive Testing by Simulation in Medical Education
Uloženo v:
| Název: | Developing an Item Bank for Progress Tests and Application of Computerized Adaptive Testing by Simulation in Medical Education |
|---|---|
| Jazyk: | English |
| Autoři: | Kosan, Aysen Melek Aytug (ORCID |
| Zdroj: | International Journal of Assessment Tools in Education. 2019 6(4):656-669. |
| Dostupnost: | International Journal of Assessment Tools in Education. Pamukkale University, Faculty of Education, Kinikli Campus, Denizli 20070, Turkey. e-mail: ijate.editor@gmail.com; Web site: https://ijate.net/index.php/ijate |
| Peer Reviewed: | Y |
| Page Count: | 14 |
| Datum vydání: | 2019 |
| Druh dokumentu: | Journal Articles Reports - Research |
| Education Level: | Higher Education Postsecondary Education |
| Descriptors: | Item Banks, Adaptive Testing, Computer Assisted Testing, Medical Education, Medical Students, Test Reliability, Foreign Countries, Test Validity, Construct Validity, Item Response Theory, Achievement Tests, Student Evaluation, Test Construction |
| Geografický termín: | Turkey (Ankara) |
| ISSN: | 2148-7456 |
| Abstrakt: | Progress Test (PT) is a form of assessment that simultaneously measures ability levels of all students in a certain educational program and their progress over time by providing them with same questions and repeating the process at regular intervals with parallel tests. Our objective was to generate an item bank for the PT and to examine the possible fit of CAT for PT application. This study is a descriptive study. 1206 medical students participated. During the analysis of the psychometric properties of PT item bank, "the Rasch model for dichotomous items was used". Several CAT simulations were performed by applying various stopping rules of different standard errors. CAT simulation estimates were compared with the estimates generated from the original calibration of the Rasch model where all items were included. After Rasch analysis, a unidimensional PT item bank consisting of 103 items was obtained. The item bank reliability was calculated as 0.77 with Person Separation Index (PSI) and Kuder-Richardson Formula 20 (KR20). A high correlation between [Theta] estimations obtained from paper-and-pencil ([Theta]RM) and CAT applications ([Theta[subscript CAT]) was detected for simulation conditions ([N(0,1)] and [N(0,3)]) at the end of our analysis. In CAT, estimation can be made with an average of 14 questions (reduced 86,4%) and 17 questions (reduced 83,4%) [for N(0,1) and [N(0,3) respectively] with reliability of 0,75. This study reveals that it is possible to develop an appropriate item bank for the PT, and the difficulty of administering large number of items in PT can be scaled down by incorporating CAT application. |
| Abstractor: | As Provided |
| Entry Date: | 2020 |
| Přístupové číslo: | EJ1243550 |
| Databáze: | ERIC |
| Abstrakt: | Progress Test (PT) is a form of assessment that simultaneously measures ability levels of all students in a certain educational program and their progress over time by providing them with same questions and repeating the process at regular intervals with parallel tests. Our objective was to generate an item bank for the PT and to examine the possible fit of CAT for PT application. This study is a descriptive study. 1206 medical students participated. During the analysis of the psychometric properties of PT item bank, "the Rasch model for dichotomous items was used". Several CAT simulations were performed by applying various stopping rules of different standard errors. CAT simulation estimates were compared with the estimates generated from the original calibration of the Rasch model where all items were included. After Rasch analysis, a unidimensional PT item bank consisting of 103 items was obtained. The item bank reliability was calculated as 0.77 with Person Separation Index (PSI) and Kuder-Richardson Formula 20 (KR20). A high correlation between [Theta] estimations obtained from paper-and-pencil ([Theta]RM) and CAT applications ([Theta[subscript CAT]) was detected for simulation conditions ([N(0,1)] and [N(0,3)]) at the end of our analysis. In CAT, estimation can be made with an average of 14 questions (reduced 86,4%) and 17 questions (reduced 83,4%) [for N(0,1) and [N(0,3) respectively] with reliability of 0,75. This study reveals that it is possible to develop an appropriate item bank for the PT, and the difficulty of administering large number of items in PT can be scaled down by incorporating CAT application. |
|---|---|
| ISSN: | 2148-7456 |