How accurately can supervised machine learning model predict a targeted psychiatric disorder?
Saved in:
| Title: | How accurately can supervised machine learning model predict a targeted psychiatric disorder? |
|---|---|
| Authors: | Haitham Jahrami, Amir H. Pakpour, Waqar Husain, Achraf Ammar, Zahra Saif, Ali Husain Alsalman, Adel Aloffi, Khaled Trabelsi, Seithikurippu R. Pandi-Perumal, Michael V. Vitiello |
| Source: | BMC Psychiatry BMC Psychiatry, Vol 24, Iss 1, Pp 1-9 (2024) |
| Publisher Information: | Springer Science and Business Media LLC, 2024. |
| Publication Year: | 2024 |
| Subject Terms: | Healthcare Analytics, Male, Adult, Psychiatric Status Rating Scales, Psychiatry, Anxiety Disorders/diagnosis [MeSH], Female [MeSH], Machine learning, Aged [MeSH], Adult [MeSH], Self Report [MeSH], Humans [MeSH], Middle Aged [MeSH], Disease Assessment, Artificial Intelligence, Psychiatric Status Rating Scales/standards [MeSH], Male [MeSH], Psychiatry [MeSH], Research, Supervised Machine Learning [MeSH], Hoarding disorders, RC435-571, Middle Aged, Anxiety Disorders, Humans, Female, Supervised Machine Learning, Self Report, Aged |
| Description: | Hoarding disorder (HD) is characterized by a compulsion to collect belongings, and to experience significant distress when parting from them. HD is often misdiagnosed for several reasons. These include patient and family lack of recognition that it is a psychiatric disorder and professionals' lack of relevant expertise with it. This study evaluates the ability of a supervised machine learning (ML) model to match the diagnostic skills of psychiatrists when presented with equivalent information pertinent to symptoms of HD.Five hundred online participants were randomly recruited and completed the Hoarding Rating Scale-Self Report (HRS-SR) and the Generalized Anxiety Disorder 7-item (GAD-7) scale. Responses to the questionnaires were read by an ML model. Responses to the HRS-SR were then converted into anonymized, random-equivalent texts. Each of these individual texts was presented in random order to two experienced psychiatrists who were independently asked for a provisional diagnosis - e.g.; the presence or absence of HD. In case of disagreement between the two assessors, a third psychiatrist broke the tie. A decision tree classification model was employed to predict clinical HD using self-report data from two psychological tests, the HRS-SR and GAD-7. The target variable was whether a participant had clinical HD, while the predictive variables were the continuous scores from the HRS-SR and GAD-7 tests. The model's performance was evaluated using a confusion matrix, which compared the observed diagnoses with the predicted diagnoses to assess accuracy.According to the psychiatrists, approximately 10% of the participants fulfilled DSM-5 diagnostic criteria for HD. 93% of the clinician-identified cases were identified by the ML model based on HRS-SR and GAD-7 scores. A decision tree plot model demonstrated that about 60% of the cases could be detected by the HRS-SR alone while the rest required a combination of HRS-SR and GAD-7 scores. ML evaluation metrics showed satisfactory performance, with a Matthews Correlation Coefficient of 55%; Area Under Curve (AUC), 79%; a Negative Predictive Value of 76%; and a False Negative Rate of 24%.Study findings strongly suggest that ML can, in the future, play a significant role in the risk assessment of psychiatric disorders prior to face-to-face consultation. By using AI to scan big data questionnaire responses, wait time for seriously ill patients can be substantially cut, and prognoses substantially improved. |
| Document Type: | Article Other literature type |
| Language: | English |
| ISSN: | 1471-244X |
| DOI: | 10.1186/s12888-024-06152-w |
| Access URL: | https://pubmed.ncbi.nlm.nih.gov/39407239 https://doaj.org/article/7add043437194b0c8cd83a6ed6255ca2 https://repository.publisso.de/resource/frl:6490306 |
| Rights: | CC BY NC ND |
| Accession Number: | edsair.doi.dedup.....c69d306857dc18fb5eb451714122dd46 |
| Database: | OpenAIRE |
Full Text 
Full Text Finder 
Nájsť tento článok vo Web of Science | Abstract: | Hoarding disorder (HD) is characterized by a compulsion to collect belongings, and to experience significant distress when parting from them. HD is often misdiagnosed for several reasons. These include patient and family lack of recognition that it is a psychiatric disorder and professionals' lack of relevant expertise with it. This study evaluates the ability of a supervised machine learning (ML) model to match the diagnostic skills of psychiatrists when presented with equivalent information pertinent to symptoms of HD.Five hundred online participants were randomly recruited and completed the Hoarding Rating Scale-Self Report (HRS-SR) and the Generalized Anxiety Disorder 7-item (GAD-7) scale. Responses to the questionnaires were read by an ML model. Responses to the HRS-SR were then converted into anonymized, random-equivalent texts. Each of these individual texts was presented in random order to two experienced psychiatrists who were independently asked for a provisional diagnosis - e.g.; the presence or absence of HD. In case of disagreement between the two assessors, a third psychiatrist broke the tie. A decision tree classification model was employed to predict clinical HD using self-report data from two psychological tests, the HRS-SR and GAD-7. The target variable was whether a participant had clinical HD, while the predictive variables were the continuous scores from the HRS-SR and GAD-7 tests. The model's performance was evaluated using a confusion matrix, which compared the observed diagnoses with the predicted diagnoses to assess accuracy.According to the psychiatrists, approximately 10% of the participants fulfilled DSM-5 diagnostic criteria for HD. 93% of the clinician-identified cases were identified by the ML model based on HRS-SR and GAD-7 scores. A decision tree plot model demonstrated that about 60% of the cases could be detected by the HRS-SR alone while the rest required a combination of HRS-SR and GAD-7 scores. ML evaluation metrics showed satisfactory performance, with a Matthews Correlation Coefficient of 55%; Area Under Curve (AUC), 79%; a Negative Predictive Value of 76%; and a False Negative Rate of 24%.Study findings strongly suggest that ML can, in the future, play a significant role in the risk assessment of psychiatric disorders prior to face-to-face consultation. By using AI to scan big data questionnaire responses, wait time for seriously ill patients can be substantially cut, and prognoses substantially improved. |
|---|---|
| ISSN: | 1471244X |
| DOI: | 10.1186/s12888-024-06152-w |