Supporting the use of standardized nursing terminologies with automatic subject heading prediction: a comparison of sentence-level text classification methods
Abstract Objective This study focuses on the task of automatically assigning standardized (topical) subject headings to free-text sentences in clinical nursing notes. The underlying motivation is to support nurses when they document patient care by developing a computer system that can assist in inc...
Saved in:
| Published in: | Journal of the American Medical Informatics Association : JAMIA Vol. 27; no. 1; pp. 81 - 88 |
|---|---|
| Main Authors: | , , , , , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
England
Oxford University Press
01.01.2020
|
| Subjects: | |
| ISSN: | 1527-974X, 1527-974X |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | Abstract
Objective
This study focuses on the task of automatically assigning standardized (topical) subject headings to free-text sentences in clinical nursing notes. The underlying motivation is to support nurses when they document patient care by developing a computer system that can assist in incorporating suitable subject headings that reflect the documented topics. Central in this study is performance evaluation of several text classification methods to assess the feasibility of developing such a system.
Materials and Methods
Seven text classification methods are evaluated using a corpus of approximately 0.5 million nursing notes (5.5 million sentences) with 676 unique headings extracted from a Finnish university hospital. Several of these methods are based on artificial neural networks. Evaluation is first done in an automatic manner for all methods, then a manual error analysis is done on a sample.
Results
We find that a method based on a bidirectional long short-term memory network performs best with an average recall of 0.5435 when allowed to suggest 1 subject heading per sentence and 0.8954 when allowed to suggest 10 subject headings per sentence. However, other methods achieve comparable results. The manual analysis indicates that the predictions are better than what the automatic evaluation suggests.
Conclusions
The results indicate that several of the tested methods perform well in suggesting the most appropriate subject headings on sentence level. Thus, we find it feasible to develop a text classification system that can support the use of standardized terminologies and save nurses time and effort on care documentation. |
|---|---|
| AbstractList | This study focuses on the task of automatically assigning standardized (topical) subject headings to free-text sentences in clinical nursing notes. The underlying motivation is to support nurses when they document patient care by developing a computer system that can assist in incorporating suitable subject headings that reflect the documented topics. Central in this study is performance evaluation of several text classification methods to assess the feasibility of developing such a system.
Seven text classification methods are evaluated using a corpus of approximately 0.5 million nursing notes (5.5 million sentences) with 676 unique headings extracted from a Finnish university hospital. Several of these methods are based on artificial neural networks. Evaluation is first done in an automatic manner for all methods, then a manual error analysis is done on a sample.
We find that a method based on a bidirectional long short-term memory network performs best with an average recall of 0.5435 when allowed to suggest 1 subject heading per sentence and 0.8954 when allowed to suggest 10 subject headings per sentence. However, other methods achieve comparable results. The manual analysis indicates that the predictions are better than what the automatic evaluation suggests.
The results indicate that several of the tested methods perform well in suggesting the most appropriate subject headings on sentence level. Thus, we find it feasible to develop a text classification system that can support the use of standardized terminologies and save nurses time and effort on care documentation. Abstract Objective This study focuses on the task of automatically assigning standardized (topical) subject headings to free-text sentences in clinical nursing notes. The underlying motivation is to support nurses when they document patient care by developing a computer system that can assist in incorporating suitable subject headings that reflect the documented topics. Central in this study is performance evaluation of several text classification methods to assess the feasibility of developing such a system. Materials and Methods Seven text classification methods are evaluated using a corpus of approximately 0.5 million nursing notes (5.5 million sentences) with 676 unique headings extracted from a Finnish university hospital. Several of these methods are based on artificial neural networks. Evaluation is first done in an automatic manner for all methods, then a manual error analysis is done on a sample. Results We find that a method based on a bidirectional long short-term memory network performs best with an average recall of 0.5435 when allowed to suggest 1 subject heading per sentence and 0.8954 when allowed to suggest 10 subject headings per sentence. However, other methods achieve comparable results. The manual analysis indicates that the predictions are better than what the automatic evaluation suggests. Conclusions The results indicate that several of the tested methods perform well in suggesting the most appropriate subject headings on sentence level. Thus, we find it feasible to develop a text classification system that can support the use of standardized terminologies and save nurses time and effort on care documentation. This study focuses on the task of automatically assigning standardized (topical) subject headings to free-text sentences in clinical nursing notes. The underlying motivation is to support nurses when they document patient care by developing a computer system that can assist in incorporating suitable subject headings that reflect the documented topics. Central in this study is performance evaluation of several text classification methods to assess the feasibility of developing such a system.OBJECTIVEThis study focuses on the task of automatically assigning standardized (topical) subject headings to free-text sentences in clinical nursing notes. The underlying motivation is to support nurses when they document patient care by developing a computer system that can assist in incorporating suitable subject headings that reflect the documented topics. Central in this study is performance evaluation of several text classification methods to assess the feasibility of developing such a system.Seven text classification methods are evaluated using a corpus of approximately 0.5 million nursing notes (5.5 million sentences) with 676 unique headings extracted from a Finnish university hospital. Several of these methods are based on artificial neural networks. Evaluation is first done in an automatic manner for all methods, then a manual error analysis is done on a sample.MATERIALS AND METHODSSeven text classification methods are evaluated using a corpus of approximately 0.5 million nursing notes (5.5 million sentences) with 676 unique headings extracted from a Finnish university hospital. Several of these methods are based on artificial neural networks. Evaluation is first done in an automatic manner for all methods, then a manual error analysis is done on a sample.We find that a method based on a bidirectional long short-term memory network performs best with an average recall of 0.5435 when allowed to suggest 1 subject heading per sentence and 0.8954 when allowed to suggest 10 subject headings per sentence. However, other methods achieve comparable results. The manual analysis indicates that the predictions are better than what the automatic evaluation suggests.RESULTSWe find that a method based on a bidirectional long short-term memory network performs best with an average recall of 0.5435 when allowed to suggest 1 subject heading per sentence and 0.8954 when allowed to suggest 10 subject headings per sentence. However, other methods achieve comparable results. The manual analysis indicates that the predictions are better than what the automatic evaluation suggests.The results indicate that several of the tested methods perform well in suggesting the most appropriate subject headings on sentence level. Thus, we find it feasible to develop a text classification system that can support the use of standardized terminologies and save nurses time and effort on care documentation.CONCLUSIONSThe results indicate that several of the tested methods perform well in suggesting the most appropriate subject headings on sentence level. Thus, we find it feasible to develop a text classification system that can support the use of standardized terminologies and save nurses time and effort on care documentation. |
| Author | Suhonen, Henry Salakoski, Tapio Salanterä, Sanna Ginter, Filip Peltonen, Laura-Maria Moen, Hans Hakala, Kai |
| Author_xml | – sequence: 1 givenname: Hans orcidid: 0000-0003-1418-7892 surname: Moen fullname: Moen, Hans email: hans.moen@utu.fi organization: Department of Future Technologies, University of Turku, Turku, Finland – sequence: 2 givenname: Kai surname: Hakala fullname: Hakala, Kai organization: Department of Future Technologies, University of Turku, Turku, Finland – sequence: 3 givenname: Laura-Maria surname: Peltonen fullname: Peltonen, Laura-Maria organization: Department of Nursing Science, University of Turku, Turku, Finland – sequence: 4 givenname: Henry surname: Suhonen fullname: Suhonen, Henry organization: Department of Nursing Science, University of Turku, Turku, Finland – sequence: 5 givenname: Filip surname: Ginter fullname: Ginter, Filip organization: Department of Future Technologies, University of Turku, Turku, Finland – sequence: 6 givenname: Tapio surname: Salakoski fullname: Salakoski, Tapio organization: Department of Future Technologies, University of Turku, Turku, Finland – sequence: 7 givenname: Sanna surname: Salanterä fullname: Salanterä, Sanna organization: Department of Nursing Science, University of Turku, Turku, Finland |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/31605490$$D View this record in MEDLINE/PubMed |
| BookMark | eNpNkUtv1TAQRi3Uij5gxxp5Rzeh4zjOTdihipdUqQtAYhdN7HGvrxI7-AG0P4bfyn2UqqsZ6TvzLeacsSMfPDH2SsBbAb283ODs8DLoe6HgGTsVql5V_ar5cfRkP2FnKW0ARFtL9ZydSNGCano4ZX-_lmUJMTt_y_OaeEnEg-UpozcYjbsnw32JaZ9TnJ0PU7h1lPhvl9ccSw4zZqd5KuOGdOZrQrODl0jG6eyCf8eR6zAvGF0Kft9OPpPXVE30i6Zt75_M9YQpOes07m74THkdTHrBji1OiV4-zHP2_eOHb1efq-ubT1-u3l9XuhFtrkRL2nZ2pAZQjlZCp1UvO2WEIo2mRdCdNGMHZiUt1q0F6O0Wt02nrLAgz9nFoXeJ4WehlIfZJU3ThJ5CSUMtQUHdrzqxRV8_oGWcyQxLdDPGu-H_T7fAmwMQyvKYChh2toa9reFgS_4DSgiOwQ |
| CitedBy_id | crossref_primary_10_1002_nop2_880 crossref_primary_10_1016_j_ijnurstu_2021_104153 crossref_primary_10_1111_jonm_13425 crossref_primary_10_1007_s12553_021_00607_w crossref_primary_10_1186_s13326_020_00229_7 crossref_primary_10_1016_j_eswa_2023_123046 crossref_primary_10_3390_info13100499 crossref_primary_10_1186_s12911_022_01901_x |
| ContentType | Journal Article |
| Copyright | The Author(s) 2019. Published by Oxford University Press on behalf of the American Medical Informatics Association. 2019 The Author(s) 2019. Published by Oxford University Press on behalf of the American Medical Informatics Association. |
| Copyright_xml | – notice: The Author(s) 2019. Published by Oxford University Press on behalf of the American Medical Informatics Association. 2019 – notice: The Author(s) 2019. Published by Oxford University Press on behalf of the American Medical Informatics Association. |
| DBID | TOX CGR CUY CVF ECM EIF NPM 7X8 |
| DOI | 10.1093/jamia/ocz150 |
| DatabaseName | Oxford Journals Open Access Collection Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed MEDLINE - Academic |
| DatabaseTitle | MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) MEDLINE - Academic |
| DatabaseTitleList | MEDLINE MEDLINE - Academic |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: TOX name: Oxford Journals - Open Access url: https://academic.oup.com/journals/ sourceTypes: Publisher – sequence: 3 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Medicine |
| EISSN | 1527-974X |
| EndPage | 88 |
| ExternalDocumentID | 31605490 10.1093/jamia/ocz150 |
| Genre | Research Support, Non-U.S. Gov't Journal Article Comparative Study |
| GeographicLocations | Finland |
| GeographicLocations_xml | – name: Finland |
| GrantInformation_xml | – fundername: Academy of Finland grantid: 315376 funderid: 10.13039/501100002341 – fundername: Business Finland grantid: 644/31/2015 |
| GroupedDBID | --- --K .DC .GJ 0R~ 18M 1B1 1TH 29L 2WC 3V. 4.4 48X 53G 5GY 5RE 5WD 6PF 7RV 7X7 7~T 88E 88I 8AF 8AO 8FE 8FG 8FI 8FJ 8FW AABZA AACZT AAEDT AAJQQ AALRI AAMVS AAOGV AAPGJ AAPQZ AAPXW AARHZ AASNB AAUAY AAUQX AAVAP AAWDT AAWTL AAXUO ABEUO ABIXL ABJNI ABNHQ ABOCM ABPTD ABQLI ABQNK ABSAR ABSMQ ABUWG ABWST ABXVV ACFRR ACGFO ACGFS ACGOD ACHQT ACUFI ACUTJ ACYHN ACZBC ADBBV ADGZP ADHKW ADHZD ADIPN ADJOM ADJQC ADMUD ADQBN ADRIX ADRTK ADVEK ADYVW AEGPL AEJOX AEKSI AEMDU AENEX AENZO AEPUE AETBJ AEWNT AFFZL AFIYH AFKRA AFOFC AFXEN AFYAG AGINJ AGKRT AGMDO AGQXC AGSYK AGUTN AHMBA AJEEA ALIPV ALMA_UNASSIGNED_HOLDINGS ALUQC APIBT APJGH AQDSO AQKUS AQUVI ARAPS ATGXG AVNTJ AVWKF AXUDD AYCSE AZQEC BAWUL BAYMD BCRHZ BENPR BEYMZ BGLVJ BHONS BKEYQ BPHCQ BTRTY BVRKM BVXVI BZKNY C1A C45 CCPQU CDBKE CS3 DAKXR DIK DILTD DU5 DWQXO E3Z EBD EBS EIHJH EJD EMOBN ENERS EO8 EX3 F5P FDB FECEO FLUFQ FOEOM FOTVD FQBLK FYUFA G-Q GAUVT GJXCC GNUQQ GX1 H13 HAR HCIFZ HMCUK IH2 IHE J21 K6V K7- KBUDW KOP KSI KSN LSO M0N M0T M1P M2P M2Q M41 MBLQV MHKGH NAPCQ NOMLY NOYVH NQ- NVLIB O9- OAUYM OAWHX OCZFY ODMLO OJQWA OJZSN OK1 OPAEJ OVD OWPYF P2P P62 PAFKI PCD PEELM PQQKQ PROAC PSQYO Q5Y R53 RIG ROL ROX ROZ RPM RPZ RUSNO RWL RXO S0X SSZ SV3 TAE TEORI TJX TMA TOX UKHRP WOQ WOW YAYTL YHZ YKOAZ YXANX ZGI ~S- ABDFA ABEJV ABGNP ABPQP ABVGC ADNBA AEMQT AFXAL AHMMS AJNCP ALXQX CGR CUY CVF ECM EIF JXSIZ NPM 77I 7X8 AHGBF AJBYB |
| ID | FETCH-LOGICAL-c416t-16ecf8fbe40a3bf308c59385d15ecad6a0c83db80d73fa26f009ffbef485f1f03 |
| IEDL.DBID | TOX |
| ISICitedReferencesCount | 10 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000548300200011&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1527-974X |
| IngestDate | Sun Nov 09 12:23:42 EST 2025 Thu Apr 03 07:08:03 EDT 2025 Wed Aug 28 03:18:50 EDT 2024 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Keywords | electronic health records text classification natural language processing nursing documentation clinical decision support |
| Language | English |
| License | This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact journals.permissions@oup.com The Author(s) 2019. Published by Oxford University Press on behalf of the American Medical Informatics Association. |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c416t-16ecf8fbe40a3bf308c59385d15ecad6a0c83db80d73fa26f009ffbef485f1f03 |
| Notes | ObjectType-Article-2 SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 23 |
| ORCID | 0000-0003-1418-7892 |
| OpenAccessLink | https://dx.doi.org/10.1093/jamia/ocz150 |
| PMID | 31605490 |
| PQID | 2305029781 |
| PQPubID | 23479 |
| PageCount | 8 |
| ParticipantIDs | proquest_miscellaneous_2305029781 pubmed_primary_31605490 oup_primary_10_1093_jamia_ocz150 |
| PublicationCentury | 2000 |
| PublicationDate | 2020-01-01 |
| PublicationDateYYYYMMDD | 2020-01-01 |
| PublicationDate_xml | – month: 01 year: 2020 text: 2020-01-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | England |
| PublicationPlace_xml | – name: England |
| PublicationTitle | Journal of the American Medical Informatics Association : JAMIA |
| PublicationTitleAlternate | J Am Med Inform Assoc |
| PublicationYear | 2020 |
| Publisher | Oxford University Press |
| Publisher_xml | – name: Oxford University Press |
| SSID | ssj0016235 |
| Score | 2.3652072 |
| Snippet | Abstract
Objective
This study focuses on the task of automatically assigning standardized (topical) subject headings to free-text sentences in clinical nursing... This study focuses on the task of automatically assigning standardized (topical) subject headings to free-text sentences in clinical nursing notes. The... |
| SourceID | proquest pubmed oup |
| SourceType | Aggregation Database Index Database Publisher |
| StartPage | 81 |
| SubjectTerms | Abstracting and Indexing - methods Electronic Health Records Finland Natural Language Processing Nursing Records Standardized Nursing Terminology Subject Headings |
| Title | Supporting the use of standardized nursing terminologies with automatic subject heading prediction: a comparison of sentence-level text classification methods |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/31605490 https://www.proquest.com/docview/2305029781 |
| Volume | 27 |
| WOSCitedRecordID | wos000548300200011&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LT9wwELYKqioufUO3DzSVeo2a4DiZ9FZVIA6w5UClvUV-lpXQZrXZcODH8FuZscMeAAkuOTnjyDOKv5lvHkL8KE3tPWKRFQplVppKZQ0X-xC60E65GnMfC4VP6ukUZ7PmbGyS1D9C4TeSGwTN9c_OXiffnIWS3Z7_nW3YArrC1ZjUfv-Fe3VrD-BjvEaO3jz7A96K1yNShN9Jte_EC794L16djlz4B3HDAzk7bgLwHwjFwdB76ALcxQbm197BSOxDSnmJ_znfA8deQQ_rLrZrhX4wHIyBi5RPD8sV78EK-wUa7GZQYZTOLTzJTrJLTjYCThsBywCcM46ikiHNpO4_in9Hh-d_jrNx2kJmCZSts6LyNmAwvsy1NEHmaFUjUblCeatdpXOL0hnMXS2DPqgCobNAy0OJKhQhl7tie9Et_CcBulJOK3RG0eUXCjQNEqyjRaZUSHhsIoCU0i5TP4028eCyjefcpnOeiO93GmvJ4JnF0AvfDX1LPpPiiVtYTMReUuVGkizIOyub_PPTG3wROwfsOsdoylexvV4N_pt4aa_W8361L7bqGdJzena6H23tFv5l1wQ |
| linkProvider | Oxford University Press |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Supporting+the+use+of+standardized+nursing+terminologies+with+automatic+subject+heading+prediction%3A+a+comparison+of+sentence-level+text+classification+methods&rft.jtitle=Journal+of+the+American+Medical+Informatics+Association+%3A+JAMIA&rft.au=Moen%2C+Hans&rft.au=Hakala%2C+Kai&rft.au=Peltonen%2C+Laura-Maria&rft.au=Suhonen%2C+Henry&rft.date=2020-01-01&rft.pub=Oxford+University+Press&rft.eissn=1527-974X&rft.volume=27&rft.issue=1&rft.spage=81&rft.epage=88&rft_id=info:doi/10.1093%2Fjamia%2Focz150&rft.externalDocID=10.1093%2Fjamia%2Focz150 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1527-974X&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1527-974X&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1527-974X&client=summon |