Evaluating the generalizability of deep learning image classification algorithms to detect middle ear disease using otoscopy

To evaluate the generalizability of artificial intelligence (AI) algorithms that use deep learning methods to identify middle ear disease from otoscopic images, between internal to external performance. 1842 otoscopic images were collected from three independent sources: (a) Van, Turkey, (b) Santiag...

Full description

Saved in:

Bibliographic Details
Published in:	Scientific reports Vol. 13; no. 1; pp. 5368 - 9
Main Authors:	Habib, Al-Rahim, Xu, Yixi, Bock, Kris, Mohanty, Shrestha, Sederholm, Tina, Weeks, William B., Dodhia, Rahul, Ferres, Juan Lavista, Perry, Chris, Sacks, Raymond, Singh, Narinder
Format:	Journal Article
Language:	English
Published:	London Nature Publishing Group UK 01.04.2023 Nature Publishing Group Nature Portfolio
Subjects:	692/308/575 692/700/139 692/700/478 Algorithms Artificial Intelligence Deep Learning Ear diseases Ear Diseases - diagnostic imaging Humanities and Social Sciences Humans Middle ear multidisciplinary Otoscopy - methods Science Science (multidisciplinary) Teaching methods
ISSN:	2045-2322, 2045-2322
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	To evaluate the generalizability of artificial intelligence (AI) algorithms that use deep learning methods to identify middle ear disease from otoscopic images, between internal to external performance. 1842 otoscopic images were collected from three independent sources: (a) Van, Turkey, (b) Santiago, Chile, and (c) Ohio, USA. Diagnostic categories consisted of (i) normal or (ii) abnormal. Deep learning methods were used to develop models to evaluate internal and external performance, using area under the curve (AUC) estimates. A pooled assessment was performed by combining all cohorts together with fivefold cross validation. AI-otoscopy algorithms achieved high internal performance (mean AUC: 0.95, 95%CI: 0.80–1.00). However, performance was reduced when tested on external otoscopic images not used for training (mean AUC: 0.76, 95%CI: 0.61–0.91). Overall, external performance was significantly lower than internal performance (mean difference in AUC: −0.19, p ≤ 0.04). Combining cohorts achieved a substantial pooled performance (AUC: 0.96, standard error: 0.01). Internally applied algorithms for otoscopy performed well to identify middle ear disease from otoscopy images. However, external performance was reduced when applied to new test cohorts. Further efforts are required to explore data augmentation and pre-processing techniques that might improve external performance and develop a robust, generalizable algorithm for real-world clinical applications.
AbstractList	Abstract To evaluate the generalizability of artificial intelligence (AI) algorithms that use deep learning methods to identify middle ear disease from otoscopic images, between internal to external performance. 1842 otoscopic images were collected from three independent sources: (a) Van, Turkey, (b) Santiago, Chile, and (c) Ohio, USA. Diagnostic categories consisted of (i) normal or (ii) abnormal. Deep learning methods were used to develop models to evaluate internal and external performance, using area under the curve (AUC) estimates. A pooled assessment was performed by combining all cohorts together with fivefold cross validation. AI-otoscopy algorithms achieved high internal performance (mean AUC: 0.95, 95%CI: 0.80–1.00). However, performance was reduced when tested on external otoscopic images not used for training (mean AUC: 0.76, 95%CI: 0.61–0.91). Overall, external performance was significantly lower than internal performance (mean difference in AUC: −0.19, p ≤ 0.04). Combining cohorts achieved a substantial pooled performance (AUC: 0.96, standard error: 0.01). Internally applied algorithms for otoscopy performed well to identify middle ear disease from otoscopy images. However, external performance was reduced when applied to new test cohorts. Further efforts are required to explore data augmentation and pre-processing techniques that might improve external performance and develop a robust, generalizable algorithm for real-world clinical applications. To evaluate the generalizability of artificial intelligence (AI) algorithms that use deep learning methods to identify middle ear disease from otoscopic images, between internal to external performance. 1842 otoscopic images were collected from three independent sources: (a) Van, Turkey, (b) Santiago, Chile, and (c) Ohio, USA. Diagnostic categories consisted of (i) normal or (ii) abnormal. Deep learning methods were used to develop models to evaluate internal and external performance, using area under the curve (AUC) estimates. A pooled assessment was performed by combining all cohorts together with fivefold cross validation. AI-otoscopy algorithms achieved high internal performance (mean AUC: 0.95, 95%CI: 0.80–1.00). However, performance was reduced when tested on external otoscopic images not used for training (mean AUC: 0.76, 95%CI: 0.61–0.91). Overall, external performance was significantly lower than internal performance (mean difference in AUC: −0.19, p ≤ 0.04). Combining cohorts achieved a substantial pooled performance (AUC: 0.96, standard error: 0.01). Internally applied algorithms for otoscopy performed well to identify middle ear disease from otoscopy images. However, external performance was reduced when applied to new test cohorts. Further efforts are required to explore data augmentation and pre-processing techniques that might improve external performance and develop a robust, generalizable algorithm for real-world clinical applications. To evaluate the generalizability of artificial intelligence (AI) algorithms that use deep learning methods to identify middle ear disease from otoscopic images, between internal to external performance. 1842 otoscopic images were collected from three independent sources: (a) Van, Turkey, (b) Santiago, Chile, and (c) Ohio, USA. Diagnostic categories consisted of (i) normal or (ii) abnormal. Deep learning methods were used to develop models to evaluate internal and external performance, using area under the curve (AUC) estimates. A pooled assessment was performed by combining all cohorts together with fivefold cross validation. AI-otoscopy algorithms achieved high internal performance (mean AUC: 0.95, 95%CI: 0.80-1.00). However, performance was reduced when tested on external otoscopic images not used for training (mean AUC: 0.76, 95%CI: 0.61-0.91). Overall, external performance was significantly lower than internal performance (mean difference in AUC: -0.19, p ≤ 0.04). Combining cohorts achieved a substantial pooled performance (AUC: 0.96, standard error: 0.01). Internally applied algorithms for otoscopy performed well to identify middle ear disease from otoscopy images. However, external performance was reduced when applied to new test cohorts. Further efforts are required to explore data augmentation and pre-processing techniques that might improve external performance and develop a robust, generalizable algorithm for real-world clinical applications.To evaluate the generalizability of artificial intelligence (AI) algorithms that use deep learning methods to identify middle ear disease from otoscopic images, between internal to external performance. 1842 otoscopic images were collected from three independent sources: (a) Van, Turkey, (b) Santiago, Chile, and (c) Ohio, USA. Diagnostic categories consisted of (i) normal or (ii) abnormal. Deep learning methods were used to develop models to evaluate internal and external performance, using area under the curve (AUC) estimates. A pooled assessment was performed by combining all cohorts together with fivefold cross validation. AI-otoscopy algorithms achieved high internal performance (mean AUC: 0.95, 95%CI: 0.80-1.00). However, performance was reduced when tested on external otoscopic images not used for training (mean AUC: 0.76, 95%CI: 0.61-0.91). Overall, external performance was significantly lower than internal performance (mean difference in AUC: -0.19, p ≤ 0.04). Combining cohorts achieved a substantial pooled performance (AUC: 0.96, standard error: 0.01). Internally applied algorithms for otoscopy performed well to identify middle ear disease from otoscopy images. However, external performance was reduced when applied to new test cohorts. Further efforts are required to explore data augmentation and pre-processing techniques that might improve external performance and develop a robust, generalizable algorithm for real-world clinical applications. To evaluate the generalizability of artificial intelligence (AI) algorithms that use deep learning methods to identify middle ear disease from otoscopic images, between internal to external performance. 1842 otoscopic images were collected from three independent sources: (a) Van, Turkey, (b) Santiago, Chile, and (c) Ohio, USA. Diagnostic categories consisted of (i) normal or (ii) abnormal. Deep learning methods were used to develop models to evaluate internal and external performance, using area under the curve (AUC) estimates. A pooled assessment was performed by combining all cohorts together with fivefold cross validation. AI-otoscopy algorithms achieved high internal performance (mean AUC: 0.95, 95%CI: 0.80-1.00). However, performance was reduced when tested on external otoscopic images not used for training (mean AUC: 0.76, 95%CI: 0.61-0.91). Overall, external performance was significantly lower than internal performance (mean difference in AUC: -0.19, p ≤ 0.04). Combining cohorts achieved a substantial pooled performance (AUC: 0.96, standard error: 0.01). Internally applied algorithms for otoscopy performed well to identify middle ear disease from otoscopy images. However, external performance was reduced when applied to new test cohorts. Further efforts are required to explore data augmentation and pre-processing techniques that might improve external performance and develop a robust, generalizable algorithm for real-world clinical applications.
ArticleNumber	5368
Author	Xu, Yixi Mohanty, Shrestha Ferres, Juan Lavista Habib, Al-Rahim Dodhia, Rahul Perry, Chris Singh, Narinder Sederholm, Tina Bock, Kris Weeks, William B. Sacks, Raymond
Author_xml	– sequence: 1 givenname: Al-Rahim surname: Habib fullname: Habib, Al-Rahim email: al-rahim.habib@sydney.edu.au organization: Faculty of Medicine and Health, University of Sydney, Department of Otolaryngology, Head and Neck Surgery, Westmead Hospital – sequence: 2 givenname: Yixi surname: Xu fullname: Xu, Yixi organization: AI for Good Lab, Microsoft – sequence: 3 givenname: Kris surname: Bock fullname: Bock, Kris organization: Azure FastTrack Engineering – sequence: 4 givenname: Shrestha surname: Mohanty fullname: Mohanty, Shrestha organization: Microsoft – sequence: 5 givenname: Tina surname: Sederholm fullname: Sederholm, Tina organization: AI for Good Lab, Microsoft – sequence: 6 givenname: William B. surname: Weeks fullname: Weeks, William B. organization: AI for Good Lab, Microsoft – sequence: 7 givenname: Rahul surname: Dodhia fullname: Dodhia, Rahul organization: AI for Good Lab, Microsoft – sequence: 8 givenname: Juan Lavista surname: Ferres fullname: Ferres, Juan Lavista organization: AI for Good Lab, Microsoft – sequence: 9 givenname: Chris surname: Perry fullname: Perry, Chris organization: University of Queensland Medical School – sequence: 10 givenname: Raymond surname: Sacks fullname: Sacks, Raymond organization: Faculty of Medicine and Health, University of Sydney – sequence: 11 givenname: Narinder surname: Singh fullname: Singh, Narinder organization: Faculty of Medicine and Health, University of Sydney, Department of Otolaryngology, Head and Neck Surgery, Westmead Hospital
BackLink	https://www.ncbi.nlm.nih.gov/pubmed/37005441$$D View this record in MEDLINE/PubMed
BookMark	eNp9Uk1vEzEQXaEiWkr_AAdkiQuXBX9tdn1CqCpQqRIXOFsTe3bjyLGD7a3Uih-Pk5TS9lBfPON573m-XjdHIQZsmreMfmRUDJ-yZJ0aWspFK5jirKUvmhNOZddywfnRA_u4Oct5TevpuJJMvWqORV8dKdlJ8-fiGvwMxYWJlBWSCQMm8O4Wls67ckPiSCzilniEFHYot4EJifGQsxudqdQYCPgpJldWm0xKrISCppCNs9YjqURiXUbISOa8k4glZhO3N2-alyP4jGd392nz6-vFz_Pv7dWPb5fnX65a00lW2rETqBSvDlBmoRqMdsYqIUbWq47LGgfDGF8qC6YXjCowYhiBDlbxhRKnzeVB10ZY622qJaQbHcHp_UNMk4ZUnPGoWT-g6kBaQC7BCtUxi71cjqZ2jvW8an0-aG3n5QatwVBqvx6JPo4Et9JTvNaM0kU_sL4qfLhTSPH3jLnojcsGvYeAcc6a90ouhkWtokLfP4Gu45xC7dUOJQbZs3157x6mdJ_LvylXAD8ATIo5JxzvIYzq3Tbpwzbpuk16v02aVtLwhGRc2U-7luX881RxoOb6T5gw_U_7GdZf5nDftg
CitedBy_id	crossref_primary_10_1002_wjo2_70019 crossref_primary_10_1007_s11042_024_18631_z crossref_primary_10_1016_j_jag_2023_103569 crossref_primary_10_1007_s12070_024_04885_4 crossref_primary_10_1109_ACCESS_2025_3597769 crossref_primary_10_1007_s00521_025_10990_4 crossref_primary_10_1016_j_compbiomed_2025_110092 crossref_primary_10_1109_ACCESS_2024_3428700 crossref_primary_10_1177_00368504251365411 crossref_primary_10_3390_jcm12226973 crossref_primary_10_1007_s12285_024_00507_8
Cites_doi	10.1117/12.2582009.full 10.1016/j.ebiom.2016.02.017 10.1136/bmjopen-2016-012799 10.1101/2021.08.05.21261672 10.1017/S0022215118001275 10.3390/diagnostics12061318 10.3390/jcm10153198 10.1017/S0022215120000717 10.1016/j.ijporl.2004.10.013 10.1542/peds.2020-034546 10.1117/12.2581902.full 10.3389/fdgth.2021.810427 10.1186/s12992-019-0531-5 10.1016/j.neunet.2020.03.023 10.1007/s10462-021-10121-0 10.1109/IDAP.2019.8875973 10.1371/journal.pone.0229226 10.1101/515007v2.full 10.1038/s41591-020-1034-x 10.1109/UBMK.2019.8907070 10.1371/journal.pone.0232776 10.1111/j.1365-2273.1995.tb00014.x 10.1097/MAO.0000000000003484 10.1016/j.irbm.2021.01.001 10.7717/peerj-cs.405 10.1111/coa.13925 10.21205/deufmd.2020226625 10.2196/33049 10.1016/j.ajem.2021.04.030 10.1002/lary.29253 10.1016/j.bbe.2019.11.001 10.1007/s00521-021-06810-0 10.1093/ckj/sfaa188 10.1016/S0031-3203(96)00142-2 10.1109/TKDE.2005.50 10.1016/j.bspc.2017.07.015 10.1177/1357633X20987783 10.1136/bmjgh-2018-000798 10.2196/25759
ContentType	Journal Article
Copyright	The Author(s) 2023 2023. The Author(s). The Author(s) 2023. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml	– notice: The Author(s) 2023 – notice: 2023. The Author(s). – notice: The Author(s) 2023. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID	C6C AAYXX CITATION CGR CUY CVF ECM EIF NPM 3V. 7X7 7XB 88A 88E 88I 8FE 8FH 8FI 8FJ 8FK ABUWG AEUYN AFKRA AZQEC BBNVY BENPR BHPHI CCPQU DWQXO FYUFA GHDGH GNUQQ HCIFZ K9. LK8 M0S M1P M2P M7P PHGZM PHGZT PIMPY PJZUB PKEHL PPXIY PQEST PQGLB PQQKQ PQUKI PRINS Q9U 7X8 5PM DOA
DOI	10.1038/s41598-023-31921-0
DatabaseName	Springer Nature OA Free Journals CrossRef Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed ProQuest Central (Corporate) Health & Medical Collection ProQuest Central (purchase pre-March 2016) Biology Database (Alumni Edition) Medical Database (Alumni Edition) Science Database (Alumni Edition) ProQuest SciTech Collection ProQuest Natural Science Collection Hospital Premium Collection Hospital Premium Collection (Alumni Edition) ProQuest Central (Alumni) (purchase pre-March 2016) ProQuest Central (Alumni) ProQuest One Sustainability ProQuest Central UK/Ireland ProQuest Central Essentials ProQuest : Biological Science Collection journals [unlimited simultaneous users] ProQuest Central (subscription) Natural Science Collection ProQuest One Community College ProQuest Central Health Research Premium Collection Health Research Premium Collection (Alumni) ProQuest Central Student SciTech Premium Collection ProQuest Health & Medical Complete (Alumni) Biological Sciences Health & Medical Collection (Alumni Edition) PML(ProQuest Medical Library) Science Database ProQuest Biological Science ProQuest Central Premium ProQuest One Academic Publicly Available Content Database ProQuest Health & Medical Research Collection ProQuest One Academic Middle East (New) ProQuest One Health & Nursing ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China ProQuest Central Basic MEDLINE - Academic PubMed Central (Full Participant titles) DOAJ Directory of Open Access Journals
DatabaseTitle	CrossRef MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) Publicly Available Content Database ProQuest Central Student ProQuest One Academic Middle East (New) ProQuest Central Essentials ProQuest Health & Medical Complete (Alumni) ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest One Health & Nursing ProQuest Natural Science Collection ProQuest Central China ProQuest Biology Journals (Alumni Edition) ProQuest Central ProQuest One Applied & Life Sciences ProQuest One Sustainability ProQuest Health & Medical Research Collection Health Research Premium Collection Health and Medicine Complete (Alumni Edition) Natural Science Collection ProQuest Central Korea Health & Medical Research Collection Biological Science Collection ProQuest Central (New) ProQuest Medical Library (Alumni) ProQuest Science Journals (Alumni Edition) ProQuest Biological Science Collection ProQuest Central Basic ProQuest Science Journals ProQuest One Academic Eastern Edition ProQuest Hospital Collection Health Research Premium Collection (Alumni) Biological Science Database ProQuest SciTech Collection ProQuest Hospital Collection (Alumni) ProQuest Health & Medical Complete ProQuest Medical Library ProQuest One Academic UKI Edition ProQuest One Academic ProQuest One Academic (New) ProQuest Central (Alumni) MEDLINE - Academic
DatabaseTitleList	CrossRef MEDLINE - Academic MEDLINE Publicly Available Content Database
Database_xml	– sequence: 1 dbid: DOA name: DOAJ Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website – sequence: 2 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 3 dbid: PIMPY name: Publicly Available Content Database url: http://search.proquest.com/publiccontent sourceTypes: Aggregation Database
DeliveryMethod	fulltext_linktorsrc
Discipline	Biology
EISSN	2045-2322
EndPage	9
ExternalDocumentID	oai_doaj_org_article_178e95a4dae24ad3951de74bfc419172 PMC10067817 37005441 10_1038_s41598_023_31921_0
Genre	Research Support, Non-U.S. Gov't Journal Article
GrantInformation_xml	– fundername: Microsoft AI for Humanitarian Action Fund Grant – fundername: Avant Foundation funderid: http://dx.doi.org/10.13039/501100020021 – fundername: Garnett Passe and Rodney Williams Memorial Foundation funderid: http://dx.doi.org/10.13039/501100003354 – fundername: ;
GroupedDBID	0R~ 3V. 4.4 53G 5VS 7X7 88A 88E 88I 8FE 8FH 8FI 8FJ AAFWJ AAJSJ AAKDD ABDBF ABUWG ACGFS ACSMW ACUHS ADBBV ADRAZ AENEX AEUYN AFKRA AJTQC ALIPV ALMA_UNASSIGNED_HOLDINGS AOIJS AZQEC BAWUL BBNVY BCNDV BENPR BHPHI BPHCQ BVXVI C6C CCPQU DIK DWQXO EBD EBLON EBS ESX FYUFA GNUQQ GROUPED_DOAJ GX1 HCIFZ HH5 HMCUK HYE KQ8 LK8 M0L M1P M2P M48 M7P M~E NAO OK1 PIMPY PQQKQ PROAC PSQYO RNT RNTTT RPM SNYQT UKHRP AASML AAYXX AFFHD AFPKN CITATION PHGZM PHGZT PJZUB PPXIY PQGLB CGR CUY CVF ECM EIF NPM 7XB 8FK K9. PKEHL PQEST PQUKI PRINS Q9U 7X8 PUEGO 5PM
ID	FETCH-LOGICAL-c541t-f53e992c54a01da2c5105cd933f17952453eac112b9dac73109ac38fa08d92693
IEDL.DBID	DOA
ISICitedReferencesCount	12
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000984099000040&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	2045-2322
IngestDate	Fri Oct 03 12:50:45 EDT 2025 Tue Nov 04 02:07:07 EST 2025 Thu Sep 04 18:27:55 EDT 2025 Tue Oct 07 07:56:36 EDT 2025 Thu Jan 02 22:52:15 EST 2025 Sat Nov 29 06:33:49 EST 2025 Tue Nov 18 21:34:50 EST 2025 Fri Feb 21 02:39:56 EST 2025
IsDoiOpenAccess	true
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	1
Language	English
License	2023. The Author(s). Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c541t-f53e992c54a01da2c5105cd933f17952453eac112b9dac73109ac38fa08d92693
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 content type line 23
OpenAccessLink	https://doaj.org/article/178e95a4dae24ad3951de74bfc419172
PMID	37005441
PQID	2793847169
PQPubID	2041939
PageCount	9
ParticipantIDs	doaj_primary_oai_doaj_org_article_178e95a4dae24ad3951de74bfc419172 pubmedcentral_primary_oai_pubmedcentral_nih_gov_10067817 proquest_miscellaneous_2794686109 proquest_journals_2793847169 pubmed_primary_37005441 crossref_primary_10_1038_s41598_023_31921_0 crossref_citationtrail_10_1038_s41598_023_31921_0 springer_journals_10_1038_s41598_023_31921_0
PublicationCentury	2000
PublicationDate	2023-04-01
PublicationDateYYYYMMDD	2023-04-01
PublicationDate_xml	– month: 04 year: 2023 text: 2023-04-01 day: 01
PublicationDecade	2020
PublicationPlace	London
PublicationPlace_xml	– name: London – name: England
PublicationTitle	Scientific reports
PublicationTitleAbbrev	Sci Rep
PublicationTitleAlternate	Sci Rep
PublicationYear	2023
Publisher	Nature Publishing Group UK Nature Publishing Group Nature Portfolio
Publisher_xml	– name: Nature Publishing Group UK – name: Nature Publishing Group – name: Nature Portfolio
References	ZhangJLiCYinYZhangJGrzegorzekMApplications of artificial neural networks in microorganism image analysis: A comprehensive review from conventional multilayer perceptron to popular convolutional neural network and potential visual transformerArtif. Intell. Rev.20221158 WahlBCossy-GantnerAGermannSSchwalbeNRArtificial intelligence (AI) and global health: how can AI contribute to health in resource-poor settings?BMJ Glob Health (Internet)201834e00079810.1136/bmjgh-2018-000798 GraydonKWaterworthCMillerHGunasekeraHGlobal burden of hearing impairment and ear diseaseJ. Laryngol. Otol.2019133118251:STN:280:DC%2BB3c7itV2jtA%3D%3D10.1017/S002221511800127530047343 LivingstoneDTalaiASChauJForkertNDBuilding an otoscopic screening prototype tool using deep learningJ. Otolaryngol. Head Neck Surg.2019486615 ViscainoMMaassJCDelanoPHTorrenteMStottCAuatCheeinFComputer-aided diagnosis of external and middle ear conditions: A machine learning approachPLoS One [Internet].20201531181:CAS:528:DC%2BB3cXmtlCqtbo%3D10.1371/journal.pone.0229226 HabibARKajbafzadehMHasanZWongEGunasekeraHPerryCArtificial intelligence to classify ear disease from otoscopy: A systematic review and meta-analysisClin. Otolaryngol.202247340141310.1111/coa.13925352533789310803 LiuXCruz RiveraSMoherDCalvertMDennistonAKSpirit-aiTCONSORT-AI extensionNat Med.202026136413741:CAS:528:DC%2BB3cXhvVyqu7%2FO10.1038/s41591-020-1034-x329082837598943 Shield, B. Evaluation of the Social and Economic Costs of Hearing Impairment: A Report for Hear-It [Internet]. https://www.hear-it.org/sites/default/files/multimedia/documents/Hear_It_Report_October_2006.pdf. Accessed 28 Aug 2022 (2006). BasaranEComertZCelikYConvolutional neural network approach for automatic tympanic membrane detection and classificationBiomed. Signal Process Control2020561114 ChenHLiCLiXRahamanMMHuWLiYIL-MCAM: An interactive learning and multi-channel attention mechanism-based weakly supervised colorectal histopathology image classification approachComput. Biol. Med.202211431:CAS:528:DC%2BB38XisFKntrzE KhanMAKwonSChooJHongSMKangSHParkIHAutomatic detection of tympanic membrane and middle ear infection from oto-endoscopic images via convolutional neural networksNeural Netw. [Internet].202012638439410.1016/j.neunet.2020.03.02332311656 AleneziEMAJajkoKReidALocatelli-SmithATaoKFMBrightTThe reliability of video otoscopy recordings and still images in the asynchronous diagnosis of middle-ear diseaseInt. J. Audiol.202161921 KleinmanKPsoterKJNyhanASolomonBSKimJMCanaresTEvaluation of digital otoscopy in pediatric patients: A prospective randomized controlled clinical trialAm. J. Emerg. Med.202114615015510.1016/j.ajem.2021.04.030 HabibARWongESacksRSinghNArtificial intelligence to detect tympanic membrane perforationsJ. Laryngol. Otol.2020134431131510.1017/S002221512000071732238202 Wang, W., Tamhane, A., Santos, C., Rzasa, J.R., Clark, J.H., Canares, T.L. et al. Pediatric Otoscopy Video Screening with Shift Contrastive Anomaly Detection. http://arxiv.org/abs/2110.13254. Accessed 25 Oct 2021 (2021). AleneziEMAJajkoKReidALocatelli-SmithAMcMahenCSETaoKFMClinician-rated quality of video otoscopy recordings and still images for the asynchronous assessment of middle-ear diseaseJ. Telemed. Telecare2021261357633X20987783 Simonyan, K., & Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. http://arxiv.org/abs/1409.1556. Accessed 4 Sep 2014 (2014). BinolHNiaziMKKEssigGShahJMattinglyJKHarrisMSDigital otoscopy videos versus composite images: A reader study to compare the accuracy of ENT physiciansLaryngoscope20211315E1668E167610.1002/lary.2925333170529 LeeJYChoiSHChungJWAutomated classification of the tympanic membrane using a convolutional neural networkAppl. Sci. (Switzerland).2019991827 HuangJLingCXUsing AUC and accuracy in evaluating learning algorithmsIEEE Trans. Knowl. Data Eng.20051732993101:CAS:528:DC%2BD2MXjslCgurg%3D10.1109/TKDE.2005.50 Chen, H., Li, C., Wang, G., Li, X., Rahaman, M., Sun, H. et al. GasHis-Transformer: A Multi-scale Visual Transformer Approach for Gastric Histopathological Image Detection. http://arxiv.org/abs/2104.14528. Accessed 29 Apr 2021 (2021). PichicheroMEPooleMDComparison of performance by otolaryngologists, pediatricians, and general practioners on an otoendoscopic diagnostic video examinationInt. J. Pediatr. Otorhinolaryngol.200569336136610.1016/j.ijporl.2004.10.01315733595 CohenJFKorevaarDAAltmanDGBrunsDEGatsonisCAHooftLSTARD 2015 guidelines for reporting diagnostic accuracy studies: Explanation and elaborationBMJ Open201661111710.1136/bmjopen-2016-012799 CamalanSNiaziMKKMoberlyACTeknosTEssigGElmaraghyCOtoMatch: Content-based eardrum image retrieval using deep learningPLoS One [Internet].20201551161:CAS:528:DC%2BB3cXpslWltrs%3D10.1371/journal.pone.0232776 SundgaardJVHarteJBrayPLaugesenSKamideYTanakaCDeep metric learning for otitis media classificationMed. Image Anal.2021171 BinolHMoberlyACNiaziMKKEssigGShahJElmaraghyCSelectStitch: Automated frame segmentation and stitching to create composite images from otoscope video clipsAppl. Sci. (Switzerland)20201017113 Binol, H., Niazi, M.K.K., Elmaraghy, C., Moberly, A.C., & Gurcan, M.N. Automated video summarization and label assignment for otoscopy videos using deep learning and natural language processing. in Medical Imaging 2021: Imaging Informatics for Healthcare, Research, and Applications [Internet] (Park, B.J., Deserno, T.M. Eds.). 28. https://www.spiedigitallibrary.org/conference-proceedings-of-spie/11601/2582009/Automated-video-summarization-and-label-assignment-for-otoscopy-videos-using/https://doi.org/10.1117/12.2582009.full. Accessed 17 Mar 2021 (SPIE, 2021). AlhudhaifACömertZPolatKOtitis media detection using tympanic membrane images with a novel multi-class machine learning algorithmPeerJ Comput. Sci.2021710.7717/peerj-cs.405338170487959604 Huang, G., Liu, Z., van der Maaten, L., & Weinberger, K.Q. Densely Connected Convolutional Networks. http://arxiv.org/abs/1608.06993. Accessed 24 Aug 2016 (2016). FutomaJSimonsMPanchTDoshi-VelezFCeliLAThe myth of generalisability in clinical research and machine learning in health careLancet202012e489e492 World Health Organization. Childhood Hearing Loss—Act Now, Here’s How! [Internet]. Geneva; 2016. https://apps.who.int/iris/handle/10665/204507. Accessed 28 Aug 2022 (2016). CrowsonMGHartnickCJDiercksGRGallagherTQFracchiaMSSetlurJMachine learning for accurate intraoperative pediatric middle ear effusion diagnosisPediatrics2021147410.1542/peds.2020-03454633731369 BradleyAPThe use of the area under the ROC curve in the evaluation of machine learning algorithmsPatter Recognit.1997307114511591997PatRe..30.1145B10.1016/S0031-3203(96)00142-2 He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. http://arxiv.org/abs/1512.03385. Accessed 10 Dec 2015 (2015). WormaldPJBrowningGGRobinsonKIs otoscopy reliable? A structured teaching method to improve otoscopic accuracy in traineesClin. Otolaryngol. Allied Sci.199520163671:STN:280:DyaK2MzgvVOktw%3D%3D10.1111/j.1365-2273.1995.tb00014.x7788938 LivingstoneDChauJOtoscopic diagnosis using computer vision: An automated machine learning approachLaryngoscope20191316 SandströmJMyburghHLaurentCSwanepoelDWLundbergTA machine learning approach to screen for otitis media using digital otoscope images labelled by an expert panelDiagnostics2022126131810.3390/diagnostics12061318357411289222011 YinJNgiamKYTeoHHRole of artificial intelligence applications in real-life clinical practice: Systematic reviewJ. Med. Internet Res. (JMIR Publications Inc.)2021232575910.2196/25759 Seok, J., Song, J.J., Koo, J.W., Kin, H.C., & Choi, B.Y. The semantic segmentation approach for normal and pathologic tympanic membrane using deep learning (internet). BioRxiv. https://doi.org/10.1101/515007v2.full. Accessed 3 Mar 2019 (2019). HabibARCrosslandGPatelHWongEKongKGunasekeraHAn artificial intelligence computer-vision algorithm to triage otoscopic images from Australian Aboriginal and Torres Strait Islander childrenOtol. Neurotol.202243448148810.1097/MAO.000000000000348435239622 MyburghHCvan ZijlWHSwanepoelDWHellströmSLaurentCOtitis media diagnosis for developing countries using tympanic membrane image-analysisEBioMedicine [Internet]2016515616010.1016/j.ebiom.2016.02.01727077122 Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T. et al. An Image is Worth 16 x 16 Words: Transformers for Image Recognition at Scale. arXiv [Internet]. 11929. http://arxiv.org/abs/2010.11929 (2020). LiuWLiCRahamanMMJiangTSunHWuXIs the aspect ratio of cells important in deep learning? A robust comparison of deep learning methods for multi-scale cytopathology cell image classification: From convolutional neural networks to visual transformersComput. Biol. Med.20221141 UçarMAkyolKAtilaUçarEClassification of different tympanic membrane conditions using fused deep hypercolumn features and bidirectional LSTMIrbm (Internet)2021111110.1016/j.irbm.2021.01.001 PichicheroMEAssessing diagnostic accuracy and tympanocentesis skills of South African physicians in management of otitis media [7]S. Afr. Med. J.200292213713811894649 BinolHKhalidMNiaziKElmaraghyCMoberlyACGurcanMNOtoXNet—Automated identification of eardrum diseases from otoscope videos: A deep learning study for video-representing imagesmedRxiv (Internet)202110.1101/2021.08.05.21261672 ByunHYuSOhJBaeJYoonMSLeeSHAn assistive role of a machine learning network in diagnosis of middle ear diseasesJ. Clin. Med.20211015319810.3390/jcm10153198343619828347824 Basaran, E., Sengur, A., Comert, Z., Budak, U., Celik, Y., & Velappan, S. Normal and acute tympanic membrane diagnosis based on gray level co-occurrence matrix and artificial neural networks. in 2019 International Conference on Artificial Intelligence and Data Processing Symposium, IDAP 2019. 5–10 (2019). ChaDPaeCLeeSANaGHurYKLeeHYDifferential biases and variabilities of deep learning-based artificial intelligence and human experts in clinical diagnosis: Retrospective cohort and survey studyJMIR Med. Z Wu (31921_CR35) 2020; 131 D Livingstone (31921_CR20) 2019; 48 H Binol (31921_CR25) 2021 J Huang (31921_CR59) 2005; 17 H Alami (31921_CR63) 2020; 16 E Başaran (31921_CR24) 2022; 34 E Basaran (31921_CR52) 2020; 56 W Liu (31921_CR36) 2022; 1 31921_CR19 MA Khan (31921_CR11) 2020; 126 AR Habib (31921_CR38) 2022; 43 PJ Wormald (31921_CR10) 1995; 20 H Binol (31921_CR28) 2021; 131 HC Myburgh (31921_CR23) 2016; 5 31921_CR56 J Zhang (31921_CR13) 2022; 1 B Wahl (31921_CR62) 2018; 3 31921_CR54 31921_CR53 H Byun (31921_CR16) 2021; 10 JV Sundgaard (31921_CR33) 2021; 1 CL Ramspek (31921_CR50) 2021; 14 31921_CR5 31921_CR4 31921_CR26 31921_CR29 AP Bradley (31921_CR58) 1997; 30 K Graydon (31921_CR12) 2019; 133 ME Pichichero (31921_CR8) 2005; 69 J Futoma (31921_CR57) 2020; 1 A Alhudhaif (31921_CR32) 2021; 7 EMA Alenezi (31921_CR61) 2021; 26 X Li (31921_CR37) 2022; 55 K Kleinman (31921_CR7) 2021; 1 AR Habib (31921_CR17) 2020; 134 Z Cömert (31921_CR34) 2020; 40 31921_CR3 31921_CR2 31921_CR1 M Uçar (31921_CR31) 2021; 1 D Cha (31921_CR6) 2021; 9 AR Habib (31921_CR39) 2022; 47 J Sandström (31921_CR14) 2022; 12 M Viscaino (31921_CR18) 2020; 15 JF Cohen (31921_CR42) 2016; 6 JY Lee (31921_CR55) 2019; 9 31921_CR30 X Liu (31921_CR43) 2020; 26 E Başaran (31921_CR44) 2020; 22 J Yin (31921_CR51) 2021; 23 EMA Alenezi (31921_CR60) 2021; 61 ME Pichichero (31921_CR9) 2002; 92 MG Crowson (31921_CR15) 2021; 147 S Camalan (31921_CR45) 2020; 15 H Binol (31921_CR27) 2020; 10 31921_CR49 31921_CR48 D Livingstone (31921_CR21) 2019; 13 HC Myburgh (31921_CR22) 2018; 39 31921_CR47 31921_CR46 31921_CR41 H Chen (31921_CR40) 2022; 1
References_xml	– reference: HabibARWongESacksRSinghNArtificial intelligence to detect tympanic membrane perforationsJ. Laryngol. Otol.2020134431131510.1017/S002221512000071732238202 – reference: AleneziEMAJajkoKReidALocatelli-SmithATaoKFMBrightTThe reliability of video otoscopy recordings and still images in the asynchronous diagnosis of middle-ear diseaseInt. J. Audiol.202161921 – reference: MyburghHCvan ZijlWHSwanepoelDWHellströmSLaurentCOtitis media diagnosis for developing countries using tympanic membrane image-analysisEBioMedicine [Internet]2016515616010.1016/j.ebiom.2016.02.01727077122 – reference: CamalanSNiaziMKKMoberlyACTeknosTEssigGElmaraghyCOtoMatch: Content-based eardrum image retrieval using deep learningPLoS One [Internet].20201551161:CAS:528:DC%2BB3cXpslWltrs%3D10.1371/journal.pone.0232776 – reference: AlhudhaifACömertZPolatKOtitis media detection using tympanic membrane images with a novel multi-class machine learning algorithmPeerJ Comput. Sci.2021710.7717/peerj-cs.405338170487959604 – reference: SundgaardJVHarteJBrayPLaugesenSKamideYTanakaCDeep metric learning for otitis media classificationMed. Image Anal.2021171 – reference: Deloitte Access Economics. The Social and Economic Costs of Hearing Loss in Australia [Internet]. https://apo.org.au/node/102776. Accessed 28 Aug 2022 (2017) – reference: SandströmJMyburghHLaurentCSwanepoelDWLundbergTA machine learning approach to screen for otitis media using digital otoscope images labelled by an expert panelDiagnostics2022126131810.3390/diagnostics12061318357411289222011 – reference: LiXLiCRahamanMMSunHLiXWuJA comprehensive review of computer-aided whole-slide image analysis: From datasets to feature extraction, segmentation, classification and detection approachesArtif. Intell. Rev.20225564809487810.1007/s10462-021-10121-0 – reference: LeeJYChoiSHChungJWAutomated classification of the tympanic membrane using a convolutional neural networkAppl. Sci. (Switzerland).2019991827 – reference: Basaran, E., Comert, Z., Sengur, A., Budak, U., Celik, Y., & Togacar, M. Chronic tympanic membrane diagnosis based on deep convolutional neural network. in UBMK 2019—Proceedings, 4th International Conference on Computer Science and Engineering. 635–638 (2019). – reference: YinJNgiamKYTeoHHRole of artificial intelligence applications in real-life clinical practice: Systematic reviewJ. Med. Internet Res. (JMIR Publications Inc.)2021232575910.2196/25759 – reference: RamspekCLJagerKJDekkerFWZoccaliCvan DiepenMExternal validation of prognostic models: What, why, how, when and where?Clin. Kidney J. (Oxford University Press)202114495810.1093/ckj/sfaa188 – reference: Wang, W., Tamhane, A., Santos, C., Rzasa, J.R., Clark, J.H., Canares, T.L. et al. Pediatric Otoscopy Video Screening with Shift Contrastive Anomaly Detection. http://arxiv.org/abs/2110.13254. Accessed 25 Oct 2021 (2021). – reference: Shield, B. Evaluation of the Social and Economic Costs of Hearing Impairment: A Report for Hear-It [Internet]. https://www.hear-it.org/sites/default/files/multimedia/documents/Hear_It_Report_October_2006.pdf. Accessed 28 Aug 2022 (2006). – reference: LiuXCruz RiveraSMoherDCalvertMDennistonAKSpirit-aiTCONSORT-AI extensionNat Med.202026136413741:CAS:528:DC%2BB3cXhvVyqu7%2FO10.1038/s41591-020-1034-x329082837598943 – reference: ViscainoMMaassJCDelanoPHTorrenteMStottCAuatCheeinFComputer-aided diagnosis of external and middle ear conditions: A machine learning approachPLoS One [Internet].20201531181:CAS:528:DC%2BB3cXmtlCqtbo%3D10.1371/journal.pone.0229226 – reference: CömertZFusing fine-tuned deep features for recognizing different tympanic membranesBiocybern. Biomed. Eng. (Internet)2020401405110.1016/j.bbe.2019.11.001 – reference: ChenHLiCLiXRahamanMMHuWLiYIL-MCAM: An interactive learning and multi-channel attention mechanism-based weakly supervised colorectal histopathology image classification approachComput. Biol. Med.202211431:CAS:528:DC%2BB38XisFKntrzE – reference: World Health Organisation. World Report on Hearing [Internet]. Geneva; 2021. https://www.who.int/publications/i/item/world-report-on-hearing. Accessed 28 Aug 2022 (2021). – reference: PichicheroMEAssessing diagnostic accuracy and tympanocentesis skills of South African physicians in management of otitis media [7]S. Afr. Med. J.200292213713811894649 – reference: Chen, H., Li, C., Wang, G., Li, X., Rahaman, M., Sun, H. et al. GasHis-Transformer: A Multi-scale Visual Transformer Approach for Gastric Histopathological Image Detection. http://arxiv.org/abs/2104.14528. Accessed 29 Apr 2021 (2021). – reference: Goshtasbi, K. Machine Learning Models to Predict Diagnosis and Surgical Outcomes in Otolaryngology [Internet]. https://escholarship.org/uc/item/1tr0c2p0 (University of California Irvine, 2020). – reference: Binol, H., Niazi, M.K.K., Elmaraghy, C., Moberly, A.C., & Gurcan, M.N. Automated video summarization and label assignment for otoscopy videos using deep learning and natural language processing. in Medical Imaging 2021: Imaging Informatics for Healthcare, Research, and Applications [Internet] (Park, B.J., Deserno, T.M. Eds.). 28. https://www.spiedigitallibrary.org/conference-proceedings-of-spie/11601/2582009/Automated-video-summarization-and-label-assignment-for-otoscopy-videos-using/https://doi.org/10.1117/12.2582009.full. Accessed 17 Mar 2021 (SPIE, 2021). – reference: He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. http://arxiv.org/abs/1512.03385. Accessed 10 Dec 2015 (2015). – reference: KhanMAKwonSChooJHongSMKangSHParkIHAutomatic detection of tympanic membrane and middle ear infection from oto-endoscopic images via convolutional neural networksNeural Netw. [Internet].202012638439410.1016/j.neunet.2020.03.02332311656 – reference: Simonyan, K., & Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. http://arxiv.org/abs/1409.1556. Accessed 4 Sep 2014 (2014). – reference: WahlBCossy-GantnerAGermannSSchwalbeNRArtificial intelligence (AI) and global health: how can AI contribute to health in resource-poor settings?BMJ Glob Health (Internet)201834e00079810.1136/bmjgh-2018-000798 – reference: HabibARKajbafzadehMHasanZWongEGunasekeraHPerryCArtificial intelligence to classify ear disease from otoscopy: A systematic review and meta-analysisClin. Otolaryngol.202247340141310.1111/coa.13925352533789310803 – reference: ChaDPaeCLeeSANaGHurYKLeeHYDifferential biases and variabilities of deep learning-based artificial intelligence and human experts in clinical diagnosis: Retrospective cohort and survey studyJMIR Med. Inform.20219123304910.2196/33049 – reference: Huang, G., Liu, Z., van der Maaten, L., & Weinberger, K.Q. Densely Connected Convolutional Networks. http://arxiv.org/abs/1608.06993. Accessed 24 Aug 2016 (2016). – reference: Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T. et al. An Image is Worth 16 x 16 Words: Transformers for Image Recognition at Scale. arXiv [Internet]. 11929. http://arxiv.org/abs/2010.11929 (2020). – reference: UçarMAkyolKAtilaUçarEClassification of different tympanic membrane conditions using fused deep hypercolumn features and bidirectional LSTMIrbm (Internet)2021111110.1016/j.irbm.2021.01.001 – reference: FutomaJSimonsMPanchTDoshi-VelezFCeliLAThe myth of generalisability in clinical research and machine learning in health careLancet202012e489e492 – reference: BaşaranECömertZCelikYVelappanSTogacarMDetermination of tympanic membrane region in the middle ear otoscope images with convolutional neural network based YOLO methodDeu Muhendislik Fakultesi Fen ve Muhendislik.2020226691992810.21205/deufmd.2020226625 – reference: LivingstoneDChauJOtoscopic diagnosis using computer vision: An automated machine learning approachLaryngoscope20191316 – reference: HuangJLingCXUsing AUC and accuracy in evaluating learning algorithmsIEEE Trans. Knowl. Data Eng.20051732993101:CAS:528:DC%2BD2MXjslCgurg%3D10.1109/TKDE.2005.50 – reference: Wang, W., Tamhane, A., Rzasa, J., Clark, J., Canares, T., & Unberath, M. Otoscopy video screening with deep anomaly detection. in (Drukker, K., Mazurowski, M.A. eds.) Medical Imaging 2021: Computer-Aided Diagnosis [Internet]. 50. https://www.spiedigitallibrary.org/conference-proceedings-of-spie/11597/2581902/Otoscopy-video-screening-with-deep-anomaly-detection/https://doi.org/10.1117/12.2581902.full. Accessed 17 Mar 2021. (SPIE, 2021) – reference: KleinmanKPsoterKJNyhanASolomonBSKimJMCanaresTEvaluation of digital otoscopy in pediatric patients: A prospective randomized controlled clinical trialAm. J. Emerg. Med.202114615015510.1016/j.ajem.2021.04.030 – reference: BinolHKhalidMNiaziKElmaraghyCMoberlyACGurcanMNOtoXNet—Automated identification of eardrum diseases from otoscope videos: A deep learning study for video-representing imagesmedRxiv (Internet)202110.1101/2021.08.05.21261672 – reference: WormaldPJBrowningGGRobinsonKIs otoscopy reliable? A structured teaching method to improve otoscopic accuracy in traineesClin. Otolaryngol. Allied Sci.199520163671:STN:280:DyaK2MzgvVOktw%3D%3D10.1111/j.1365-2273.1995.tb00014.x7788938 – reference: AlamiHAlamiHRivardLRivardLLehouxPLehouxPArtificial intelligence in health care: Laying the foundation for responsible, sustainable, and inclusive innovation in low- and middle-income countriesGlobal Health.202016116 – reference: BinolHMoberlyACNiaziMKKEssigGShahJElmaraghyCSelectStitch: Automated frame segmentation and stitching to create composite images from otoscope video clipsAppl. Sci. (Switzerland)20201017113 – reference: GraydonKWaterworthCMillerHGunasekeraHGlobal burden of hearing impairment and ear diseaseJ. Laryngol. Otol.2019133118251:STN:280:DC%2BB3c7itV2jtA%3D%3D10.1017/S002221511800127530047343 – reference: CrowsonMGHartnickCJDiercksGRGallagherTQFracchiaMSSetlurJMachine learning for accurate intraoperative pediatric middle ear effusion diagnosisPediatrics2021147410.1542/peds.2020-03454633731369 – reference: CohenJFKorevaarDAAltmanDGBrunsDEGatsonisCAHooftLSTARD 2015 guidelines for reporting diagnostic accuracy studies: Explanation and elaborationBMJ Open201661111710.1136/bmjopen-2016-012799 – reference: WuZLinZLiLPanHChenGFuYDeep learning for classification of pediatric otitis mediaLaryngoscope.202013118 – reference: World Health Organization. Childhood Hearing Loss—Act Now, Here’s How! [Internet]. Geneva; 2016. https://apps.who.int/iris/handle/10665/204507. Accessed 28 Aug 2022 (2016). – reference: LiuWLiCRahamanMMJiangTSunHWuXIs the aspect ratio of cells important in deep learning? A robust comparison of deep learning methods for multi-scale cytopathology cell image classification: From convolutional neural networks to visual transformersComput. Biol. Med.20221141 – reference: BasaranEComertZCelikYConvolutional neural network approach for automatic tympanic membrane detection and classificationBiomed. Signal Process Control2020561114 – reference: ByunHYuSOhJBaeJYoonMSLeeSHAn assistive role of a machine learning network in diagnosis of middle ear diseasesJ. Clin. Med.20211015319810.3390/jcm10153198343619828347824 – reference: AleneziEMAJajkoKReidALocatelli-SmithAMcMahenCSETaoKFMClinician-rated quality of video otoscopy recordings and still images for the asynchronous assessment of middle-ear diseaseJ. Telemed. Telecare2021261357633X20987783 – reference: ZhangJLiCYinYZhangJGrzegorzekMApplications of artificial neural networks in microorganism image analysis: A comprehensive review from conventional multilayer perceptron to popular convolutional neural network and potential visual transformerArtif. Intell. Rev.20221158 – reference: MyburghHCJoseSSwanepoelDWLaurentCTowards low cost automated smartphone- and cloud-based otitis media diagnosisBiomed. Signal Process. Control [Internet].201839345210.1016/j.bspc.2017.07.015 – reference: BinolHNiaziMKKEssigGShahJMattinglyJKHarrisMSDigital otoscopy videos versus composite images: A reader study to compare the accuracy of ENT physiciansLaryngoscope20211315E1668E167610.1002/lary.2925333170529 – reference: BradleyAPThe use of the area under the ROC curve in the evaluation of machine learning algorithmsPatter Recognit.1997307114511591997PatRe..30.1145B10.1016/S0031-3203(96)00142-2 – reference: Seok, J., Song, J.J., Koo, J.W., Kin, H.C., & Choi, B.Y. The semantic segmentation approach for normal and pathologic tympanic membrane using deep learning (internet). BioRxiv. https://doi.org/10.1101/515007v2.full. Accessed 3 Mar 2019 (2019). – reference: Basaran, E., Sengur, A., Comert, Z., Budak, U., Celik, Y., & Velappan, S. Normal and acute tympanic membrane diagnosis based on gray level co-occurrence matrix and artificial neural networks. in 2019 International Conference on Artificial Intelligence and Data Processing Symposium, IDAP 2019. 5–10 (2019). – reference: World Health Organization. Global Costs of Unaddressed Hearing Loss and Cost-Effectiveness of Interventions: A WHO Report [Internet]. Geneva; 2017. https://apps.who.int/iris/bitstream/handle/10665/254659/9789241512046-eng.pdf. Accessed 28 Aug 2022 (2017). – reference: PichicheroMEPooleMDComparison of performance by otolaryngologists, pediatricians, and general practioners on an otoendoscopic diagnostic video examinationInt. J. Pediatr. Otorhinolaryngol.200569336136610.1016/j.ijporl.2004.10.01315733595 – reference: LivingstoneDTalaiASChauJForkertNDBuilding an otoscopic screening prototype tool using deep learningJ. Otolaryngol. Head Neck Surg.2019486615 – reference: BaşaranECömertZÇelikYNeighbourhood component analysis and deep feature-based diagnosis model for middle ear otoscope imagesNeural Comput. Appl.20223486027603810.1007/s00521-021-06810-0 – reference: HabibARCrosslandGPatelHWongEKongKGunasekeraHAn artificial intelligence computer-vision algorithm to triage otoscopic images from Australian Aboriginal and Torres Strait Islander childrenOtol. Neurotol.202243448148810.1097/MAO.000000000000348435239622 – ident: 31921_CR26 doi: 10.1117/12.2582009.full – volume: 5 start-page: 156 year: 2016 ident: 31921_CR23 publication-title: EBioMedicine [Internet] doi: 10.1016/j.ebiom.2016.02.017 – volume: 6 start-page: 1 issue: 11 year: 2016 ident: 31921_CR42 publication-title: BMJ Open doi: 10.1136/bmjopen-2016-012799 – year: 2021 ident: 31921_CR25 publication-title: medRxiv (Internet) doi: 10.1101/2021.08.05.21261672 – volume: 133 start-page: 18 issue: 1 year: 2019 ident: 31921_CR12 publication-title: J. Laryngol. Otol. doi: 10.1017/S0022215118001275 – ident: 31921_CR1 – volume: 12 start-page: 1318 issue: 6 year: 2022 ident: 31921_CR14 publication-title: Diagnostics doi: 10.3390/diagnostics12061318 – ident: 31921_CR4 – volume: 10 start-page: 3198 issue: 15 year: 2021 ident: 31921_CR16 publication-title: J. Clin. Med. doi: 10.3390/jcm10153198 – volume: 134 start-page: 311 issue: 4 year: 2020 ident: 31921_CR17 publication-title: J. Laryngol. Otol. doi: 10.1017/S0022215120000717 – volume: 69 start-page: 361 issue: 3 year: 2005 ident: 31921_CR8 publication-title: Int. J. Pediatr. Otorhinolaryngol. doi: 10.1016/j.ijporl.2004.10.013 – volume: 147 issue: 4 year: 2021 ident: 31921_CR15 publication-title: Pediatrics doi: 10.1542/peds.2020-034546 – ident: 31921_CR29 doi: 10.1117/12.2581902.full – ident: 31921_CR41 – volume: 131 start-page: 1 year: 2020 ident: 31921_CR35 publication-title: Laryngoscope. – volume: 1 start-page: 141 year: 2022 ident: 31921_CR36 publication-title: Comput. Biol. Med. – ident: 31921_CR49 – ident: 31921_CR30 doi: 10.3389/fdgth.2021.810427 – volume: 16 start-page: 1 issue: 1 year: 2020 ident: 31921_CR63 publication-title: Global Health. doi: 10.1186/s12992-019-0531-5 – volume: 1 start-page: 143 year: 2022 ident: 31921_CR40 publication-title: Comput. Biol. Med. – volume: 9 start-page: 1827 issue: 9 year: 2019 ident: 31921_CR55 publication-title: Appl. Sci. (Switzerland). – volume: 126 start-page: 384 year: 2020 ident: 31921_CR11 publication-title: Neural Netw. [Internet]. doi: 10.1016/j.neunet.2020.03.023 – volume: 55 start-page: 4809 issue: 6 year: 2022 ident: 31921_CR37 publication-title: Artif. Intell. Rev. doi: 10.1007/s10462-021-10121-0 – ident: 31921_CR5 – ident: 31921_CR53 doi: 10.1109/IDAP.2019.8875973 – volume: 15 start-page: 1 issue: 3 year: 2020 ident: 31921_CR18 publication-title: PLoS One [Internet]. doi: 10.1371/journal.pone.0229226 – ident: 31921_CR56 doi: 10.1101/515007v2.full – volume: 92 start-page: 137 issue: 2 year: 2002 ident: 31921_CR9 publication-title: S. Afr. Med. J. – volume: 10 start-page: 1 issue: 17 year: 2020 ident: 31921_CR27 publication-title: Appl. Sci. (Switzerland) – ident: 31921_CR48 – volume: 26 start-page: 1364 year: 2020 ident: 31921_CR43 publication-title: Nat Med. doi: 10.1038/s41591-020-1034-x – ident: 31921_CR54 doi: 10.1109/UBMK.2019.8907070 – volume: 15 start-page: 1 issue: 5 year: 2020 ident: 31921_CR45 publication-title: PLoS One [Internet]. doi: 10.1371/journal.pone.0232776 – volume: 1 start-page: 1 year: 2022 ident: 31921_CR13 publication-title: Artif. Intell. Rev. – volume: 1 start-page: 71 year: 2021 ident: 31921_CR33 publication-title: Med. Image Anal. – volume: 20 start-page: 63 issue: 1 year: 1995 ident: 31921_CR10 publication-title: Clin. Otolaryngol. Allied Sci. doi: 10.1111/j.1365-2273.1995.tb00014.x – volume: 1 start-page: e489 issue: 2 year: 2020 ident: 31921_CR57 publication-title: Lancet – ident: 31921_CR2 – volume: 43 start-page: 481 issue: 4 year: 2022 ident: 31921_CR38 publication-title: Otol. Neurotol. doi: 10.1097/MAO.0000000000003484 – ident: 31921_CR47 – volume: 1 start-page: 1 year: 2021 ident: 31921_CR31 publication-title: Irbm (Internet) doi: 10.1016/j.irbm.2021.01.001 – volume: 7 year: 2021 ident: 31921_CR32 publication-title: PeerJ Comput. Sci. doi: 10.7717/peerj-cs.405 – volume: 47 start-page: 401 issue: 3 year: 2022 ident: 31921_CR39 publication-title: Clin. Otolaryngol. doi: 10.1111/coa.13925 – volume: 22 start-page: 919 issue: 66 year: 2020 ident: 31921_CR44 publication-title: Deu Muhendislik Fakultesi Fen ve Muhendislik. doi: 10.21205/deufmd.2020226625 – volume: 9 start-page: 33049 issue: 12 year: 2021 ident: 31921_CR6 publication-title: JMIR Med. Inform. doi: 10.2196/33049 – volume: 1 start-page: 150 issue: 46 year: 2021 ident: 31921_CR7 publication-title: Am. J. Emerg. Med. doi: 10.1016/j.ajem.2021.04.030 – volume: 13 start-page: 1 year: 2019 ident: 31921_CR21 publication-title: Laryngoscope – volume: 131 start-page: E1668 issue: 5 year: 2021 ident: 31921_CR28 publication-title: Laryngoscope doi: 10.1002/lary.29253 – volume: 40 start-page: 40 issue: 1 year: 2020 ident: 31921_CR34 publication-title: Biocybern. Biomed. Eng. (Internet) doi: 10.1016/j.bbe.2019.11.001 – volume: 48 start-page: 1 issue: 66 year: 2019 ident: 31921_CR20 publication-title: J. Otolaryngol. Head Neck Surg. – volume: 61 start-page: 921 year: 2021 ident: 31921_CR60 publication-title: Int. J. Audiol. – ident: 31921_CR19 – volume: 34 start-page: 6027 issue: 8 year: 2022 ident: 31921_CR24 publication-title: Neural Comput. Appl. doi: 10.1007/s00521-021-06810-0 – volume: 14 start-page: 49 year: 2021 ident: 31921_CR50 publication-title: Clin. Kidney J. (Oxford University Press) doi: 10.1093/ckj/sfaa188 – volume: 30 start-page: 1145 issue: 7 year: 1997 ident: 31921_CR58 publication-title: Patter Recognit. doi: 10.1016/S0031-3203(96)00142-2 – volume: 17 start-page: 299 issue: 3 year: 2005 ident: 31921_CR59 publication-title: IEEE Trans. Knowl. Data Eng. doi: 10.1109/TKDE.2005.50 – ident: 31921_CR3 – volume: 39 start-page: 34 year: 2018 ident: 31921_CR22 publication-title: Biomed. Signal Process. Control [Internet]. doi: 10.1016/j.bspc.2017.07.015 – volume: 56 start-page: 1 issue: 1 year: 2020 ident: 31921_CR52 publication-title: Biomed. Signal Process Control – volume: 26 start-page: 1357633X2098778 year: 2021 ident: 31921_CR61 publication-title: J. Telemed. Telecare doi: 10.1177/1357633X20987783 – volume: 3 start-page: e000798 issue: 4 year: 2018 ident: 31921_CR62 publication-title: BMJ Glob Health (Internet) doi: 10.1136/bmjgh-2018-000798 – volume: 23 start-page: 25759 year: 2021 ident: 31921_CR51 publication-title: J. Med. Internet Res. (JMIR Publications Inc.) doi: 10.2196/25759 – ident: 31921_CR46
SSID	ssj0000529419
Score	2.4838762
Snippet	To evaluate the generalizability of artificial intelligence (AI) algorithms that use deep learning methods to identify middle ear disease from otoscopic... Abstract To evaluate the generalizability of artificial intelligence (AI) algorithms that use deep learning methods to identify middle ear disease from...
SourceID	doaj pubmedcentral proquest pubmed crossref springer
SourceType	Open Website Open Access Repository Aggregation Database Index Database Enrichment Source Publisher
StartPage	5368
SubjectTerms	692/308/575 692/700/139 692/700/478 Algorithms Artificial Intelligence Deep Learning Ear diseases Ear Diseases - diagnostic imaging Humanities and Social Sciences Humans Middle ear multidisciplinary Otoscopy - methods Science Science (multidisciplinary) Teaching methods
SummonAdditionalLinks	– databaseName: Science Database dbid: M2P link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV3Nb9MwFH-CARIXvscCAxmJG0SLHaeJTwjQJg4w7QBoN8u1nazSmnRNhlSJP573HLdT-diFW1vbqpP34Z_9nn8P4HWZIwh2XODepMINSsFdOsWFOa0FolMrjXGhnM_3z-XxcXV6qk7igVsf0yrXPjE4atdZOiM_EKhI5Ekn6t3iIqWqURRdjSU0bsItRDacUrq-iJPNGQtFsSRX8a5MllcHPa5XdKdM0KUyJXAnvbUeBdr-v2HNP1Mmf4ubhuXo6P7_PsgDuBeBKHs_as5DuOHbR3BnLE25egw_DyMNeNswxIisGempKQeMsmlXrKuZ837BYtmJhs3m6JqYJTRO6UdB4sycN_jfw9m8Z0OHAyhkwebhUIThQBbjQ4zS7xvWDR3dklk9gW9Hh18_fkpjpYbUFpIPaV3kXimBX0zGncEPCNusU3leo8EXQmK7sQjtpsoZWxIbqbF5VZusckpMVL4LO23X-j1gpZeFRQyDbtdLozIzNT4z3nqZOUSbPAG-lpe2kcacqmmc6xBOzys9ylijjHWQsc4SeLMZsxhJPK7t_YHUYNOTCLjDD92y0dGeNS8rrwojnfFCokIjUHW-lNPaStoBiwT219LX0Sv0-kr0CbzaNKM9U5DGtL67DH3kpCIS_ASejjq3mUleEsKW-A6qLW3cmup2Szs7C5zhPMASXibwdq24V_P697t4dv1jPIe7gmwpZDLtw86wvPQv4Lb9Mcz65ctgjL8A5YU9HA priority: 102 providerName: ProQuest
Title	Evaluating the generalizability of deep learning image classification algorithms to detect middle ear disease using otoscopy
URI	https://link.springer.com/article/10.1038/s41598-023-31921-0 https://www.ncbi.nlm.nih.gov/pubmed/37005441 https://www.proquest.com/docview/2793847169 https://www.proquest.com/docview/2794686109 https://pubmed.ncbi.nlm.nih.gov/PMC10067817 https://doaj.org/article/178e95a4dae24ad3951de74bfc419172
Volume	13
WOSCitedRecordID	wos000984099000040&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVAON databaseName: DOAJ Directory of Open Access Journals customDbUrl: eissn: 2045-2322 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000529419 issn: 2045-2322 databaseCode: DOA dateStart: 20110101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2045-2322 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000529419 issn: 2045-2322 databaseCode: M~E dateStart: 20110101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre – providerCode: PRVPQU databaseName: Biological Science Database customDbUrl: eissn: 2045-2322 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000529419 issn: 2045-2322 databaseCode: M7P dateStart: 20110101 isFulltext: true titleUrlDefault: http://search.proquest.com/biologicalscijournals providerName: ProQuest – providerCode: PRVPQU databaseName: Health & Medical Collection customDbUrl: eissn: 2045-2322 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000529419 issn: 2045-2322 databaseCode: 7X7 dateStart: 20110101 isFulltext: true titleUrlDefault: https://search.proquest.com/healthcomplete providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Central customDbUrl: eissn: 2045-2322 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000529419 issn: 2045-2322 databaseCode: BENPR dateStart: 20110101 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest – providerCode: PRVPQU databaseName: Publicly Available Content Database customDbUrl: eissn: 2045-2322 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000529419 issn: 2045-2322 databaseCode: PIMPY dateStart: 20110101 isFulltext: true titleUrlDefault: http://search.proquest.com/publiccontent providerName: ProQuest – providerCode: PRVPQU databaseName: Science Database (subscription) customDbUrl: eissn: 2045-2322 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000529419 issn: 2045-2322 databaseCode: M2P dateStart: 20110101 isFulltext: true titleUrlDefault: https://search.proquest.com/sciencejournals providerName: ProQuest
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV3db9MwED_BBhIviG8yRmUk3qBa4ji1_chQJ5BYFSFA5SlybSertCZTmyFV2h_PnZOWlc8XXqy2tiXrPuzf1effAbyUKYJgl3CMTRQGKFnihjM8mIclR3RqhTEulPP58kFOJmo61fm1Ul-UE9bRA3eCO0qk8jozwhnPBc5EROC8FLPSCgo1wu4bS30tmOpYvbnG_v6VTJyqoxWeVPSajNNzMs0xht45iQJh_-9Q5q_Jkj_dmIaD6OQe3O0RJHvTrfw-3PD1A7jd1ZRcP4Srcc_fXVcMwR2rOl5pSt6iNNg1a0rmvL9gfb2Iis0XuKcwSzCa8oaCqpg5r5rlvD1brFjb4AS6a2CL8G8Gw4msv9hhlDdfsaZt6HnL-hF8Phl_evtu2JdYGNpMJO2wzFKvNccvJk6cwQ-It6zTaVqip2ZcYL-xiMlm2hkriUbU2FSVJlZO85FOH8Ne3dT-KTDpRWYRfOB-6YXRsZkZHxtvvYgdwsQkgmQj7sL2_ONUBuO8CPfgqSo6FRWooiKoqIgjeLWdc9Gxb_x19DFpcTuSmLPDD2hPRW9Pxb_sKYLDjQ0UvTuvCo67GB3jIx3Bi203OiLdrpjaN5dhjBgpYq-P4ElnMtuVpJKgsUAZqB1j2lnqbk89Pwtk30nAE4mM4PXG7n6s68-yOPgfsngGdzg5TEhUOoS9dnnpn8Mt-62dr5YDuCmnMrRqAPvH40n-cRC8ENtTnlMrsd3P35_mX78DN7E2Bw
linkProvider	Directory of Open Access Journals
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1bb9MwFD6aBgheuMMCA4wETxAtcZwmeUCIy6ZNK9UeBtqbcW0nq7QmXZuBKvGb-I2c41ymctnbHnhra1uy3c_H3_G5AbxIIiTBJuSom6SooMSh8cd4Mfs5R3aqhVLGlfP5MkxGo_ToKDtYg59dLAy5VXYy0QlqU2l6I9_iCCSSpIPs7ezUp6pRZF3tSmg0sNi3y--osi3e7H3E__cl5zvbhx92_baqgK9jEdZ-Hkc2yzh-UUFoFH5AiqENKvY5gjPmAtuVRhoyzozSCWXOVDpKcxWkJuMDSr6EIv-KoMxi5CrID_o3HbKaiTBrY3OCKN1a4P1IMWycgtgyjpr7yv3nygT8jdv-6aL5m53WXX87t_63jbsNN1uizd41J-MOrNnyLlxrSm8u78GP7TbNeVkw5MCsaNJvk48beQsvWZUzY-2MtWU1CjaZouhlmrQNcq9yiGbqpMC11sfTBasrHEAmGTZ1jz4MB7LW_sUovKBgVV1RFNDyPny-lKU_gPWyKu0GsMSKWCNHw2vFCpUFaqxsoKy2IjDIpkMPwg4fUrdp2qlayIl07gJRKhtMScSUdJiSgQev-jGzJknJhb3fE-z6npRg3P1QzQvZyisZJqnNYiWMslzggUUibmwixrkWpOFzDzY7tMlW6i3kOdQ8eN43o7wiI5QqbXXm-ohBSkn-PXjYYLyfSZSQBiFwD9IV9K9MdbWlnBy7nOiho11h4sHr7qCcz-vfe_Ho4mU8g-u7h5-Gcrg32n8MNzidY-e1tQnr9fzMPoGr-ls9WcyfOkHA4OtlH6BfDK6Ybw
linkToPdf	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1Lj9MwELZWXUBceD8CCxgJThA1cZwmOSAE7FZUu1Q9AFpOXtd2spW2SWmyoEr8Mn4dM47TVXnsbQ_c2tqWbPeb8Tee8Qwhz5IISLAOGdgmKRgocaj9KRzMfs6AnSoupbblfD4fJONxeniYTbbIz-4tDIZVdjrRKmpdKbwj7zMAEmrSQdbPXVjEZHf4evHVxwpS6Gntymm0ENk3q-9gvtWvRrvwXz9nbLj38d1731UY8FXMw8bP48hkGYMvMgi1hA9AN5QGIz8HoMaMQ7tUQEmmmZYqwSyaUkVpLoNUZ2yAiZhA_W8DJeesR7Ynow-TL-sbHvSh8TBzL3WCKO3XcFriizaGT9oyBnb8xmloiwb8jen-GbD5m9fWHobD6__zNt4g1xwFp29amblJtkx5i1xui3KubpMfey4BellQYMe0aBNzY_QbxhGvaJVTbcyCuoIbBZ3NQSlThXYIBl5ZrFN5UsBam-N5TZsKBqCzhs7tdRCFgdR5xig-PCho1VT4Pmh1h3y6kKXfJb2yKs19QhPDYwXsDQ4cw2UWyKk0gTTK8EADzw49EnZYEcolcMc6IifCBhJEqWjxJQBfwuJLBB55sR6zaNOXnNv7LUJw3RNTj9sfqmUhnCYTYZKaLJZcS8M4iDJQdG0SPs0VR9ufeWSnQ55w-rAWZ7DzyNN1M2gydE_J0lSntg8fpJj-3yP3WryvZxIlaFtw2IN0QxI2prrZUs6Obbb00BKyMPHIy05ozub17714cP4ynpArIDfiYDTef0iuMhRpG861Q3rN8tQ8IpfUt2ZWLx87rUDJ0UVL0C-v5aK4
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Evaluating+the+generalizability+of+deep+learning+image+classification+algorithms+to+detect+middle+ear+disease+using+otoscopy&rft.jtitle=Scientific+reports&rft.au=Habib%2C+Al-Rahim&rft.au=Xu%2C+Yixi&rft.au=Bock%2C+Kris&rft.au=Mohanty%2C+Shrestha&rft.date=2023-04-01&rft.issn=2045-2322&rft.eissn=2045-2322&rft.volume=13&rft.issue=1&rft.spage=5368&rft_id=info:doi/10.1038%2Fs41598-023-31921-0&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2045-2322&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2045-2322&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2045-2322&client=summon