Generalized linear mixed models for multi-reader multi-case studies of diagnostic tests
Diagnostic tests are often compared in multi-reader multi-case (MRMC) studies in which a number of cases (subjects with or without the disease in question) are examined by several readers using all tests to be compared. One of the commonly used methods for analyzing MRMC data is the Obuchowski-Rocke...
Saved in:
| Published in: | Statistical methods in medical research Vol. 26; no. 3; pp. 1373 - 1388 |
|---|---|
| Main Authors: | , , , , , , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
England
01.06.2017
|
| Subjects: | |
| ISSN: | 1477-0334 |
| Online Access: | Get more information |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | Diagnostic tests are often compared in multi-reader multi-case (MRMC) studies in which a number of cases (subjects with or without the disease in question) are examined by several readers using all tests to be compared. One of the commonly used methods for analyzing MRMC data is the Obuchowski-Rockette (OR) method, which assumes that the true area under the receiver operating characteristic curve (AUC) for each combination of reader and test follows a linear mixed model with fixed effects for test and random effects for reader and the reader-test interaction. This article proposes generalized linear mixed models which generalize the OR model by incorporating a range-appropriate link function that constrains the true AUCs to the unit interval. The proposed models can be estimated by maximizing a pseudo-likelihood based on the approximate normality of AUC estimates. A Monte Carlo expectation-maximization algorithm can be used to maximize the pseudo-likelihood, and a non-parametric bootstrap procedure can be used for inference. The proposed method is evaluated in a simulation study and applied to an MRMC study of breast cancer detection. |
|---|---|
| AbstractList | Diagnostic tests are often compared in multi-reader multi-case (MRMC) studies in which a number of cases (subjects with or without the disease in question) are examined by several readers using all tests to be compared. One of the commonly used methods for analyzing MRMC data is the Obuchowski-Rockette (OR) method, which assumes that the true area under the receiver operating characteristic curve (AUC) for each combination of reader and test follows a linear mixed model with fixed effects for test and random effects for reader and the reader-test interaction. This article proposes generalized linear mixed models which generalize the OR model by incorporating a range-appropriate link function that constrains the true AUCs to the unit interval. The proposed models can be estimated by maximizing a pseudo-likelihood based on the approximate normality of AUC estimates. A Monte Carlo expectation-maximization algorithm can be used to maximize the pseudo-likelihood, and a non-parametric bootstrap procedure can be used for inference. The proposed method is evaluated in a simulation study and applied to an MRMC study of breast cancer detection. |
| Author | Zhang, Hui Zhang, Zhiwei Liu, Wei Pantoja-Galicia, Norberto Zhang, Bo Pennello, Gene Jacob, Jessie Kotz, Richard M |
| Author_xml | – sequence: 1 givenname: Wei surname: Liu fullname: Liu, Wei organization: 2 Division of Biostatistics, Office of Surveillance and Biometrics, Center for Devices and Radiological Health, Food and Drug Administration, Silver Spring, MD, USA – sequence: 2 givenname: Norberto surname: Pantoja-Galicia fullname: Pantoja-Galicia, Norberto organization: 2 Division of Biostatistics, Office of Surveillance and Biometrics, Center for Devices and Radiological Health, Food and Drug Administration, Silver Spring, MD, USA – sequence: 3 givenname: Bo surname: Zhang fullname: Zhang, Bo organization: 2 Division of Biostatistics, Office of Surveillance and Biometrics, Center for Devices and Radiological Health, Food and Drug Administration, Silver Spring, MD, USA – sequence: 4 givenname: Richard M surname: Kotz fullname: Kotz, Richard M organization: 2 Division of Biostatistics, Office of Surveillance and Biometrics, Center for Devices and Radiological Health, Food and Drug Administration, Silver Spring, MD, USA – sequence: 5 givenname: Gene surname: Pennello fullname: Pennello, Gene organization: 2 Division of Biostatistics, Office of Surveillance and Biometrics, Center for Devices and Radiological Health, Food and Drug Administration, Silver Spring, MD, USA – sequence: 6 givenname: Hui surname: Zhang fullname: Zhang, Hui organization: 3 Department of Biostatistics, St Jude Children's Research Hospital, Memphis, Tennessee, USA – sequence: 7 givenname: Jessie surname: Jacob fullname: Jacob, Jessie organization: 4 Medical Affairs, GE Healthcare, Milwaukee, Wisconsin, USA – sequence: 8 givenname: Zhiwei surname: Zhang fullname: Zhang, Zhiwei organization: 2 Division of Biostatistics, Office of Surveillance and Biometrics, Center for Devices and Radiological Health, Food and Drug Administration, Silver Spring, MD, USA |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/25847911$$D View this record in MEDLINE/PubMed |
| BookMark | eNo1UEtLxDAYDKK4D717khy9VL-keTRHWXQVFrwoHkseXyWStmvTgvrrLbh7GubBDMyKnHZ9h4RcMbhlTOs7MIrzCjiTUhuh1QlZMqF1AWUpFmSV8ycAaBDmnCy4rIQ2jC3J-xY7HGyKvxhoih3agbbxeyZtHzBl2vSzMKUxFgPagEfibUaaxylEzLRvaIj2o-vzGD0dMY_5gpw1NmW8POCavD0-vG6eit3L9nlzvyt8qeRYSECnAJsA4NE48BI9NI12KEPgxmolvEJQbvZCabhRzDKHViIzTkrP1-Tmv3c_9F_TvFy3MXtMyXbYT7lmFVeqVCDEHL0-RCfXYqj3Q2zt8FMfz-B_omhipA |
| CitedBy_id | crossref_primary_10_1002_sim_9400 crossref_primary_10_1007_s00330_022_08947_5 crossref_primary_10_1002_sim_7530 crossref_primary_10_1002_jmv_28118 crossref_primary_10_1016_j_ejrad_2023_110865 crossref_primary_10_1186_s13244_022_01252_1 crossref_primary_10_3390_vaccines9111357 crossref_primary_10_1002_jmv_29241 crossref_primary_10_1016_j_eururo_2018_09_033 |
| ContentType | Journal Article |
| DBID | CGR CUY CVF ECM EIF NPM 7U8 7X8 C1K JXQ |
| DOI | 10.1177/0962280215579476 |
| DatabaseName | Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed TOXLINE MEDLINE - Academic Environmental Sciences and Pollution Management Toxline |
| DatabaseTitle | MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) TOXLINE MEDLINE - Academic Environmental Sciences and Pollution Management |
| DatabaseTitleList | MEDLINE TOXLINE |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | no_fulltext_linktorsrc |
| Discipline | Medicine Statistics Mathematics |
| EISSN | 1477-0334 |
| EndPage | 1388 |
| ExternalDocumentID | 25847911 |
| Genre | Journal Article |
| GroupedDBID | --- -TM .2G .2J .2N 0-V 01A 0R~ 123 1~K 29Q 31S 31U 31X 31Y 31Z 36B 3V. 4.4 53G 54M 5RE 5VS 6PF 7X7 88E 88I 8C1 8FE 8FG 8FI 8FJ 8R4 8R5 AABMB AABOD AACKU AACMV AACTG AADTT AADUE AAEWN AAGGD AAJIQ AAJOX AAJPV AAMGE AANSI AAPEO AAQDB AAQXH AAQXI AARDL AARIX AATAA AATBZ AAWTL AAYTG ABAWP ABCCA ABCJG ABDLQ ABDWY ABEIX ABFWQ ABHKI ABHQH ABJCF ABJIS ABKRH ABLUO ABPGX ABPNF ABQKF ABQXT ABRHV ABTDE ABUJY ABUWG ABVFX ABVVC ABYTW ACARO ACDSZ ACDXX ACFEJ ACFMA ACGBL ACGFS ACGOD ACGZU ACIWK ACJER ACLHI ACLZU ACOFE ACOXC ACROE ACRPL ACSBE ACSIQ ACTQU ACUAV ACUIR ACXKE ACXMB ADBBV ADEIA ADNMO ADNON ADRRZ ADTBJ ADUKL ADVBO ADYCS AECGH AECVZ AEDTQ AENEX AEPTA AEQLS AERKM AESZF AEUHG AEUIJ AEWDL AEWHI AEXNY AFEET AFKBI AFKRA AFKRG AFMOU AFQAA AFUIA AFWMB AGKLV AGNHF AGWFA AGWNL AHDMH AHHFK AHMBA AIOMO AJEFB AJMMQ AJUZI AJXAJ ALIPV ALKWR ALMA_UNASSIGNED_HOLDINGS ALSLI AMCVQ ANDLU ARALO ARTOV ASOEW ASPBG AUTPY AUVAJ AVWKF AYAKG AZFZN AZQEC B8O B8R B8Z B93 B94 BBRGL BDDNI BENPR BGLVJ BKIIM BPACV BPHCQ BSEHC BVXVI BYIEH C45 CAG CBRKF CCPQU CFDXU CGR COF CORYS CQQTX CS3 CUY CVF DC- DD- DD0 DE- DF0 DO- DOPDO DU5 DV7 DWQXO D~Y EAD EAP EBS ECM EIF EJD EMB EMK EMOBN ESX F5P FEDTE FHBDP FYUFA GNUQQ GROUPED_SAGE_PREMIER_JOURNAL_COLLECTION H13 HCIFZ HEHIP HF~ HMCUK HVGLF HZ~ J8X K.F K.J L6V M1P M2P M2S M4V M7S N9A NPM O9- OVD P.B P2P PQQKQ PROAC PSQYO PTHSS Q1R Q2X Q7K Q7L Q7X Q82 Q83 RIG ROL S01 SAUOL SCNPE SDB SFB SFC SFK SFN SFT SGA SGP SGR SGV SGX SGZ SHG SNB SPJ SPV SQCSI STM SV3 TEORI TN5 UKHRP YHZ ZONMY ZPPRI ZRKOI 7U8 7X8 AAPII ABIDT ADDLC ADEBD AJGYC AJHME AJVBE AMVHM C1K JXQ SASJQ |
| ID | FETCH-LOGICAL-c365t-50eb60efd00ce9b0c5ec0ff7be5dd29a764c6e06b9b0d392961a1bea5e19b55c2 |
| IEDL.DBID | 7X8 |
| ISICitedReferencesCount | 10 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000403319000020&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Thu Oct 02 11:29:07 EDT 2025 Wed Feb 19 02:42:21 EST 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 3 |
| Keywords | random effect diagnostic medicine ROC curve EM algorithm pseudo-likelihood AUC |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c365t-50eb60efd00ce9b0c5ec0ff7be5dd29a764c6e06b9b0d392961a1bea5e19b55c2 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| PMID | 25847911 |
| PQID | 1826636044 |
| PQPubID | 23479 |
| PageCount | 16 |
| ParticipantIDs | proquest_miscellaneous_1826636044 pubmed_primary_25847911 |
| PublicationCentury | 2000 |
| PublicationDate | 2017-Jun 20170601 |
| PublicationDateYYYYMMDD | 2017-06-01 |
| PublicationDate_xml | – month: 06 year: 2017 text: 2017-Jun |
| PublicationDecade | 2010 |
| PublicationPlace | England |
| PublicationPlace_xml | – name: England |
| PublicationTitle | Statistical methods in medical research |
| PublicationTitleAlternate | Stat Methods Med Res |
| PublicationYear | 2017 |
| SSID | ssj0007049 |
| Score | 2.1980333 |
| Snippet | Diagnostic tests are often compared in multi-reader multi-case (MRMC) studies in which a number of cases (subjects with or without the disease in question) are... |
| SourceID | proquest pubmed |
| SourceType | Aggregation Database Index Database |
| StartPage | 1373 |
| SubjectTerms | Algorithms Area Under Curve Breast Neoplasms - diagnosis Diagnostic Tests, Routine - methods Female Humans Likelihood Functions Linear Models Monte Carlo Method ROC Curve |
| Title | Generalized linear mixed models for multi-reader multi-case studies of diagnostic tests |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/25847911 https://www.proquest.com/docview/1826636044 |
| Volume | 26 |
| WOSCitedRecordID | wos000403319000020&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LS8NAEB7UitSDj_p-sYLXpdsku5ucRMTipaUHxd7KPibQg21tqoi_3tkktSdB8BJYQkKY55eZ4RuAm9imKvVK8FzJnCeYKG7IpTgKZzWmGaqOKZdN6H4_HQ6zQV1wK-qxymVMLAO1n7pQI28HHBy4rZLkdvbGw9ao0F2tV2isQyMmKBMcUw9XbOGa4O-qNdkmtB7IXyjJSbJCrX4HlWVy6e7-97P2YKeGleyusoN9WMNJC7Z7P5ysRQu2enUbvQXNADErhuYDeKmZp8df6FkAnWbOXsefdCi35BSMYC0r5w75vJx7rg-O8h8rqjFENs2Zr4b26KWM4OuiOITn7sPT_SOv1y1wFyu54FKgVQJzL4TDzAon0Yk81xal91FmtEqcQqEs3fMBVpEaOxaNxE5mpXTREWxMphM8AaacoZ_0jjGoIxJ0WFFJurAqtj5NUqNP4XopzRGZc-hRmAlO34vRSp6ncFypZDSreDdGUWjpUnA--8PT59CMQgIu6yUX0MjJmfESNt0HiXd-VdoJXfuD3jety8pr |
| linkProvider | ProQuest |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Generalized+linear+mixed+models+for+multi-reader+multi-case+studies+of+diagnostic+tests&rft.jtitle=Statistical+methods+in+medical+research&rft.au=Liu%2C+Wei&rft.au=Pantoja-Galicia%2C+Norberto&rft.au=Zhang%2C+Bo&rft.au=Kotz%2C+Richard+M&rft.date=2017-06-01&rft.eissn=1477-0334&rft.volume=26&rft.issue=3&rft.spage=1373&rft.epage=1388&rft_id=info:doi/10.1177%2F0962280215579476&rft.externalDBID=NO_FULL_TEXT |