Do Differential Response Rates to Patient Surveys Between Organizations Lead to Unfair Performance Comparisons?: Evidence From the English Cancer Patient Experience Survey
Patient surveys typically have variable response rates between organizations, leading to concerns that such differences may affect the validity of performance comparisons. To explore the size and likely sources of associations between hospital-level survey response rates and patient experience. Cros...
Saved in:
| Published in: | Medical care Vol. 54; no. 1; pp. 45 - 54 |
|---|---|
| Main Authors: | , , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
United States
01.01.2016
|
| Subjects: | |
| ISSN: | 1537-1948 |
| Online Access: | Get more information |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | Patient surveys typically have variable response rates between organizations, leading to concerns that such differences may affect the validity of performance comparisons.
To explore the size and likely sources of associations between hospital-level survey response rates and patient experience.
Cross-sectional mail survey including 60 patient experience items sent to 101,771 cancer survivors recently treated by 158 English NHS hospitals. Age, sex, race/ethnicity, socioeconomic status, clinical diagnosis, hospital type, and region were available for respondents and nonrespondents.
The overall response rate was 67% (range, 39% to 77% between hospitals). Hospitals with higher response rates had higher scores for all items (Spearman correlation range, 0.03-0.44), particularly questions regarding hospital-level administrative processes, for example, procedure cancellations or medical note availability.From multivariable analysis, associations between individual patient experience and hospital-level response rates were statistically significant (P<0.05) for 53/59 analyzed questions, decreasing to 37/59 after adjusting for case-mix, and 25/59 after further adjusting for hospital-level characteristics.Predicting responses of nonrespondents, and re-estimating hypothetical hospital scores assuming a 100% response rate, we found that currently low performing hospitals would have attained even lower scores. Overall nationwide attainment would have decreased slightly to that currently observed.
Higher response rate hospitals have more positive experience scores, and this is only partly explained by patient case-mix. High response rates may be a marker of efficient hospital administration, and higher quality that should not, therefore, be adjusted away in public reporting. Although nonresponse may result in slightly overestimating overall national levels of performance, it does not appear to meaningfully bias comparisons of case-mix-adjusted hospital results. |
|---|---|
| AbstractList | Patient surveys typically have variable response rates between organizations, leading to concerns that such differences may affect the validity of performance comparisons.
To explore the size and likely sources of associations between hospital-level survey response rates and patient experience.
Cross-sectional mail survey including 60 patient experience items sent to 101,771 cancer survivors recently treated by 158 English NHS hospitals. Age, sex, race/ethnicity, socioeconomic status, clinical diagnosis, hospital type, and region were available for respondents and nonrespondents.
The overall response rate was 67% (range, 39% to 77% between hospitals). Hospitals with higher response rates had higher scores for all items (Spearman correlation range, 0.03-0.44), particularly questions regarding hospital-level administrative processes, for example, procedure cancellations or medical note availability.From multivariable analysis, associations between individual patient experience and hospital-level response rates were statistically significant (P<0.05) for 53/59 analyzed questions, decreasing to 37/59 after adjusting for case-mix, and 25/59 after further adjusting for hospital-level characteristics.Predicting responses of nonrespondents, and re-estimating hypothetical hospital scores assuming a 100% response rate, we found that currently low performing hospitals would have attained even lower scores. Overall nationwide attainment would have decreased slightly to that currently observed.
Higher response rate hospitals have more positive experience scores, and this is only partly explained by patient case-mix. High response rates may be a marker of efficient hospital administration, and higher quality that should not, therefore, be adjusted away in public reporting. Although nonresponse may result in slightly overestimating overall national levels of performance, it does not appear to meaningfully bias comparisons of case-mix-adjusted hospital results. BACKGROUNDPatient surveys typically have variable response rates between organizations, leading to concerns that such differences may affect the validity of performance comparisons.OBJECTIVETo explore the size and likely sources of associations between hospital-level survey response rates and patient experience.RESEARCH DESIGN, SUBJECTS, AND MEASURESCross-sectional mail survey including 60 patient experience items sent to 101,771 cancer survivors recently treated by 158 English NHS hospitals. Age, sex, race/ethnicity, socioeconomic status, clinical diagnosis, hospital type, and region were available for respondents and nonrespondents.RESULTSThe overall response rate was 67% (range, 39% to 77% between hospitals). Hospitals with higher response rates had higher scores for all items (Spearman correlation range, 0.03-0.44), particularly questions regarding hospital-level administrative processes, for example, procedure cancellations or medical note availability.From multivariable analysis, associations between individual patient experience and hospital-level response rates were statistically significant (P<0.05) for 53/59 analyzed questions, decreasing to 37/59 after adjusting for case-mix, and 25/59 after further adjusting for hospital-level characteristics.Predicting responses of nonrespondents, and re-estimating hypothetical hospital scores assuming a 100% response rate, we found that currently low performing hospitals would have attained even lower scores. Overall nationwide attainment would have decreased slightly to that currently observed.CONCLUSIONSHigher response rate hospitals have more positive experience scores, and this is only partly explained by patient case-mix. High response rates may be a marker of efficient hospital administration, and higher quality that should not, therefore, be adjusted away in public reporting. Although nonresponse may result in slightly overestimating overall national levels of performance, it does not appear to meaningfully bias comparisons of case-mix-adjusted hospital results. |
| Author | Abel, Gary A Saunders, Catherine L Lyratzopoulos, Georgios Elliott, Marc N |
| Author_xml | – sequence: 1 givenname: Catherine L surname: Saunders fullname: Saunders, Catherine L organization: Cambridge Centre for Health Services Research, University of Cambridge†RAND Europe, Cambridge, UK‡RAND Corporation, Santa Monica, CA§Department of Epidemiology & Public Health, Health Behaviour Research Centre, University College London, London, UK – sequence: 2 givenname: Marc N surname: Elliott fullname: Elliott, Marc N – sequence: 3 givenname: Georgios surname: Lyratzopoulos fullname: Lyratzopoulos, Georgios – sequence: 4 givenname: Gary A surname: Abel fullname: Abel, Gary A |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/26595223$$D View this record in MEDLINE/PubMed |
| BookMark | eNpNkFFPwjAUhRujEUH_gTF99AXsunXrfDEKQ00wEJRn0q13ULO1sx0o_iX_pEPQeF9ucs6Xe3JuGx1qowGhc4_0PBJHV0-jaY_8n4BFB-jEY37U9eKAt1DbuVdCvMhn9Bi1aMhiRql_gr4GBg9UnoMFXStR4Cm4ymgHeCpqcLg2eCJq1Zj4eWXXsHH4Dup3AI3HdiG0-mzchscjEHJLz3QulMUTsLmxpdAZ4L4pK2GVa7Cba5yslYStPLSmxPUScKIXhXJL3N_S9i8v-ajAqh90F32KjnJRODjb7w6aDZOX_kN3NL5_7N-OulnAad2lkmdB08jPmCchZiyQqQj8MEt5yoFJ2ighCVPCZRRzEsoszYNMgp-yPA5ESjvocne3suZtBa6el8plUBRCg1m5uReFhEch416DXuzRVVqCnFdWlcJu5r8Ppt-SBYHm |
| CitedBy_id | crossref_primary_10_1016_j_resuscitation_2020_04_004 crossref_primary_10_1097_MLR_0000000000001077 crossref_primary_10_1001_jamahealthforum_2023_4929 crossref_primary_10_5435_JAAOSGlobal_D_21_00052 crossref_primary_10_1111_codi_15423 crossref_primary_10_1016_j_acap_2017_03_012 crossref_primary_10_1097_MLR_0000000000000580 crossref_primary_10_1097_MLR_0000000000001030 crossref_primary_10_1097_SLA_0000000000006382 crossref_primary_10_3171_2018_8_SPINE18594 crossref_primary_10_1016_j_mayocp_2019_11_028 crossref_primary_10_1097_MLR_0000000000002127 crossref_primary_10_1111_codi_70042 crossref_primary_10_1007_s00431_017_2919_7 crossref_primary_10_1016_j_acap_2016_07_008 crossref_primary_10_1111_1475_6773_13853 crossref_primary_10_1016_j_drugalcdep_2018_02_018 crossref_primary_10_1097_DCR_0000000000001656 crossref_primary_10_1111_ajt_14074 crossref_primary_10_1111_codi_15606 crossref_primary_10_1186_s41687_025_00922_0 crossref_primary_10_1097_MLR_0000000000002197 crossref_primary_10_2196_51272 crossref_primary_10_3171_2018_8_SPINE181024 crossref_primary_10_1017_S1368980016002524 crossref_primary_10_1111_jrh_12813 crossref_primary_10_1177_1355819620986814 crossref_primary_10_1097_MLR_0000000000001784 crossref_primary_10_1080_01621424_2017_1320698 crossref_primary_10_1097_MLR_0000000000001005 crossref_primary_10_2196_51931 crossref_primary_10_3390_ijerph15081772 crossref_primary_10_1016_j_jdeveco_2023_103106 crossref_primary_10_1080_2994399X_2025_2507642 crossref_primary_10_1177_00258024241259327 crossref_primary_10_3390_ijerph14111319 crossref_primary_10_1016_j_jpainsymman_2018_07_014 crossref_primary_10_1111_ped_15445 crossref_primary_10_1371_journal_pone_0249298 crossref_primary_10_1016_j_apmr_2017_10_019 crossref_primary_10_1002_mpr_1592 crossref_primary_10_1016_j_semnephrol_2024_151551 crossref_primary_10_1016_j_surg_2018_08_028 crossref_primary_10_12788_jhm_3037 crossref_primary_10_1093_intqhc_mzw114 crossref_primary_10_1136_bmjopen_2020_037641 crossref_primary_10_1016_j_jaad_2017_03_051 crossref_primary_10_1186_s12913_020_5012_2 |
| ContentType | Journal Article |
| DBID | CGR CUY CVF ECM EIF NPM 7X8 |
| DOI | 10.1097/MLR.0000000000000457 |
| DatabaseName | Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed MEDLINE - Academic |
| DatabaseTitle | MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) MEDLINE - Academic |
| DatabaseTitleList | MEDLINE MEDLINE - Academic |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | no_fulltext_linktorsrc |
| Discipline | Medicine Public Health |
| EISSN | 1537-1948 |
| EndPage | 54 |
| ExternalDocumentID | 26595223 |
| Genre | Research Support, Non-U.S. Gov't Journal Article |
| GrantInformation_xml | – fundername: Cancer Research UK grantid: A18180 – fundername: Cancer Research UK grantid: 18180 |
| GroupedDBID | --- -~X .-D ..I .3C .GJ .Z2 01R 0R~ 1J1 3R3 40H 4Q1 4Q2 4Q3 53G 5GY 5VS 71W 77Y 7O~ 8L- AAAAV AAAXR AAGIX AAHPQ AAIQE AAMOA AAMTA AAQKA AAQQT AARTV AASCR AASOK AAWTL AAWTO AAXQO AAYEP ABASU ABAWQ ABBHK ABBUW ABDIG ABIVO ABJNI ABPPZ ABPXF ABQDR ABVCZ ABXSQ ABXVJ ABZAD ABZZY ACCJW ACDDN ACDIW ACEWG ACGFO ACGFS ACHJO ACHQT ACILI ACLDA ACLED ACNWC ACWDW ACWRI ACXJB ACXNZ ACZKN ADFPA ADGGA ADHPY ADNKB ADULT AE3 AE6 AEETU AELLO AENEX AEUPB AFBFQ AFDTB AFRAH AFUWQ AGINI AHMBA AHOMT AHQNM AHRYX AHVBC AIJEX AINUH AJCLO AJIOK AJNWD AJNYG AJRGT AJZMW AKBRZ AKCTQ AKULP ALKUP ALMA_UNASSIGNED_HOLDINGS ALMTX AMJPA AMKUR AMNEI AOHHW AOQMC BOYCO BQLVK BS7 BYPQX C45 CGR CS3 CUY CVF DIWNM DU5 DUNZO E.X EBS ECM EEVPB EIF EJD ERAAH EX3 F2K F2L F2M F2N F5P FCALG FL- FW0 GIFXF GNXGY GQDEL H0~ HGD HLJTE HVGLF HZ~ H~9 IH2 IKREB IKYAY IN~ IPNFZ IPSME JAA JAAYA JBMMH JBZCM JENOY JF9 JG8 JHFFW JK3 JK8 JKQEH JLEZI JLXEF JPL JST K8S KD2 KMI L-C L7B N4W N9A NPM N~7 N~B N~M O9- OAG OAH OCUKA ODA ODZKP OL1 OLG OLH OLL OLU OLV OLY OLZ OPUJH OPX ORVUJ OUVQU OVD OVDNE OVIDH OVLEI OWBYB OWU OWV OWW OWX OWY OWZ OXXIT P-K P-O P2P PQQKQ QZG R58 RIG RLZ S4R S4S SA0 T8P TEORI TN5 TSPGW V2I VQP VVN W3M WH7 WOQ WOW X3V X3W XFW XXN XYM YFH YHZ YOC ZFV ZGI ZXP ZY1 ZZMQN 7X8 ADKSD ADSXY |
| ID | FETCH-LOGICAL-c482t-2d8c4ffe3c51de9554dba436cb8b8e5d2554606b08d79806dcbf4cde3b5f94ab2 |
| IEDL.DBID | 7X8 |
| ISICitedReferencesCount | 60 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000366940000009&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Mon Sep 08 03:05:55 EDT 2025 Sat May 31 02:06:24 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c482t-2d8c4ffe3c51de9554dba436cb8b8e5d2554606b08d79806dcbf4cde3b5f94ab2 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| OpenAccessLink | https://pubmed.ncbi.nlm.nih.gov/PMC4674144 |
| PMID | 26595223 |
| PQID | 1760876581 |
| PQPubID | 23479 |
| PageCount | 10 |
| ParticipantIDs | proquest_miscellaneous_1760876581 pubmed_primary_26595223 |
| PublicationCentury | 2000 |
| PublicationDate | 2016-01-01 |
| PublicationDateYYYYMMDD | 2016-01-01 |
| PublicationDate_xml | – month: 01 year: 2016 text: 2016-01-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States |
| PublicationTitle | Medical care |
| PublicationTitleAlternate | Med Care |
| PublicationYear | 2016 |
| SSID | ssj0017352 |
| Score | 2.420387 |
| SecondaryResourceType | review_article |
| Snippet | Patient surveys typically have variable response rates between organizations, leading to concerns that such differences may affect the validity of performance... BACKGROUNDPatient surveys typically have variable response rates between organizations, leading to concerns that such differences may affect the validity of... |
| SourceID | proquest pubmed |
| SourceType | Aggregation Database Index Database |
| StartPage | 45 |
| SubjectTerms | Adult Aged Bias Cross-Sectional Studies England Female Health Care Surveys - methods Hospitals - standards Humans Inpatients - psychology Inpatients - statistics & numerical data Male Middle Aged Neoplasms - epidemiology Neoplasms - therapy Patient Satisfaction - statistics & numerical data Quality Indicators, Health Care Surveys and Questionnaires |
| Title | Do Differential Response Rates to Patient Surveys Between Organizations Lead to Unfair Performance Comparisons?: Evidence From the English Cancer Patient Experience Survey |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/26595223 https://www.proquest.com/docview/1760876581 |
| Volume | 54 |
| WOSCitedRecordID | wos000366940000009&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1JS8NAFB7cEEFc6r7xBK_BNJkkM15Eq8WDLaVa6a1kloCgSe32p_yTvslMLB4EwRxymmES3pvJl7d8HyEXNBIU3ZZ7OqLaozrlXqp97gkRJ4pnKmRlle_LY9Jus36fd1zAbezKKqszsTyoVSFNjPyynsSGPS1i9evhh2dUo0x21UloLJLlEKGM8eqkP88iJIguqnY5nly2HruWrrC6aJT8DizLD0xz87-PtkU2HLSEG-sL22RB5zWy2nLJ8xpZtyE6sJ1HO-TzroA7p4-C-_wNurZeVkPXAFCYFNCxrKvwNB3N0ORwa6u64EcHJxiZTjO6l5vsEHTmvQjQ-JY5HF9fQSVhCs1R8Q4IPcF1EUPDjB59rzcnYHZL75Je8_658eA57QZPUhZMvEAxSfENQhnVleYIWpRIaRhLwQTTkQpMdZwfC5-phDM_VlJkVCodiijjNBXBHlnKi1wfEIhVKlOVhehCuuRjU-hHCGvx5wuxo_YPyXlllgHuDZPwSHNdTMeDuWEOyb617WBoSTwGgSFSRGx09IfZx2QNcZKLvJyQ5QxPBn1KVuRs8joenZVOh_d2p_UFiUbl7g |
| linkProvider | ProQuest |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Do+Differential+Response+Rates+to+Patient+Surveys+Between+Organizations+Lead+to+Unfair+Performance+Comparisons%3F%3A+Evidence+From+the+English+Cancer+Patient+Experience+Survey&rft.jtitle=Medical+care&rft.au=Saunders%2C+Catherine+L&rft.au=Elliott%2C+Marc+N&rft.au=Lyratzopoulos%2C+Georgios&rft.au=Abel%2C+Gary+A&rft.date=2016-01-01&rft.eissn=1537-1948&rft.volume=54&rft.issue=1&rft.spage=45&rft.epage=54&rft_id=info:doi/10.1097%2FMLR.0000000000000457&rft.externalDBID=NO_FULL_TEXT |