Q-LEARNING WITH CENSORED DATA
We develop methodology for a multistage-decision problem with flexible number of stages in which the rewards are survival times that are subject to censoring. We present a novel Q-learning algorithm that is adjusted for censored data and allows a flexible number of stages. We provide finite sample b...
Uložené v:
| Vydané v: | The Annals of statistics Ročník 40; číslo 1; s. 529 |
|---|---|
| Hlavní autori: | , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
United States
01.02.2012
|
| ISSN: | 0090-5364 |
| On-line prístup: | Zistit podrobnosti o prístupe |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | We develop methodology for a multistage-decision problem with flexible number of stages in which the rewards are survival times that are subject to censoring. We present a novel Q-learning algorithm that is adjusted for censored data and allows a flexible number of stages. We provide finite sample bounds on the generalization error of the policy learned by the algorithm, and show that when the optimal Q-function belongs to the approximation space, the expected survival time for policies obtained by the algorithm converges to that of the optimal policy. We simulate a multistage clinical trial with flexible number of stages and apply the proposed censored-Q-learning algorithm to find individualized treatment regimens. The methodology presented in this paper has implications in the design of personalized medicine trials in cancer and in other life-threatening diseases. |
|---|---|
| AbstractList | We develop methodology for a multistage-decision problem with flexible number of stages in which the rewards are survival times that are subject to censoring. We present a novel Q-learning algorithm that is adjusted for censored data and allows a flexible number of stages. We provide finite sample bounds on the generalization error of the policy learned by the algorithm, and show that when the optimal Q-function belongs to the approximation space, the expected survival time for policies obtained by the algorithm converges to that of the optimal policy. We simulate a multistage clinical trial with flexible number of stages and apply the proposed censored-Q-learning algorithm to find individualized treatment regimens. The methodology presented in this paper has implications in the design of personalized medicine trials in cancer and in other life-threatening diseases.We develop methodology for a multistage-decision problem with flexible number of stages in which the rewards are survival times that are subject to censoring. We present a novel Q-learning algorithm that is adjusted for censored data and allows a flexible number of stages. We provide finite sample bounds on the generalization error of the policy learned by the algorithm, and show that when the optimal Q-function belongs to the approximation space, the expected survival time for policies obtained by the algorithm converges to that of the optimal policy. We simulate a multistage clinical trial with flexible number of stages and apply the proposed censored-Q-learning algorithm to find individualized treatment regimens. The methodology presented in this paper has implications in the design of personalized medicine trials in cancer and in other life-threatening diseases. We develop methodology for a multistage-decision problem with flexible number of stages in which the rewards are survival times that are subject to censoring. We present a novel Q-learning algorithm that is adjusted for censored data and allows a flexible number of stages. We provide finite sample bounds on the generalization error of the policy learned by the algorithm, and show that when the optimal Q-function belongs to the approximation space, the expected survival time for policies obtained by the algorithm converges to that of the optimal policy. We simulate a multistage clinical trial with flexible number of stages and apply the proposed censored-Q-learning algorithm to find individualized treatment regimens. The methodology presented in this paper has implications in the design of personalized medicine trials in cancer and in other life-threatening diseases. |
| Author | Goldberg, Yair Kosorok, Michael R |
| Author_xml | – sequence: 1 givenname: Yair surname: Goldberg fullname: Goldberg, Yair organization: Department of Biostatistics, The University of North Carolina At Chapel Hill, Chapel Hill, NC 27599, U.S.A – sequence: 2 givenname: Michael R surname: Kosorok fullname: Kosorok, Michael R |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/22754029$$D View this record in MEDLINE/PubMed |
| BookMark | eNo1j8tOg0AUQGdRYx-68AM0LN2MzutemCWh2JIQiC3GJZnCkNTwqExZ9O81sa7O5uQkZ0lm_dBbQh44e-GCq1cuaJjvNQYzsmBMMwoS1ZwsnftijIFW8pbMhfBBMaEX5PGdpnG4y5Js430mxdaL4myf7-K1tw6L8I7cNKZ19v7KFfl4i4toS9N8k0RhSiul_DM96BoDA0xK8Ctp0DbKt6qRSvooZIAoKw0cABsfK8QaoNacATNSyEroRqzI81_3NA7fk3Xnsju6yrat6e0wuZIHAgG0APxVn67qdOhsXZ7GY2fGS_m_JH4ANC1Fwg |
| CitedBy_id | crossref_primary_10_1146_annurev_statistics_022513_115553 crossref_primary_10_1093_biomet_asu050 crossref_primary_10_1093_biomet_asy017 crossref_primary_10_1002_bimj_201700181 crossref_primary_10_1007_s40501_015_0050_9 crossref_primary_10_1111_biom_12627 crossref_primary_10_1038_s41409_020_0871_z crossref_primary_10_1002_bimj_202200285 crossref_primary_10_1002_sim_6558 crossref_primary_10_1002_sim_8735 crossref_primary_10_1177_09622802241236954 crossref_primary_10_1177_1740774514525691 crossref_primary_10_1111_biom_13711 crossref_primary_10_1016_j_spl_2012_03_023 crossref_primary_10_1111_biom_12743 crossref_primary_10_1007_s10985_023_09605_8 crossref_primary_10_1080_01621459_2016_1155993 crossref_primary_10_1002_sim_9543 crossref_primary_10_3390_axioms13040212 crossref_primary_10_1158_1078_0432_CCR_17_1355 crossref_primary_10_3390_cancers13184624 crossref_primary_10_1002_sim_8976 crossref_primary_10_1080_01621459_2021_2008402 crossref_primary_10_1002_sim_9589 crossref_primary_10_1016_j_spl_2025_110357 crossref_primary_10_1111_biom_13084 crossref_primary_10_6339_23_JDS1107 crossref_primary_10_3390_stats4040046 crossref_primary_10_1111_rssb_12201 crossref_primary_10_1080_03610926_2020_1808686 crossref_primary_10_1080_01621459_2017_1330204 crossref_primary_10_1080_01621459_2014_937488 crossref_primary_10_1007_s13042_023_01869_8 crossref_primary_10_1146_annurev_statistics_030718_105251 crossref_primary_10_1002_sim_10223 crossref_primary_10_1007_s10985_022_09554_8 crossref_primary_10_1111_biom_12539 crossref_primary_10_1007_s12561_024_09471_4 crossref_primary_10_1093_aje_kwz272 crossref_primary_10_1093_biomet_asy043 crossref_primary_10_1111_biom_12894 crossref_primary_10_1080_01621459_2020_1862671 crossref_primary_10_1111_biom_13872 crossref_primary_10_1080_01621459_2019_1672557 crossref_primary_10_1111_insr_12583 crossref_primary_10_1145_3477600 crossref_primary_10_1002_sim_9198 crossref_primary_10_1007_s11684_013_0245_7 crossref_primary_10_1093_biomet_asac047 crossref_primary_10_1093_biostatistics_kxae002 crossref_primary_10_1080_01621459_2015_1086353 crossref_primary_10_1093_biostatistics_kxz042 crossref_primary_10_1007_s10985_022_09566_4 crossref_primary_10_1186_s12874_022_01811_6 crossref_primary_10_1111_rssa_12250 crossref_primary_10_1214_17_EJS1231 crossref_primary_10_1002_sim_6859 crossref_primary_10_1080_00949655_2020_1793341 crossref_primary_10_1093_jrsssa_qnaf123 crossref_primary_10_1002_sim_8678 crossref_primary_10_1080_01621459_2022_2108816 crossref_primary_10_1177_0962280220959118 crossref_primary_10_1002_sim_8473 crossref_primary_10_1080_01621459_2019_1629939 crossref_primary_10_1093_biostatistics_kxy062 crossref_primary_10_1214_24_AOAS1984 crossref_primary_10_1080_01621459_2020_1863224 crossref_primary_10_1002_sim_6783 crossref_primary_10_1080_01621459_2016_1200914 crossref_primary_10_1111_rssc_12266 crossref_primary_10_1080_01621459_2017_1321545 crossref_primary_10_1177_1740774514532570 crossref_primary_10_1080_01621459_2022_2152343 crossref_primary_10_1002_sim_6104 crossref_primary_10_1016_j_artmed_2020_101964 crossref_primary_10_1093_biomtc_ujaf082 crossref_primary_10_1186_s40345_014_0018_5 crossref_primary_10_1080_01621459_2018_1529597 crossref_primary_10_1177_09622802241262525 crossref_primary_10_1002_sim_8363 crossref_primary_10_1080_01621459_2022_2068420 crossref_primary_10_1108_IR_01_2019_0002 crossref_primary_10_1109_TNNLS_2020_2981377 |
| ContentType | Journal Article |
| DBID | NPM 7X8 |
| DOI | 10.1214/12-AOS968 |
| DatabaseName | PubMed MEDLINE - Academic |
| DatabaseTitle | PubMed MEDLINE - Academic |
| DatabaseTitleList | MEDLINE - Academic PubMed |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | no_fulltext_linktorsrc |
| Discipline | Statistics Mathematics |
| ExternalDocumentID | 22754029 |
| Genre | Journal Article |
| GrantInformation_xml | – fundername: NCI NIH HHS grantid: P01 CA142538 |
| GroupedDBID | -~X 123 23M 2AX 2FS 2WC 3R3 5RE 6J9 85S AAFWJ AAWIL AAYJJ ABAWQ ABBHK ABFAN ABPFR ABPQH ABXSQ ABYWD ABZEH ACGFO ACHJO ACIPV ACIWK ACMTB ACNCT ACTMH ACUBG ADLSF ADNWM ADODI ADULT AECCQ AENEX AETVE AEUPB AFFOW AFVYC AFXHP AGLNM AIHAF ALMA_UNASSIGNED_HOLDINGS ALRMG CJ0 CS3 D0L DQDLB DSRWC E3Z EBS ECEWR EJD F5P GR0 HDK HQ6 IPSME JAAYA JAS JBMMH JBZCM JENOY JHFFW JKQEH JLEZI JLXEF JMS JPL JST L7B N9A NPM OFU OK1 P2P PQQKQ PUASD RBU REI RNS RPE SA0 SJN TN5 TR2 UPT WH7 WS9 XSW ZCG ZY4 7X8 AFHLI |
| ID | FETCH-LOGICAL-c447t-b9d68a503357c3a6ef47e4f34376238663c951556f76c66d55d91050a323c29f2 |
| IEDL.DBID | 7X8 |
| ISICitedReferencesCount | 101 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000304684900020&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0090-5364 |
| IngestDate | Thu Oct 02 18:36:52 EDT 2025 Sat May 31 02:05:16 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c447t-b9d68a503357c3a6ef47e4f34376238663c951556f76c66d55d91050a323c29f2 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| PMID | 22754029 |
| PQID | 1826559256 |
| PQPubID | 23479 |
| ParticipantIDs | proquest_miscellaneous_1826559256 pubmed_primary_22754029 |
| PublicationCentury | 2000 |
| PublicationDate | 2012-02-01 |
| PublicationDateYYYYMMDD | 2012-02-01 |
| PublicationDate_xml | – month: 02 year: 2012 text: 2012-02-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States |
| PublicationTitle | The Annals of statistics |
| PublicationTitleAlternate | Ann Stat |
| PublicationYear | 2012 |
| SSID | ssj0005943 |
| Score | 2.4170566 |
| Snippet | We develop methodology for a multistage-decision problem with flexible number of stages in which the rewards are survival times that are subject to censoring.... |
| SourceID | proquest pubmed |
| SourceType | Aggregation Database Index Database |
| StartPage | 529 |
| Title | Q-LEARNING WITH CENSORED DATA |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/22754029 https://www.proquest.com/docview/1826559256 |
| Volume | 40 |
| WOSCitedRecordID | wos000304684900020&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpZ3LT8JAEMY3Kh7w4ANfqJiaeN3Qbrfb7sk0UIRECgJGbs2yj2NBUf9-Z9siJxMTL71t0kx3Z37dmXwfQveBNp7rigUmjBhMF67EgjCBgV2lIVpJRlRhNhGmaTSf83F14bauxio3ObFI1Gop7R1523Iw0C9U6IfVG7auUba7Wllo7KKaDyhjD2Y436qFB5upOe7iwGe0UhYiHm17BMejKWfR72RZVJje0X_f7RgdVmzpxOVmOEE7Om-gg-GPMOu6geoWLktt5lPUesZPSTxJB-mj8zqY9Z1Okk5Hk6TrdONZfIZeesms08eVXQKWlIYfeMEVi4RtSwah9AXThoaaGp9CDoHCDGghuTV0YSZkkjEVBApYIXCFT3xJuCHnaC9f5voSOfDnGqlQcdeF5ZHV-As8RZTvMQF8EIkmutsEIoPtaHsMItfLz3W2DUUTXZTRzFalbkZGSAh8SPjVH1ZfozqgCSnno29QzcBh1C20L78gSO-3xXeGZzoefgOVRKvv |
| linkProvider | ProQuest |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Q-LEARNING+WITH+CENSORED+DATA&rft.jtitle=The+Annals+of+statistics&rft.au=Goldberg%2C+Yair&rft.au=Kosorok%2C+Michael+R&rft.date=2012-02-01&rft.issn=0090-5364&rft.volume=40&rft.issue=1&rft.spage=529&rft_id=info:doi/10.1214%2F12-AOS968&rft_id=info%3Apmid%2F22754029&rft_id=info%3Apmid%2F22754029&rft.externalDocID=22754029 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0090-5364&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0090-5364&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0090-5364&client=summon |