Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activity
With the recent proliferation of mobile health technologies, health scientists are increasingly interested in developing just-in-time adaptive interventions (JITAIs), typically delivered via notifications on mobile devices and designed to help users prevent negative health outcomes and to promote th...
Uloženo v:
| Vydáno v: | Proceedings of ACM on interactive, mobile, wearable and ubiquitous technologies Ročník 4; číslo 1 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
United States
01.03.2020
|
| Témata: | |
| ISSN: | 2474-9567, 2474-9567 |
| On-line přístup: | Zjistit podrobnosti o přístupu |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | With the recent proliferation of mobile health technologies, health scientists are increasingly interested in developing just-in-time adaptive interventions (JITAIs), typically delivered via notifications on mobile devices and designed to help users prevent negative health outcomes and to promote the adoption and maintenance of healthy behaviors. A JITAI involves a sequence of decision rules (i.e., treatment policies) that take the user's current context as input and specify whether and what type of intervention should be provided at the moment. In this work, we describe a reinforcement learning (RL) algorithm that continuously learns and improves the treatment policy embedded in the JITAI as data is being collected from the user. This work is motivated by our collaboration on designing an RL algorithm for HeartSteps V2 based on data collected HeartSteps V1. HeartSteps is a physical activity mobile health application. The RL algorithm developed in this work is being used in HeartSteps V2 to decide, five times per day, whether to deliver a context-tailored activity suggestion. |
|---|---|
| AbstractList | With the recent proliferation of mobile health technologies, health scientists are increasingly interested in developing just-in-time adaptive interventions (JITAIs), typically delivered via notifications on mobile devices and designed to help users prevent negative health outcomes and to promote the adoption and maintenance of healthy behaviors. A JITAI involves a sequence of decision rules (i.e., treatment policies) that take the user's current context as input and specify whether and what type of intervention should be provided at the moment. In this work, we describe a reinforcement learning (RL) algorithm that continuously learns and improves the treatment policy embedded in the JITAI as data is being collected from the user. This work is motivated by our collaboration on designing an RL algorithm for HeartSteps V2 based on data collected HeartSteps V1. HeartSteps is a physical activity mobile health application. The RL algorithm developed in this work is being used in HeartSteps V2 to decide, five times per day, whether to deliver a context-tailored activity suggestion.With the recent proliferation of mobile health technologies, health scientists are increasingly interested in developing just-in-time adaptive interventions (JITAIs), typically delivered via notifications on mobile devices and designed to help users prevent negative health outcomes and to promote the adoption and maintenance of healthy behaviors. A JITAI involves a sequence of decision rules (i.e., treatment policies) that take the user's current context as input and specify whether and what type of intervention should be provided at the moment. In this work, we describe a reinforcement learning (RL) algorithm that continuously learns and improves the treatment policy embedded in the JITAI as data is being collected from the user. This work is motivated by our collaboration on designing an RL algorithm for HeartSteps V2 based on data collected HeartSteps V1. HeartSteps is a physical activity mobile health application. The RL algorithm developed in this work is being used in HeartSteps V2 to decide, five times per day, whether to deliver a context-tailored activity suggestion. With the recent proliferation of mobile health technologies, health scientists are increasingly interested in developing just-in-time adaptive interventions (JITAIs), typically delivered via notifications on mobile devices and designed to help users prevent negative health outcomes and to promote the adoption and maintenance of healthy behaviors. A JITAI involves a sequence of decision rules (i.e., treatment policies) that take the user's current context as input and specify whether and what type of intervention should be provided at the moment. In this work, we describe a reinforcement learning (RL) algorithm that continuously learns and improves the treatment policy embedded in the JITAI as data is being collected from the user. This work is motivated by our collaboration on designing an RL algorithm for HeartSteps V2 based on data collected HeartSteps V1. HeartSteps is a physical activity mobile health application. The RL algorithm developed in this work is being used in HeartSteps V2 to decide, five times per day, whether to deliver a context-tailored activity suggestion. |
| Author | Greenewald, Kristjan Murphy, Susan Klasnja, Predrag Liao, Peng |
| Author_xml | – sequence: 1 givenname: Peng surname: Liao fullname: Liao, Peng organization: University of Michigan – sequence: 2 givenname: Kristjan surname: Greenewald fullname: Greenewald, Kristjan organization: IBM Research – sequence: 3 givenname: Predrag surname: Klasnja fullname: Klasnja, Predrag organization: University of Michigan – sequence: 4 givenname: Susan surname: Murphy fullname: Murphy, Susan organization: Harvard University |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/34527853$$D View this record in MEDLINE/PubMed |
| BookMark | eNpNkF1LwzAUhoNM3JzDfyC59KaatGnTeFeGOmGw4cedUJLsdIu0aW1SYfv1RpzgORfvgefhvTjnaGRbCwhdUnJDKUtvkySnhPATNIkZZ5FIMz76d4_RzLkPQggVwST8DI0TlsY8T5MJel9D71ora3OADV6A7P2Lh87d4QI_g7FV22towHq8DMwau8VFvW1743cNDhCvOm8ac_gB693eGS1rXGhvvozfX6DTStYOZsecoreH-9f5IlquHp_mxTLSCRc-2mQiYUpnSgNVGVU6DK-4IFqpKpfAhYK8kikXQoJQkmrNqExjqXgeZBZP0fVvb9e3nwM4XzbGaahraaEdXBmnPGFhcx7Uq6M6qAY2ZdebRvb78u8j8TcQXmYS |
| CitedBy_id | crossref_primary_10_1093_jamia_ocab001 crossref_primary_10_1145_3661143 crossref_primary_10_1145_3637351 crossref_primary_10_1145_3448117 crossref_primary_10_2196_38342 crossref_primary_10_3390_ijerph19042267 crossref_primary_10_1002_sim_9748 crossref_primary_10_1109_JIOT_2023_3331715 crossref_primary_10_3390_ijerph18116059 crossref_primary_10_2196_47774 crossref_primary_10_1214_25_AOS2501 crossref_primary_10_1038_s41539_024_00289_9 crossref_primary_10_1093_abm_kaac051 crossref_primary_10_2196_24494 crossref_primary_10_3389_fdgth_2023_1215187 crossref_primary_10_1145_3686925 crossref_primary_10_1007_s10916_021_01773_0 crossref_primary_10_3389_fdgth_2025_1435917 crossref_primary_10_1038_s41746_020_0302_y crossref_primary_10_3390_a15080255 crossref_primary_10_2196_52161 crossref_primary_10_1161_CIRCOUTCOMES_125_012416 crossref_primary_10_1145_3749476 crossref_primary_10_1177_02683962241280287 crossref_primary_10_1145_3432218 crossref_primary_10_2196_31327 crossref_primary_10_1111_insr_12583 crossref_primary_10_2196_77532 crossref_primary_10_1093_biomet_asab033 crossref_primary_10_1145_3596260 crossref_primary_10_1007_s10994_021_05995_8 crossref_primary_10_2196_65026 crossref_primary_10_1093_pnasnexus_pgaf246 crossref_primary_10_1145_3696425 crossref_primary_10_1287_msom_2023_0548 crossref_primary_10_1161_JAHA_123_030807 crossref_primary_10_1093_abm_kaab028 crossref_primary_10_1027_2151_2604_a000539 crossref_primary_10_1038_s41746_024_01028_5 crossref_primary_10_1177_09622802251366843 crossref_primary_10_1002_aaai_12126 crossref_primary_10_1002_aaai_12206 crossref_primary_10_2196_33568 crossref_primary_10_1214_25_AOAS2030 crossref_primary_10_2196_48974 crossref_primary_10_1080_08897077_2022_2060446 crossref_primary_10_1007_s10994_024_06526_x crossref_primary_10_1093_biostatistics_kxae025 crossref_primary_10_2196_49669 crossref_primary_10_3389_fphar_2022_1094281 |
| ContentType | Journal Article |
| DBID | NPM 7X8 |
| DOI | 10.1145/3381007 |
| DatabaseName | PubMed MEDLINE - Academic |
| DatabaseTitle | PubMed MEDLINE - Academic |
| DatabaseTitleList | MEDLINE - Academic PubMed |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | no_fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 2474-9567 |
| ExternalDocumentID | 34527853 |
| Genre | Journal Article |
| GrantInformation_xml | – fundername: NCI NIH HHS grantid: U01 CA229437 – fundername: NIBIB NIH HHS grantid: U54 EB020404 – fundername: NIDA NIH HHS grantid: P50 DA039838 – fundername: NIAAA NIH HHS grantid: R01 AA023187 |
| GroupedDBID | AAKMM AAYFX ACM ADPZR AIKLT ALMA_UNASSIGNED_HOLDINGS GUFHI LHSKQ NPM ROL 7X8 AEFXT |
| ID | FETCH-LOGICAL-c379t-d6934bc6bce1b61bcccc7f790cbbf8ae79be8fa5799ae9ba1cc41a52ab78bcc42 |
| IEDL.DBID | 7X8 |
| ISICitedReferencesCount | 86 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000908394100018&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2474-9567 |
| IngestDate | Fri Jul 11 08:12:00 EDT 2025 Thu Jan 02 22:39:52 EST 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Keywords | Mobile Health Reinforcement Learning Applied computing → Health care information systems Computing methodologies → Machine learning algorithms Just-in-Time Adaptive Intervention |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c379t-d6934bc6bce1b61bcccc7f790cbbf8ae79be8fa5799ae9ba1cc41a52ab78bcc42 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| PMID | 34527853 |
| PQID | 2573434387 |
| PQPubID | 23479 |
| ParticipantIDs | proquest_miscellaneous_2573434387 pubmed_primary_34527853 |
| PublicationCentury | 2000 |
| PublicationDate | 2020-03-01 |
| PublicationDateYYYYMMDD | 2020-03-01 |
| PublicationDate_xml | – month: 03 year: 2020 text: 2020-03-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States |
| PublicationTitle | Proceedings of ACM on interactive, mobile, wearable and ubiquitous technologies |
| PublicationTitleAlternate | Proc ACM Interact Mob Wearable Ubiquitous Technol |
| PublicationYear | 2020 |
| SSID | ssj0001933807 |
| Score | 2.5252182 |
| Snippet | With the recent proliferation of mobile health technologies, health scientists are increasingly interested in developing just-in-time adaptive interventions... |
| SourceID | proquest pubmed |
| SourceType | Aggregation Database Index Database |
| Title | Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activity |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/34527853 https://www.proquest.com/docview/2573434387 |
| Volume | 4 |
| WOSCitedRecordID | wos000908394100018&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV07T8MwELaAMsDA-1FeMhJrVOdpmwVFiKoLJUIgdUCq7ItTKtG0kMDQX885SVUWJCQyeHGsROfL5bvz3XeEXIVMC6bCyIm0QQfFSLSDyg8dASySwCSkvG42wft9MRjIpAm4FU1a5cImVoY6nYKNkXdQtXxbBSn4zezdsV2j7Olq00JjlbR8hDI2pYsPxDLGgt56XTHtBTxw0BXgdd0sOgFhx7fkVoz_Di2rX0x3-78vt0O2GnBJ41obdsmKyffI5g_KwX3ykizQ99yktId6XtpEr-KaxvTRVDSqUEUMacO8OqLx2wifVb5OKE7SB7Qxk_HcTiTNHtMY6h4UB-S5e_d023OaDgsO-FyWThpJP9AQaTCujlwNePGMSwZaZ0IZLrURmQq5tBzeWrkAgatCT2ku8ObAOyRr-TQ3x4SmbqZYCloZrQMJSoDHtFRKCBdSn-k2uVyIbogabI8lVG6mn8VwKbw2OarlP5zVVBtDPwg9joji5A-rT8mGZ53hKkHsjLQy_H7NOVmHr3JcfFxUqoFjP7n_BvdZx04 |
| linkProvider | ProQuest |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Personalized+HeartSteps%3A+A+Reinforcement+Learning+Algorithm+for+Optimizing+Physical+Activity&rft.jtitle=Proceedings+of+ACM+on+interactive%2C+mobile%2C+wearable+and+ubiquitous+technologies&rft.au=Liao%2C+Peng&rft.au=Greenewald%2C+Kristjan&rft.au=Klasnja%2C+Predrag&rft.au=Murphy%2C+Susan&rft.date=2020-03-01&rft.eissn=2474-9567&rft.volume=4&rft.issue=1&rft_id=info:doi/10.1145%2F3381007&rft_id=info%3Apmid%2F34527853&rft_id=info%3Apmid%2F34527853&rft.externalDocID=34527853 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2474-9567&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2474-9567&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2474-9567&client=summon |