Efficient Algorithms and Implementation of a Semiparametric Joint Model for Longitudinal and Competing Risks Data: With Applications to Massive Biobank Data

Semiparametric joint models of longitudinal and competing risks data are computationally costly and their current implementations do not scale well to massive biobank data. This paper identifies and addresses some key computational barriers in a semiparametric joint model for longitudinal and compet...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org
Main Authors: Li, Shanpeng, Li, Ning, Wang, Hong, Zhou, Jin, Zhou, Hua, Li, Gang
Format: Paper
Language:English
Published: Ithaca Cornell University Library, arXiv.org 08.02.2022
Subjects:
Algorithms
Biobanks
Numerical integration
Run time (computers)
Standard error
ISSN:2331-8422
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Semiparametric joint models of longitudinal and competing risks data are computationally costly and their current implementations do not scale well to massive biobank data. This paper identifies and addresses some key computational barriers in a semiparametric joint model for longitudinal and competing risks survival data. By developing and implementing customized linear scan algorithms, we reduce the computational complexities from \(O(n^2)\) or \(O(n^3)\) to \(O(n)\) in various components including numerical integration, risk set calculation, and standard error estimation, where \(n\) is the number of subjects. Using both simulated and real world biobank data, we demonstrate that these linear scan algorithms generate drastic speed-up of up to hundreds of thousands fold when \(n>10^4\), sometimes reducing the run-time from days to minutes. We have developed an R-package, FastJM, based on the proposed algorithms for joint modeling of longitudinal and time-to-event data with and without competing risks, and made it publicly available on the Comprehensive R Archive Network (CRAN).
AbstractList Semiparametric joint models of longitudinal and competing risks data are computationally costly and their current implementations do not scale well to massive biobank data. This paper identifies and addresses some key computational barriers in a semiparametric joint model for longitudinal and competing risks survival data. By developing and implementing customized linear scan algorithms, we reduce the computational complexities from \(O(n^2)\) or \(O(n^3)\) to \(O(n)\) in various components including numerical integration, risk set calculation, and standard error estimation, where \(n\) is the number of subjects. Using both simulated and real world biobank data, we demonstrate that these linear scan algorithms generate drastic speed-up of up to hundreds of thousands fold when \(n>10^4\), sometimes reducing the run-time from days to minutes. We have developed an R-package, FastJM, based on the proposed algorithms for joint modeling of longitudinal and time-to-event data with and without competing risks, and made it publicly available on the Comprehensive R Archive Network (CRAN).
Author Li, Shanpeng
Zhou, Jin
Li, Ning
Zhou, Hua
Li, Gang
Wang, Hong
Author_xml – sequence: 1
  givenname: Shanpeng
  surname: Li
  fullname: Li, Shanpeng
– sequence: 2
  givenname: Ning
  surname: Li
  fullname: Li, Ning
– sequence: 3
  givenname: Hong
  surname: Wang
  fullname: Wang, Hong
– sequence: 4
  givenname: Jin
  surname: Zhou
  fullname: Zhou, Jin
– sequence: 5
  givenname: Hua
  surname: Zhou
  fullname: Zhou, Hua
– sequence: 6
  givenname: Gang
  surname: Li
  fullname: Li, Gang
BookMark eNotkN1OAjEQhRujiYg8gHeTeA22XbpbvEPEv0BMlMRLMtsfLOy267YQH8aHdYNeTTLnnG9y5oKc-uANIVeMjsZSCHqD7bc7jDjrFmwsOT8hPZ5lbCjHnJ-TQYxbSinPCy5E1iM_c2udcsYnmFab0Lr0WUdAr-G5bipTdwImFzwECwjvpnYNtlib1DoFL8F1uWXQpgIbWlgEv3Fpr53H6siYhboxyfkNvLm4i3CPCW_hozsC06apnDqyI6QAS4zRHQzcuVCi3x2tl-TMYhXN4H_2yephvpo9DRevj8-z6WKIgvOhnijJJ0rzrNCZlkWZm1KyIlMGc0XLnDOhLTLJSsWktDSXRa6RK6Us1UIVWZ9c_2GbNnztTUzrbdi3XYe45kJKJibdw7Jfi3huog
ContentType Paper
Copyright 2022. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml – notice: 2022. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID 8FE
8FG
ABJCF
ABUWG
AFKRA
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
HCIFZ
L6V
M7S
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
DOI 10.48550/arxiv.2110.14822
DatabaseName ProQuest SciTech Collection
ProQuest Technology Collection
Materials Science & Engineering Collection
ProQuest Central (Alumni)
ProQuest Central UK/Ireland
ProQuest Central Essentials - QC
ProQuest Central
ProQuest Technology Collection
ProQuest One Community College
ProQuest Central Korea
SciTech Premium Collection
ProQuest Engineering Collection
Engineering Database
Proquest Central Premium
ProQuest One Academic (New)
ProQuest Publicly Available Content Database
ProQuest One Academic Middle East (New)
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Applied & Life Sciences
ProQuest One Academic (retired)
ProQuest One Academic UKI Edition
ProQuest Central China
Engineering Collection
DatabaseTitle Publicly Available Content Database
Engineering Database
Technology Collection
ProQuest One Academic Middle East (New)
ProQuest Central Essentials
ProQuest One Academic Eastern Edition
ProQuest Central (Alumni Edition)
SciTech Premium Collection
ProQuest One Community College
ProQuest Technology Collection
ProQuest SciTech Collection
ProQuest Central China
ProQuest Central
ProQuest One Applied & Life Sciences
ProQuest Engineering Collection
ProQuest One Academic UKI Edition
ProQuest Central Korea
Materials Science & Engineering Collection
ProQuest Central (New)
ProQuest One Academic
ProQuest One Academic (New)
Engineering Collection
DatabaseTitleList Publicly Available Content Database
Database_xml – sequence: 1
  dbid: PIMPY
  name: Publicly Available Content Database
  url: http://search.proquest.com/publiccontent
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Physics
EISSN 2331-8422
Genre Working Paper/Pre-Print
GroupedDBID 8FE
8FG
ABJCF
ABUWG
AFKRA
ALMA_UNASSIGNED_HOLDINGS
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
FRJ
HCIFZ
L6V
M7S
M~E
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
ID FETCH-LOGICAL-a522-d9c829cd237d3d87b6eb8173cea6c0b6215dfa181bc188f06876da2cccf0d5c73
IEDL.DBID M7S
IngestDate Mon Jun 30 09:27:05 EDT 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a522-d9c829cd237d3d87b6eb8173cea6c0b6215dfa181bc188f06876da2cccf0d5c73
Notes SourceType-Working Papers-1
ObjectType-Working Paper/Pre-Print-1
content type line 50
OpenAccessLink https://www.proquest.com/docview/2588159002?pq-origsite=%requestingapplication%
PQID 2588159002
PQPubID 2050157
ParticipantIDs proquest_journals_2588159002
PublicationCentury 2000
PublicationDate 20220208
PublicationDateYYYYMMDD 2022-02-08
PublicationDate_xml – month: 02
  year: 2022
  text: 20220208
  day: 08
PublicationDecade 2020
PublicationPlace Ithaca
PublicationPlace_xml – name: Ithaca
PublicationTitle arXiv.org
PublicationYear 2022
Publisher Cornell University Library, arXiv.org
Publisher_xml – name: Cornell University Library, arXiv.org
SSID ssj0002672553
Score 1.784793
SecondaryResourceType preprint
Snippet Semiparametric joint models of longitudinal and competing risks data are computationally costly and their current implementations do not scale well to massive...
SourceID proquest
SourceType Aggregation Database
SubjectTerms Algorithms
Biobanks
Numerical integration
Run time (computers)
Standard error
Title Efficient Algorithms and Implementation of a Semiparametric Joint Model for Longitudinal and Competing Risks Data: With Applications to Massive Biobank Data
URI https://www.proquest.com/docview/2588159002
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LT8MwDI6AgcSJt3jLB66FNV2TlAtiYwgQTBUgHieUJilUYy2sHeLH8GNxQnlISFw4Vq3byortL7bjj5AtmiqpQ91EEzfaa3G0Odw2U88gdAgFCkWuN-fqlPd64uYmiuuEW1m3VX76ROeodaFsjnyHhkL4luKS7j09e5Y1ylZXawqNcdKwUxJ817p38ZVjoYwjYg4-ipludNeOHL5mL9t217NtJ2DSXy7YxZXDmf_-0SxpxPLJDOfImMnnyZTr51TlAnnrutEQGFFg__EepaqHQQky1-DGAQ_qE0c5FClIuDCDzM4AH1h6LQUnRYZyliXtERDTwmlhSY1G2hJouXd0HNjGoAfnWdkv4UBWcheu8SOw_6MgDlUBZwjO0aFCO0PHkffdo4vk8rB72TnyaiIGTyI883SkBI2UpgHXgRY8YSYRPg-UkUw1E4aa16lEqJAoX4i0ydDDakmVUmlTh4oHS2QiL3KzTED6RkaUM5PysJUyJpk0JqRGqEglkpsVsv6p67vamMq7b0Wv_n17jUxTezrBNlWLdTJRDUdmg0yqlyorh5uk0e724vNNt0bwKj4-i2_fAfCXyzM
linkProvider ProQuest
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1LT9wwEB7xKIITtAXx7hzaY2DXIbZTCSGegu6yQmUF3FaO7UAEm9BNeP0X_gL_kbHZbStV6m0PnBM7kWc834w9Mx_AV5ZqZSJToy1uTbAhaM9R2MwCS65DJGlQ7HNzzpqi1ZIXF_HJCLwMamFcWuXAJnpDbQrtzsjXWSRl3VFcsq3bX4FjjXK3qwMKjTe1aNinBwrZys2jPZLvN8YO9tu7h0GfVSBQ5GsEJtaSxdqwUJjQSJFwm8i6CLVVXNcSThBoUkW4l-i6lGmNk7kwimmt05qJtAhp2lEYJy-CxT5T8PT3kQ7jghz08O3u1HcKW1e9x-x-zQVZa67hJvvH4nsYO5h-ZwswA-Mn6tb2PsKIzT_BhM9W1eVneN73jS8IL3H75pJ-srrqlqhyg77ZcbdfT5VjkaLCU9vNXIfzriMP0_ijyGic44C7QfLYsVk4yqY74-jB_By7PpQgSMefWXld4p6q1Hc8p4_g9l_X_VgVeEyhB8EF7mRkFvNr_-ostIexHnMwlhe5nQdUdatiJrhNRbSRcq64sjZiVupYJ0rYBVgeiLbTNxVl549cF___-AtMHraPm53mUauxBFPM1WG49HG5DGNV786uwAd9X2Vlb9WrJUJnyFrwCiBLJXc
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Efficient+Algorithms+and+Implementation+of+a+Semiparametric+Joint+Model+for+Longitudinal+and+Competing+Risks+Data%3A+With+Applications+to+Massive+Biobank+Data&rft.jtitle=arXiv.org&rft.au=Li%2C+Shanpeng&rft.au=Li%2C+Ning&rft.au=Wang%2C+Hong&rft.au=Zhou%2C+Jin&rft.date=2022-02-08&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422&rft_id=info:doi/10.48550%2Farxiv.2110.14822