Multimodal integration of longitudinal noninvasive diagnostics for survival prediction in immunotherapy using deep learning
Gespeichert in:
| Titel: | Multimodal integration of longitudinal noninvasive diagnostics for survival prediction in immunotherapy using deep learning |
|---|---|
| Autoren: | Melda Yeghaian, Zuhir Bodalal, Daan van den Broek, John B A G Haanen, Regina G H Beets-Tan, Stefano Trebeschi, Marcel A J van Gerven |
| Quelle: | J Am Med Inform Assoc Jamia. Journal of the American Medical Informatics Association, 32, 8, pp. 1267-1275 Journal of the American Medical Informatics Association, vol. 32, no. 8, pp. 1267-1275 |
| Publication Status: | Preprint |
| Verlagsinformationen: | Oxford University Press (OUP), 2025. |
| Publikationsjahr: | 2025 |
| Schlagwörter: | FOS: Computer and information sciences, Computer Science - Machine Learning, Computer Science - Artificial Intelligence, longitudinal study, deep learning, Cognitive artificial intelligence, artificial intelligence, Research and Applications, Quantitative Biology - Quantitative Methods, Machine Learning (cs.LG), Artificial Intelligence (cs.AI), FOS: Biological sciences, multimodal data integration, immunotherapy, Humans, Deep Learning, Immunotherapy, Neoplasms/mortality, Neoplasms/therapy, Prognosis, Neural Networks, Computer, Male, Female, Quantitative Methods (q-bio.QM) |
| Beschreibung: | Objectives Immunotherapies have revolutionized the landscape of cancer treatments. However, our understanding of response patterns in advanced cancers treated with immunotherapy remains limited. By leveraging routinely collected noninvasive longitudinal and multimodal data with artificial intelligence, we could unlock the potential to transform immunotherapy for cancer patients, paving the way for personalized treatment approaches. Materials and Methods In this study, we developed a novel artificial neural network architecture, multimodal transformer-based simple temporal attention (MMTSimTA) network, building upon a combination of recent successful developments. We integrated pre- and on-treatment blood measurements, prescribed medications, and CT-based volumes of organs from a large pan-cancer cohort of 694 patients treated with immunotherapy to predict mortality at 3, 6, 9, and 12 months. Different variants of our extended MMTSimTA network were implemented and compared to baseline methods, incorporating intermediate and late fusion-based integration methods. Results The strongest prognostic performance was demonstrated using a variant of the MMTSimTA model with area under the curves of 0.84 ± 0.04, 0.83 ± 0.02, 0.82 ± 0.02, 0.81 ± 0.03 for 3-, 6-, 9-, and 12-month survival prediction, respectively. Discussion Our findings show that integrating noninvasive longitudinal data using our novel architecture yields an improved multimodal prognostic performance, especially in short-term survival prediction. Conclusion Our study demonstrates that multimodal longitudinal integration of noninvasive data using deep learning may offer a promising approach for personalized prognostication in immunotherapy-treated cancer patients. |
| Publikationsart: | Article Other literature type |
| Dateibeschreibung: | application/pdf |
| Sprache: | English |
| ISSN: | 1527-974X 1067-5027 |
| DOI: | 10.1093/jamia/ocaf074 |
| DOI: | 10.48550/arxiv.2411.18253 |
| Zugangs-URL: | http://arxiv.org/abs/2411.18253 https://cris.maastrichtuniversity.nl/en/publications/f4652115-db49-4ffb-bbd0-43ce79af8e4a https://doi.org/10.1093/jamia/ocaf074 https://hdl.handle.net/2066/312797 https://repository.ubn.ru.nl//bitstream/handle/2066/312797/312797.pdf http://nbn-resolving.org/urn/resolver.pl?urn=urn:nbn:ch:serval-BIB_C9F310DE15E50 https://serval.unil.ch/notice/serval:BIB_C9F310DE15E5 https://serval.unil.ch/resource/serval:BIB_C9F310DE15E5.P001/REF.pdf |
| Rights: | CC BY NC CC BY NC SA CC BY URL: http://creativecommons.org/licenses/by-nc/4.0/This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (http://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact reprints@oup.com for reprints and translation rights for reprints. All other permissions can be obtained through our RightsLink service via the Permissions link on the article page on our site—for further information please contact journals.permissions@oup.com. |
| Dokumentencode: | edsair.doi.dedup.....07a0f1e215e8f5aa7795150ddd19035d |
| Datenbank: | OpenAIRE |
Nájsť tento článok vo Web of Science | Abstract: | Objectives Immunotherapies have revolutionized the landscape of cancer treatments. However, our understanding of response patterns in advanced cancers treated with immunotherapy remains limited. By leveraging routinely collected noninvasive longitudinal and multimodal data with artificial intelligence, we could unlock the potential to transform immunotherapy for cancer patients, paving the way for personalized treatment approaches. Materials and Methods In this study, we developed a novel artificial neural network architecture, multimodal transformer-based simple temporal attention (MMTSimTA) network, building upon a combination of recent successful developments. We integrated pre- and on-treatment blood measurements, prescribed medications, and CT-based volumes of organs from a large pan-cancer cohort of 694 patients treated with immunotherapy to predict mortality at 3, 6, 9, and 12 months. Different variants of our extended MMTSimTA network were implemented and compared to baseline methods, incorporating intermediate and late fusion-based integration methods. Results The strongest prognostic performance was demonstrated using a variant of the MMTSimTA model with area under the curves of 0.84 ± 0.04, 0.83 ± 0.02, 0.82 ± 0.02, 0.81 ± 0.03 for 3-, 6-, 9-, and 12-month survival prediction, respectively. Discussion Our findings show that integrating noninvasive longitudinal data using our novel architecture yields an improved multimodal prognostic performance, especially in short-term survival prediction. Conclusion Our study demonstrates that multimodal longitudinal integration of noninvasive data using deep learning may offer a promising approach for personalized prognostication in immunotherapy-treated cancer patients. |
|---|---|
| ISSN: | 1527974X 10675027 |
| DOI: | 10.1093/jamia/ocaf074 |