In EDS ansehen

Blood Glucose Prediction Algorithms Require Clinically Relevant Performance Criteria Beyond Accuracy.

Gespeichert in:
Bibliographische Detailangaben
Titel: Blood Glucose Prediction Algorithms Require Clinically Relevant Performance Criteria Beyond Accuracy.
Autoren: Wolff MK; Department of ICT and Natural Sciences, Norwegian University of Science and Technology, Ålesund, Norway., Schaathun HG; Department of ICT and Natural Sciences, Norwegian University of Science and Technology, Ålesund, Norway., Gros S; Department of Engineering Cybernetics, Norwegian University of Science and Technology, Trondheim, Norway., Volden R; Department of ICT and Natural Sciences, Norwegian University of Science and Technology, Ålesund, Norway., Steinert M; Department of Mechanical and Industrial Engineering, Norwegian University of Science and Technology, Trondheim, Norway., Fougner AL; Department of Engineering Cybernetics, Norwegian University of Science and Technology, Trondheim, Norway.
Quelle: Diabetes technology & therapeutics [Diabetes Technol Ther] 2025 Oct; Vol. 27 (10), pp. 858-870. Date of Electronic Publication: 2025 Apr 29.
Publikationsart: Journal Article
Sprache: English
Info zur Zeitschrift: Publisher: Mary Ann Liebert, Inc Country of Publication: United States NLM ID: 100889084 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1557-8593 (Electronic) Linking ISSN: 15209156 NLM ISO Abbreviation: Diabetes Technol Ther Subsets: MEDLINE
Imprint Name(s): Original Publication: Larchmont, NY : Mary Ann Liebert, Inc., c1999-
MeSH-Schlagworte: Blood Glucose*/analysis , Algorithms* , Diabetes Mellitus*/blood, Humans ; Blood Glucose Self-Monitoring ; Hypoglycemia ; Hyperglycemia/blood ; Prediction Algorithms
Abstract: Background: The root mean squared error (RMSE) is commonly used to evaluate blood glucose prediction algorithms. However, it primarily measures how well predictions align with the most likely future values, rather than supporting optimal and proactive treatment decisions. Since diabetes management data predominantly features blood glucose values within the target range, RMSE tends to favor models that consistently predict target-range values, often at the expense of detecting clinically critical events such as rapid fluctuations, hypoglycemia, or hyperglycemia. This study examines how and why RMSE biases evaluations toward trivial models, highlighting the need for alternative performance criteria that better reflect clinical priorities. Methods: We developed the composite glucose prediction metric (CGPM) to integrate three components: RMSE, temporal gain and geometric mean (glycemic event prediction). A custom loss function was designed to emphasize clinically critical predictions during model training. Pareto frontier analysis was used to assess trade-offs among models with comparable performance. Results: CGPM was computed for five blood glucose prediction techniques (zero-order hold, naïve linear regression, ridge regression, ridge regression trained with a custom loss function, and a physiology-based model) applied to the OhioT1DM dataset. The data-driven model with the lowest RMSE performed poorly on glycemic event prediction, highlighting RMSE's bias toward target-range predictions. In contrast, the ridge regressor trained with the custom loss function improved event prediction, showing that clinically weighted optimization mitigates biases. Conclusions: Blood glucose prediction algorithms require evaluation and optimization criteria beyond accuracy to better support optimal treatment decisions. This study introduced the CGPM as an alternative evaluation framework, along with a loss function designed for model optimization that emphasizes clinically critical but rare events. Further clinical validation is needed to refine these criteria and ensure they align more closely with the needs of diabetes management.
Contributed Indexing: Keywords: Pareto frontier analysis; blood glucose prediction; evaluation metric; loss function; machine learning; predictive modeling
Substance Nomenclature: 0 (Blood Glucose)
Entry Date(s): Date Created: 20250429 Date Completed: 20251007 Latest Revision: 20251007
Update Code: 20251007
DOI: 10.1089/dia.2025.0074
PMID: 40300777
Datenbank: MEDLINE
Beschreibung
Abstract:Background: The root mean squared error (RMSE) is commonly used to evaluate blood glucose prediction algorithms. However, it primarily measures how well predictions align with the most likely future values, rather than supporting optimal and proactive treatment decisions. Since diabetes management data predominantly features blood glucose values within the target range, RMSE tends to favor models that consistently predict target-range values, often at the expense of detecting clinically critical events such as rapid fluctuations, hypoglycemia, or hyperglycemia. This study examines how and why RMSE biases evaluations toward trivial models, highlighting the need for alternative performance criteria that better reflect clinical priorities. Methods: We developed the composite glucose prediction metric (CGPM) to integrate three components: RMSE, temporal gain and geometric mean (glycemic event prediction). A custom loss function was designed to emphasize clinically critical predictions during model training. Pareto frontier analysis was used to assess trade-offs among models with comparable performance. Results: CGPM was computed for five blood glucose prediction techniques (zero-order hold, naïve linear regression, ridge regression, ridge regression trained with a custom loss function, and a physiology-based model) applied to the OhioT1DM dataset. The data-driven model with the lowest RMSE performed poorly on glycemic event prediction, highlighting RMSE's bias toward target-range predictions. In contrast, the ridge regressor trained with the custom loss function improved event prediction, showing that clinically weighted optimization mitigates biases. Conclusions: Blood glucose prediction algorithms require evaluation and optimization criteria beyond accuracy to better support optimal treatment decisions. This study introduced the CGPM as an alternative evaluation framework, along with a loss function designed for model optimization that emphasizes clinically critical but rare events. Further clinical validation is needed to refine these criteria and ensure they align more closely with the needs of diabetes management.
ISSN:1557-8593
DOI:10.1089/dia.2025.0074