Zobrazit v EDS

Ensuring Model Performance Reliability through a Data-Centric Approach

Uloženo v:
Podrobná bibliografie
Název: Ensuring Model Performance Reliability through a Data-Centric Approach
Autoři: Sanka, Praveen Gupta, Minukuri, Vidya Sagar
Informace o vydavateli: Zenodo, 2025.
Rok vydání: 2025
Témata: model performance, data reliability, data quality
Popis: Businesses optimize ML models for marginal performance gains, but how often are the business decisions made with full awareness of data quality? The importance and the level of effort to maintain data quality is not new. However, the industry still lacks a standard way to quantify and monitor data quality. While companies rigorously optimize the models, data issues can quietly undermine performance, introduce bias and can lead to costly failures. For example, credit agency’s misreporting of inquiry counts and tradeline ages can contribute to major financial losses. This study introduces the Data Reliability Score (DRS), a longitudinal metric for assessing data quality across training and inference. Similar to performance metrics such as Accuracy and Mean Squared Error, DRS provides continuous monitoring across six key pillars: Lineage, Completeness, Consistency, Bias, Frequency, and Accuracy anchored in data driven methodology. By proactively identifying issues, DRS helps businesses ensure data reliability, preventing failures. Just as low-performing models aren’t deployed, data with a low DRS should not be trusted for making business decisions.
Druh dokumentu: Conference object
Jazyk: English
DOI: 10.5281/zenodo.17073070
DOI: 10.5281/zenodo.17073071
Rights: CC BY
Přístupové číslo: edsair.doi.dedup.....32ce9f67a4d938364c71e70307cefa1a
Databáze: OpenAIRE
Popis
Abstrakt:Businesses optimize ML models for marginal performance gains, but how often are the business decisions made with full awareness of data quality? The importance and the level of effort to maintain data quality is not new. However, the industry still lacks a standard way to quantify and monitor data quality. While companies rigorously optimize the models, data issues can quietly undermine performance, introduce bias and can lead to costly failures. For example, credit agency’s misreporting of inquiry counts and tradeline ages can contribute to major financial losses. This study introduces the Data Reliability Score (DRS), a longitudinal metric for assessing data quality across training and inference. Similar to performance metrics such as Accuracy and Mean Squared Error, DRS provides continuous monitoring across six key pillars: Lineage, Completeness, Consistency, Bias, Frequency, and Accuracy anchored in data driven methodology. By proactively identifying issues, DRS helps businesses ensure data reliability, preventing failures. Just as low-performing models aren’t deployed, data with a low DRS should not be trusted for making business decisions.
DOI:10.5281/zenodo.17073070