Parameter-Transferred Irreducible LSTM for Traffic Data Imputation
We propose an imputation algorithm for missing spatiotemporal data based on long short-term memory (LSTM) model factorization in a traffic environment where the roadside units (RSUs) collect traffic speed data. We considered a scenario where data collection by RSUs occurs for each road segment, but...
Gespeichert in:
| Veröffentlicht in: | IEEE sensors journal Jg. 24; H. 14; S. 22178 - 22188 |
|---|---|
| Hauptverfasser: | , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
New York
IEEE
15.07.2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Schlagworte: | |
| ISSN: | 1530-437X, 1558-1748 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | We propose an imputation algorithm for missing spatiotemporal data based on long short-term memory (LSTM) model factorization in a traffic environment where the roadside units (RSUs) collect traffic speed data. We considered a scenario where data collection by RSUs occurs for each road segment, but the absence of RSUs results in an incomplete dataset. To enhance imputation accuracy, we mitigate the risk of error propagation of model training on the entire dataset and take into account the spatiotemporal correlation of the dataset. The proposed algorithm can reduce the dimensionality of the input dataset by employing an adjacency matrix to identify data both highly correlated and connected to the target road segment, subsequently transforming parallel datasets into a serial format. Then, we extrapolate the missing data using an irreducible LSTM model, which is a factorization of a standard LSTM model. To enhance imputation performance, we also adopt spatial interpolation on extrapolated data across multiple paths that lead to the target road segment. Extensive experiment results using synthetic and real-world datasets confirm that the proposed algorithm outperforms other imputation algorithms in terms of imputation accuracy measured by the root mean square error (RMSE) and the mean absolute error (MAE) as well as the space complexity measured by the number of model parameters. In particular, the experiments with real-world datasets show that the proposed algorithm consistently achieves high imputation accuracy across a wide range of traffic scenarios, including actual traffic congestion and rapid traffic fluctuations. |
|---|---|
| AbstractList | We propose an imputation algorithm for missing spatiotemporal data based on long short-term memory (LSTM) model factorization in a traffic environment where the roadside units (RSUs) collect traffic speed data. We considered a scenario where data collection by RSUs occurs for each road segment, but the absence of RSUs results in an incomplete dataset. To enhance imputation accuracy, we mitigate the risk of error propagation of model training on the entire dataset and take into account the spatiotemporal correlation of the dataset. The proposed algorithm can reduce the dimensionality of the input dataset by employing an adjacency matrix to identify data both highly correlated and connected to the target road segment, subsequently transforming parallel datasets into a serial format. Then, we extrapolate the missing data using an irreducible LSTM model, which is a factorization of a standard LSTM model. To enhance imputation performance, we also adopt spatial interpolation on extrapolated data across multiple paths that lead to the target road segment. Extensive experiment results using synthetic and real-world datasets confirm that the proposed algorithm outperforms other imputation algorithms in terms of imputation accuracy measured by the root mean square error (RMSE) and the mean absolute error (MAE) as well as the space complexity measured by the number of model parameters. In particular, the experiments with real-world datasets show that the proposed algorithm consistently achieves high imputation accuracy across a wide range of traffic scenarios, including actual traffic congestion and rapid traffic fluctuations. |
| Author | Kwon, Jungmin Park, Hyunggon |
| Author_xml | – sequence: 1 givenname: Jungmin surname: Kwon fullname: Kwon, Jungmin email: jungmin.kwon@ewha.ac.kr organization: Department of Electronic and Electrical Engineering, Ewha Womans University, Seoul, South Korea – sequence: 2 givenname: Hyunggon orcidid: 0000-0002-5079-1504 surname: Park fullname: Park, Hyunggon email: hyunggon.park@ewha.ac.kr organization: Department of Electronic and Electrical Engineering and Smart Factory Multidisciplinary Program, Ewha Womans University, Seoul, South Korea |
| BookMark | eNp9kLtOwzAUhi1UJNrCAyAxRGJOsWMntkcoBYrKRWqR2CxfJVdtUmxn4O1JlA6IgcXHw_edyz8Bo7qpLQCXCM4Qgvzmeb14nRWwIDOMecExOwFjVJYsR5SwUf_HMCeYfp6BSYxbCBGnJR2Du3cZ5N4mG_JNkHV0NgRrsmX_ttqrnc1W681L5pqQdYBzXmf3MslsuT-0SSbf1Ofg1MldtBfHOgUfD4vN_ClfvT0u57erXBecpFwbBiU0SpWGM0e4rspKcWewIsRwAqXjlabcaK61VlJTVWGNOJSqMIpAi6fgeuh7CM1Xa2MS26YNdTdSYMgQo7C7vKPoQOnQxBisE9oPe6Yg_U4gKPrARB-Y6AMTx8A6E_0xD8HvZfj-17kaHG-t_cWXqCo4xT8GpHlO |
| CODEN | ISJEAZ |
| CitedBy_id | crossref_primary_10_1016_j_compeleceng_2024_109988 |
| Cites_doi | 10.1109/TITS.2021.3119638 10.1175/1520-0442(2000)013<2217:RTCSTC>2.0.CO;2 10.1109/JSEN.2017.2777786 10.1016/j.trc.2022.103826 10.1049/iet-its.2013.0052 10.1109/JIOT.2023.3297849 10.1016/j.knosys.2021.107114 10.18637/jss.v045.i03 10.1109/TBDATA.2022.3154097 10.24963/ijcai.2018/322 10.1016/j.knosys.2020.106705 10.1109/TITS.2018.2869768 10.1109/IWCMC.2016.7577067 10.1155/2019/7602792 10.1080/15472450.2020.1713772 10.3390/ijerph110909101 10.1080/13658816.2020.1725016 10.1016/j.trc.2021.103228 10.1016/j.trc.2020.102673 10.3390/en11040705 10.1007/s12652-018-0820-5 10.1109/LWC.2018.2795605 10.1016/j.neucom.2008.11.026 10.1016/j.trc.2022.103737 10.1371/journal.pone.0195957 10.1007/s10462-019-09709-4 10.1002/atr.180 10.21437/Interspeech.2016-1446 10.1109/JSEN.2020.3007809 10.1109/TITS.2020.3030546 10.1016/j.proeng.2012.09.545 10.1109/TVT.2018.2826067 10.1049/cp:19991218 10.1109/JIOT.2022.3171780 10.1109/ACCESS.2023.3264216 10.1016/j.trc.2016.09.015 10.1109/JSEN.2023.3263539 10.1007/s10618-022-00903-7 10.23919/JCC.2021.07.001 10.1109/TPAMI.2021.3066551 10.1109/JSEN.2021.3100324 10.1109/TITS.2019.2950416 10.1007/978-3-030-89880-9_34 10.3390/electronics8080896 10.1016/j.energy.2018.01.177 10.3141/1748-12 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024 |
| DBID | 97E RIA RIE AAYXX CITATION 7SP 7U5 8FD L7M |
| DOI | 10.1109/JSEN.2024.3392938 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Electronics & Communications Abstracts Solid State and Superconductivity Abstracts Technology Research Database Advanced Technologies Database with Aerospace |
| DatabaseTitle | CrossRef Solid State and Superconductivity Abstracts Technology Research Database Advanced Technologies Database with Aerospace Electronics & Communications Abstracts |
| DatabaseTitleList | Solid State and Superconductivity Abstracts |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Geography Engineering |
| EISSN | 1558-1748 |
| EndPage | 22188 |
| ExternalDocumentID | 10_1109_JSEN_2024_3392938 10516297 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Institute of Information and Communications Technology Planning and Evaluation through the Korean Government Development of Distributed/Cooperative AI-based 5G+ Network Data Analytics Functions and Control Technology grantid: 2021-0-00739 funderid: 10.13039/501100012389 |
| GroupedDBID | -~X 0R~ 29I 4.4 5GY 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK AENEX AGQYO AHBIQ AJQPL AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 EBS F5P HZ~ IFIPE IPLJI JAVBF LAI M43 O9- OCL P2P RIA RIE RNS TWZ AAYXX CITATION 7SP 7U5 8FD L7M |
| ID | FETCH-LOGICAL-c294t-cd80a0dbb5d98f49c656b9fd3b44d940af96c79dc9cccbac7b63c190ab2db40e3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 2 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001273156700104&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1530-437X |
| IngestDate | Mon Jun 30 07:51:22 EDT 2025 Tue Nov 18 21:47:44 EST 2025 Sat Nov 29 06:40:01 EST 2025 Wed Aug 27 02:03:43 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 14 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c294t-cd80a0dbb5d98f49c656b9fd3b44d940af96c79dc9cccbac7b63c190ab2db40e3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0002-5079-1504 |
| PQID | 3081870392 |
| PQPubID | 75733 |
| PageCount | 11 |
| ParticipantIDs | ieee_primary_10516297 proquest_journals_3081870392 crossref_citationtrail_10_1109_JSEN_2024_3392938 crossref_primary_10_1109_JSEN_2024_3392938 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-07-15 |
| PublicationDateYYYYMMDD | 2024-07-15 |
| PublicationDate_xml | – month: 07 year: 2024 text: 2024-07-15 day: 15 |
| PublicationDecade | 2020 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE sensors journal |
| PublicationTitleAbbrev | JSEN |
| PublicationYear | 2024 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 ref12 ref15 ref14 Karevan (ref36) 2018 ref10 Cao (ref39) ref17 ref16 Reza (ref44) ref19 ref18 ref51 ref50 ref46 ref45 ref48 ref47 ref42 ref41 ref43 ref49 ref8 Abadi (ref53) ref7 ref9 ref4 ref3 ref6 ref5 Kingma (ref52) 2014 ref40 ref35 ref34 ref31 ref30 ref33 ref32 ref2 ref1 ref38 ref24 ref23 ref26 ref25 ref20 ref22 ref21 ref28 Xingjian (ref37); 28 ref27 ref29 Subramaniam (ref11) |
| References_xml | – ident: ref41 doi: 10.1109/TITS.2021.3119638 – ident: ref46 doi: 10.1175/1520-0442(2000)013<2217:RTCSTC>2.0.CO;2 – ident: ref4 doi: 10.1109/JSEN.2017.2777786 – ident: ref20 doi: 10.1016/j.trc.2022.103826 – ident: ref8 doi: 10.1049/iet-its.2013.0052 – start-page: 187 volume-title: Proc. 32nd Int. Conf. Very Large Data Bases ident: ref11 article-title: Online outlier detection in sensor data using non-parametric models – ident: ref32 doi: 10.1109/JIOT.2023.3297849 – ident: ref29 doi: 10.1016/j.knosys.2021.107114 – ident: ref51 doi: 10.18637/jss.v045.i03 – ident: ref31 doi: 10.1109/TBDATA.2022.3154097 – ident: ref34 doi: 10.24963/ijcai.2018/322 – ident: ref43 doi: 10.1016/j.knosys.2020.106705 – ident: ref14 doi: 10.1109/TITS.2018.2869768 – ident: ref6 doi: 10.1109/IWCMC.2016.7577067 – ident: ref26 doi: 10.1155/2019/7602792 – ident: ref21 doi: 10.1080/15472450.2020.1713772 – ident: ref24 doi: 10.3390/ijerph110909101 – ident: ref25 doi: 10.1080/13658816.2020.1725016 – ident: ref45 doi: 10.1016/j.trc.2021.103228 – volume: 28 start-page: 802 volume-title: Proc. 28th Int. Conf. Neural Inf. Process. Syst. (NIPS) ident: ref37 article-title: Convolutional LSTM network: A machine learning approach for precipitation nowcasting – ident: ref28 doi: 10.1016/j.trc.2020.102673 – ident: ref16 doi: 10.3390/en11040705 – ident: ref1 doi: 10.1007/s12652-018-0820-5 – ident: ref17 doi: 10.1109/LWC.2018.2795605 – ident: ref50 doi: 10.1016/j.neucom.2008.11.026 – ident: ref30 doi: 10.1016/j.trc.2022.103737 – ident: ref23 doi: 10.1371/journal.pone.0195957 – ident: ref18 doi: 10.1007/s10462-019-09709-4 – ident: ref22 doi: 10.1002/atr.180 – year: 2014 ident: ref52 article-title: Adam: A method for stochastic optimization publication-title: arXiv:1412.6980 – ident: ref12 doi: 10.21437/Interspeech.2016-1446 – ident: ref38 doi: 10.1109/JSEN.2020.3007809 – start-page: 265 volume-title: Proc. 12th USENIX Symp. Oper. Syst. Design Implement. ident: ref53 article-title: TensorFlow: A system for large-scale machine learning – ident: ref15 doi: 10.1109/TITS.2020.3030546 – ident: ref49 doi: 10.1016/j.proeng.2012.09.545 – ident: ref7 doi: 10.1109/TVT.2018.2826067 – ident: ref33 doi: 10.1049/cp:19991218 – year: 2018 ident: ref36 article-title: Spatio-temporal stacked LSTM for temperature prediction in weather forecasting publication-title: arXiv:1811.06341 – ident: ref19 doi: 10.1109/JIOT.2022.3171780 – start-page: 6775 volume-title: Proc. Adv. Neural Inf. Process. Syst. (NeurIPS) ident: ref39 article-title: BRITS: Bidirectional recurrent imputation for time series – ident: ref10 doi: 10.1109/ACCESS.2023.3264216 – ident: ref9 doi: 10.1016/j.trc.2016.09.015 – ident: ref3 doi: 10.1109/JSEN.2023.3263539 – ident: ref42 doi: 10.1007/s10618-022-00903-7 – ident: ref2 doi: 10.23919/JCC.2021.07.001 – ident: ref27 doi: 10.1109/TPAMI.2021.3066551 – ident: ref40 doi: 10.1109/JSEN.2021.3100324 – ident: ref48 doi: 10.1109/TITS.2019.2950416 – ident: ref35 doi: 10.1007/978-3-030-89880-9_34 – volume-title: Proc. Int. Conf. Artif. Intell. ident: ref44 article-title: A convolutional recurrent autoencoder for spatio-temporal missing data imputation – ident: ref5 doi: 10.3390/electronics8080896 – ident: ref13 doi: 10.1016/j.energy.2018.01.177 – ident: ref47 doi: 10.3141/1748-12 |
| SSID | ssj0019757 |
| Score | 2.415746 |
| Snippet | We propose an imputation algorithm for missing spatiotemporal data based on long short-term memory (LSTM) model factorization in a traffic environment where... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 22178 |
| SubjectTerms | Accuracy Algorithms Complexity theory Data collection Data imputation Data models Datasets Error analysis extrapolation Factorization Interpolation Long short term memory long short-term memory (LSTM) Missing data parameter transfer Parameters Roads Roads & highways Roadsides Root-mean-square errors Segments Sensors spatial interpolation Spatiotemporal data Traffic congestion Traffic information Traffic speed Training |
| Title | Parameter-Transferred Irreducible LSTM for Traffic Data Imputation |
| URI | https://ieeexplore.ieee.org/document/10516297 https://www.proquest.com/docview/3081870392 |
| Volume | 24 |
| WOSCitedRecordID | wos001273156700104&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-1748 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0019757 issn: 1530-437X databaseCode: RIE dateStart: 20010101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8QwEB5UBPXgW1xf9OBJqKZN2jRHn6josrAKeyvJJMEFWWXtCv57kzTKgih4KT0kpcyXSb5kMt8AHGaZdjSAOQQqFCnjGlNZapEWxgvwWeNIeUD6jne71WAgejFZPeTCGGPC5TNz7F9DLF-_4MQflTkPL7IyF3wWZjkv22St75CB4EHW03kwSRnlgxjCzIg4ue1fdt1WMGfH1NMBn4sytQiFqio_puKwvlyt_PPPVmE5EsnktEV-DWbMaB2WpuQF12EhVjh_-tiAs57017CcFdOwPFkzHhud3PjnBIfq2SR3_Yf7xHHYxDXwwhLJhWxkcuOrPgT4NuHx6vLh_DqN9RNSzAVrUtQVkUQrVWhRWSbQcTclrKaKMS0YkVaUyIVGgYhKIlclRUcQpMq1YsTQLZgbvYzMNiTUcKOtVbKizuu1lJRmmVKSWml5iaYD5MugNUZxcV_j4rkOmwwiao9B7TGoIwYdOPru8toqa_zVeNMbfapha-8O7H3BVkfne6upl-lzM5nId37ptguL_uv-jDYr9mCuGU_MPszjezN8Gx-EcfUJ0vjLKQ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB58gXrwWbFadQ-ehNXsJrvZHH1itS2FVuhtSSYJClKltoL_3iRdpSAKXpY9JOwyXyb5ksl8A3CcJNrRAOYQKFDEjGuMZa5FnBkvwGeNI-UB6RbvdIrBQHSrZPWQC2OMCZfPzKl_DbF8_YITf1TmPDxL8lTweVjMGEvJNF3rO2ggeBD2dD5MYkb5oApiJkSc3fWuO24zmLJT6gmBz0aZWYZCXZUfk3FYYW7W__lvG7BWUcnofIr9JsyZ4RaszggMbsFyVeP88WMbLrrSX8RydozDAmXNaGR01PTPCT6pZxO1ev125Fhs5Bp4aYnoSo5l1PR1HwKANXi4ue5f3sZVBYUYU8HGMeqCSKKVyrQoLBPo2JsSVlPFmBaMSCty5EKjQEQlkaucoqMIUqVaMWLoDiwMX4ZmFyJquNHWKllQ5_daSkqTRClJrbQ8R1MH8mXQEit5cV_l4rkM2wwiSo9B6TEoKwzqcPLd5XWqrfFX45o3-kzDqb3r0PiCrazc762kXqjPzWUi3ful2xEs3_bbrbLV7Nzvw4r_kj-xTbIGLIxHE3MAS_g-fnobHYYx9gmNFc5w |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Parameter-Transferred+Irreducible+LSTM+for+Traffic+Data+Imputation&rft.jtitle=IEEE+sensors+journal&rft.au=Kwon%2C+Jungmin&rft.au=Park%2C+Hyunggon&rft.date=2024-07-15&rft.issn=1530-437X&rft.eissn=1558-1748&rft.volume=24&rft.issue=14&rft.spage=22178&rft.epage=22188&rft_id=info:doi/10.1109%2FJSEN.2024.3392938&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_JSEN_2024_3392938 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1530-437X&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1530-437X&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1530-437X&client=summon |