LLMAir: Adaptive Reprogramming Large Language Model for Air Quality Prediction
Accurate and timely air quality prediction is crucial for cities and individuals to effectively take necessary precautions against potential air pollution. Existing studies typically rely on building prediction models based on large-scale monitoring data, often designed for specific tasks. Recently,...
Gespeichert in:
| Veröffentlicht in: | Proceedings - International Conference on Parallel and Distributed Systems S. 423 - 430 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Tagungsbericht |
| Sprache: | Englisch |
| Veröffentlicht: |
IEEE
10.10.2024
|
| Schlagworte: | |
| ISSN: | 2690-5965 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Accurate and timely air quality prediction is crucial for cities and individuals to effectively take necessary precautions against potential air pollution. Existing studies typically rely on building prediction models based on large-scale monitoring data, often designed for specific tasks. Recently, pre-trained large language models (LLMs) have achieved significant progress in various time series analysis tasks due to their powerful representation and inference capabilities. However, their application to air quality data with spatio-temporal features remains largely unexplored. In this work, we propose LLMAir, an adaptive reprogramming approach that adapts pre-trained LLMs for air quality prediction. We first construct spatiotemporal tokens based on monitoring stations by integrating value, node, and time embeddings. Next, we design an adaptive semantic-enhanced reprogramming module to compute similarity matching scores between our spatiotemporal tokens and pre-trained word embeddings for alignment. We employ a semantic regulator to generate the optimal length of word prototypes, which serve as prompt prefixes for adaptive reprogramming and guiding the spatiotemporal token embeddings into the frozen LLM. Additionally, we jointly optimize predictive error and alignment loss to train our model. Experimental results demonstrate that LLMAir achieves state-of-the-art performance in air quality prediction and few-shot forecasting across two real-world datasets. |
|---|---|
| AbstractList | Accurate and timely air quality prediction is crucial for cities and individuals to effectively take necessary precautions against potential air pollution. Existing studies typically rely on building prediction models based on large-scale monitoring data, often designed for specific tasks. Recently, pre-trained large language models (LLMs) have achieved significant progress in various time series analysis tasks due to their powerful representation and inference capabilities. However, their application to air quality data with spatio-temporal features remains largely unexplored. In this work, we propose LLMAir, an adaptive reprogramming approach that adapts pre-trained LLMs for air quality prediction. We first construct spatiotemporal tokens based on monitoring stations by integrating value, node, and time embeddings. Next, we design an adaptive semantic-enhanced reprogramming module to compute similarity matching scores between our spatiotemporal tokens and pre-trained word embeddings for alignment. We employ a semantic regulator to generate the optimal length of word prototypes, which serve as prompt prefixes for adaptive reprogramming and guiding the spatiotemporal token embeddings into the frozen LLM. Additionally, we jointly optimize predictive error and alignment loss to train our model. Experimental results demonstrate that LLMAir achieves state-of-the-art performance in air quality prediction and few-shot forecasting across two real-world datasets. |
| Author | Ma, Huadong Fan, Jinxiao Chu, Haolin Liu, Liang |
| Author_xml | – sequence: 1 givenname: Jinxiao surname: Fan fullname: Fan, Jinxiao email: jinxiaofan@bupt.edu.cn organization: Beijing University of Posts and Telecommunications,Beijing,China,100876 – sequence: 2 givenname: Haolin surname: Chu fullname: Chu, Haolin email: haolinchu@bupt.edu.cn organization: Beijing University of Posts and Telecommunications,Beijing,China,100876 – sequence: 3 givenname: Liang surname: Liu fullname: Liu, Liang email: liangliu@bupt.edu.cn organization: Beijing University of Posts and Telecommunications,Beijing,China,100876 – sequence: 4 givenname: Huadong surname: Ma fullname: Ma, Huadong email: mhd@bupt.edu.cn organization: Beijing University of Posts and Telecommunications,Beijing,China,100876 |
| BookMark | eNotj8tOwzAURA0Cibb0DxDyD6Rcx7Eds4tKgUoplNe6uk5uIqM8KjdF6t8TCTYzZ3NGmim76PqOGLsVsBAC7N16uc0ePrSUChYxxMkCAHR8xubW2FRKoYSyWp-zSawtRCOrKzY9HL4BYhidCXvJ803mwz3PStwP_of4O-1DXwdsW9_VPMdQ05hdfcQRNn1JDa_6wEeJvx2x8cOJbwOVvhh8312zywqbA83_e8a-Hlefy-cof31aL7M88sLoIYqxFK4i0BJROpsYoW0qHCGlRQWlcsIkgkRSKKfGC65Ak2pMlZNUpVg6OWM3f7ueiHb74FsMp50Ao6VJQP4C1hdRsA |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/ICPADS63350.2024.00062 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISBN | 9798331515966 |
| EISSN | 2690-5965 |
| EndPage | 430 |
| ExternalDocumentID | 10763740 |
| Genre | orig-research |
| GroupedDBID | 29O 6IE 6IF 6IH 6IK 6IL 6IM 6IN AAJGR AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IPLJI OCL RIE RIL RNS |
| ID | FETCH-LOGICAL-i176t-2ad1bfe063aa3b94716981beae8cf0d5b1741e14c5b5515bca786a85b3ef8adb3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 1 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001481011800052&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 01:59:32 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i176t-2ad1bfe063aa3b94716981beae8cf0d5b1741e14c5b5515bca786a85b3ef8adb3 |
| PageCount | 8 |
| ParticipantIDs | ieee_primary_10763740 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-Oct.-10 |
| PublicationDateYYYYMMDD | 2024-10-10 |
| PublicationDate_xml | – month: 10 year: 2024 text: 2024-Oct.-10 day: 10 |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings - International Conference on Parallel and Distributed Systems |
| PublicationTitleAbbrev | ICPADS |
| PublicationYear | 2024 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0020350 |
| Score | 2.2782125 |
| Snippet | Accurate and timely air quality prediction is crucial for cities and individuals to effectively take necessary precautions against potential air pollution.... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 423 |
| SubjectTerms | Air quality Air quality prediction Analytical models Atmospheric modeling Data models Large language models Monitoring Predictive models Regulators Semantics Spatiotemporal phenomena Urban data analysis |
| Title | LLMAir: Adaptive Reprogramming Large Language Model for Air Quality Prediction |
| URI | https://ieeexplore.ieee.org/document/10763740 |
| WOSCitedRecordID | wos001481011800052&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA5aPHiqj4pvcvC6utt9JPFWqkVhXRZ80FvJYxYK2pY-BP-9M9lt9eLBy7IEQiDDzHyTeXyMXaFPVRrtXAAWxUDjTwIDAgJJg0achkx5HrK3XBSFHA5V2TSr-14YAPDFZ3BNvz6X76Z2RU9lqOGoDSLBCH1bCFE3a22iK0qRNS3AUahuHvtl7-45i3EVo8AuzcgOiRLnF4eKdyGD9j8P32Odn2Y8Xm7czD7bgskBa6_ZGHijnIesyPOn3nh-y3tOz8iIcQTXdfXVB27kOdV847d-n-REgvbOEbJy3MTrURpfeBAlbkhYHfY6uH_pPwQNW0IwjkS2DLraRaYChBxax0YlNAYHMSlokLYKXWow9oggSmxqECWlxmohMy1TE0MltTPxEWtNphM4ZtwqIyhllkhlErR_Wrm4gswnDRGSqBPWofsZzeqBGKP11Zz-sX7GdkkEZPKj8Jy1lvMVXLAd-7kcL-aXXozfXAud2A |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEB6kCnqqj4pv9-A1mjSvXW-lWlpMQ8EqvZXd7AQK2pbaCv57Z5K0evHgJYSFZWGHmflm5_EB3JBPVZrsnIMZiYHHnzgGY3QkDxqxGiNV8JC9JnGaytFIDapm9aIXBhGL4jO85d8il29n2YqfykjDSRvigCL07TAIml7ZrrWJrzhJVjUBe66667UHrYfnyKdVigObPCXbZVKcXywqhRPp1P95_D40ftrxxGDjaA5gC6eHUF_zMYhKPY8gTZJ-a7K4Fy2r52zGBMHrsv7qnTaKhKu-6Vu-UAqmQXsTBFoFbRLlMI0vOohTNyyuBrx0HoftrlPxJTgTL46WTlNbz-RIoENr36iAB-EQKkWNMstdGxqKPjz0giw0hJNCk-lYRlqGxsdcamv8Y6hNZ1M8AZEpE3PSLJDKBGQBtbJ-jlGRNiRQok6hwfcznpcjMcbrqzn7Y_0adrvDfjJOeunTOeyxONgBeO4F1JaLFV7CTva5nHwsrgqRfgPB6qEf |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+-+International+Conference+on+Parallel+and+Distributed+Systems&rft.atitle=LLMAir%3A+Adaptive+Reprogramming+Large+Language+Model+for+Air+Quality+Prediction&rft.au=Fan%2C+Jinxiao&rft.au=Chu%2C+Haolin&rft.au=Liu%2C+Liang&rft.au=Ma%2C+Huadong&rft.date=2024-10-10&rft.pub=IEEE&rft.eissn=2690-5965&rft.spage=423&rft.epage=430&rft_id=info:doi/10.1109%2FICPADS63350.2024.00062&rft.externalDocID=10763740 |