Q-Learning Based Adaptive Flow Control
With the rapid development of the internet and the expansion of its application scale, the volatility and complexity of network flow in highly concurrent environments have become increasingly prominent. Traditional static control strategies have been difficult to meet the actual needs. Traditional f...
Uloženo v:
| Vydáno v: | 2023 5th International Conference on Data-driven Optimization of Complex Systems (DOCS) s. 1 - 6 |
|---|---|
| Hlavní autoři: | , , , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
22.09.2023
|
| Témata: | |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | With the rapid development of the internet and the expansion of its application scale, the volatility and complexity of network flow in highly concurrent environments have become increasingly prominent. Traditional static control strategies have been difficult to meet the actual needs. Traditional flow control requires manual parameter setting based on thresholds, which not only requires a large number of parameter settings but also affects system availability. This paper proposes an adaptive flow control algorithm based on q-learning, designs and implements a q-learning model for adaptive flow control, and considers the three elements of state, action and reward, where the state includes CPU utilization, average response time, requests per second, and concurrent threads;actions include flow control decisions, that is pass or reject; rewards represent the feedback results of each decision. The algorithm can use the trained q-learning model to select the best flow control decision based on the current state, so as to realize the adaptive adjustment and control of network flow. In the case of high network load, the algorithm will intelligently adjust the flow control strategy to reduce network congestion and delay, improve network performance and reliability, and solve the problem of system instability caused by high throughput under high concurrency. |
|---|---|
| AbstractList | With the rapid development of the internet and the expansion of its application scale, the volatility and complexity of network flow in highly concurrent environments have become increasingly prominent. Traditional static control strategies have been difficult to meet the actual needs. Traditional flow control requires manual parameter setting based on thresholds, which not only requires a large number of parameter settings but also affects system availability. This paper proposes an adaptive flow control algorithm based on q-learning, designs and implements a q-learning model for adaptive flow control, and considers the three elements of state, action and reward, where the state includes CPU utilization, average response time, requests per second, and concurrent threads;actions include flow control decisions, that is pass or reject; rewards represent the feedback results of each decision. The algorithm can use the trained q-learning model to select the best flow control decision based on the current state, so as to realize the adaptive adjustment and control of network flow. In the case of high network load, the algorithm will intelligently adjust the flow control strategy to reduce network congestion and delay, improve network performance and reliability, and solve the problem of system instability caused by high throughput under high concurrency. |
| Author | Zhao, Tianhai Zhang, Hui Wang, Yunlan Zhang, Xiaoping Hu, Shuai |
| Author_xml | – sequence: 1 givenname: Xiaoping surname: Zhang fullname: Zhang, Xiaoping email: zhangxp207@chinaunicom.cn organization: School of Software, Northwestern Polytechnical University,Xi'an,China – sequence: 2 givenname: Yunlan surname: Wang fullname: Wang, Yunlan email: wangyl@nwpu.edu.cn organization: School of Computer Science, Northwestern Polytechnical University,Xi'an,China – sequence: 3 givenname: Tianhai surname: Zhao fullname: Zhao, Tianhai email: zhaoth@nwpu.edu.cn organization: School of Computer Science, Northwestern Polytechnical University,Xi'an,China – sequence: 4 givenname: Shuai surname: Hu fullname: Hu, Shuai email: hus18@chinaunicom.cn organization: China Unicom Western Innovation Research Institute,R&D Department 1,Xi'an,China – sequence: 5 givenname: Hui surname: Zhang fullname: Zhang, Hui email: zhangh373@chinaunicom.cn organization: China Unicom Western Innovation Research Institute,R&D Department 1,Xi'an,China |
| BookMark | eNo1jsFKw0AQQFfQg9b-gWBO3hJndrKbzLFGq0KglOq5TLK7shA3JQ2Kf6-gnt7tvXehTtOYvFLXCAUi8O39ptlZ4KoqNGgqEDSXbPSJWnLFNRkgJqPxXN1s89bLlGJ6y-7k6F22cnKY44fP1sP4mTVjmqdxuFRnQYajX_5xoV7XDy_NU95uHp-bVZtHRJ5zcYA_2drZijpjOke91NaI7mzpoIRQc-gI0YkXJNtLXwYIwJ3TzpsAtFBXv97ovd8fpvgu09f-f5--Ad3YPkU |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/DOCS60977.2023.10294952 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 9798350393521 |
| EndPage | 6 |
| ExternalDocumentID | 10294952 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Key Research and Development Plan of Shaanxi Province grantid: 2023-ZDLGY-10 funderid: 10.13039/501100015401 |
| GroupedDBID | 6IE 6IL CBEJK RIE RIL |
| ID | FETCH-LOGICAL-i119t-ad016098d673b55bd3ca865a2b64d040f89fb311daea136cac4f0f09bd2de5f03 |
| IEDL.DBID | RIE |
| IngestDate | Wed Jan 10 09:28:11 EST 2024 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i119t-ad016098d673b55bd3ca865a2b64d040f89fb311daea136cac4f0f09bd2de5f03 |
| PageCount | 6 |
| ParticipantIDs | ieee_primary_10294952 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-Sept.-22 |
| PublicationDateYYYYMMDD | 2023-09-22 |
| PublicationDate_xml | – month: 09 year: 2023 text: 2023-Sept.-22 day: 22 |
| PublicationDecade | 2020 |
| PublicationTitle | 2023 5th International Conference on Data-driven Optimization of Complex Systems (DOCS) |
| PublicationTitleAbbrev | DOCS |
| PublicationYear | 2023 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| Score | 1.8453473 |
| Snippet | With the rapid development of the internet and the expansion of its application scale, the volatility and complexity of network flow in highly concurrent... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 1 |
| SubjectTerms | Adaptation models adaptive flow control network flow Concurrent computing high concurrency Machine learning algorithms Q-learning sentinel Stability analysis Throughput Training |
| Title | Q-Learning Based Adaptive Flow Control |
| URI | https://ieeexplore.ieee.org/document/10294952 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA5aPHhSccU3exBvWTfJJrs5arV4kFrxQW8lyUykIN1SW_37Jtut4sGDtyQEwgzMI8k33xBy5lRIo4VBKkyZ00JqpLaUmvI4V7owsiGrfrkr-_1qONSDtli9qYVBxAZ8hlkcNn_5ULtFfCoLFs51SOiDx10vS7Us1moxWyzXF9f33UeVh4Qmiz3Bs9XuX31TmrDR2_rngdsk-SnASwffoWWHrOFkl5w_0JYN9TW9CsEH0ksw0-iu0t5b_Zl2l6jzhDz3bp66t7Rtc0DHjOk5NRBZ3nQFqhRWSgvCmUpJw60qINiYr7S3gjEwaJhQzrjC5z7XFjig9LnYI51JPcF9kkLlGTBuWLy4aVUZBpo7bzlqhcriAUmikKPpkslitJLv8I_1I7IZVRnxEZwfk858tsATsuE-5uP32Wmj_y9HXIZg |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA5SBT2pWPHtHsRbah6b7Oao1VKx1opVeit5TKQg3VJb_fsm263iwYO3JATCJMwjyTffIHRmZQijuQbMdUZwKhRgkwmFWexLlWpRklW_dLJuNx8MVK9KVi9zYQCgBJ9BIzbLv3xX2Hl8KgsazlQI6IPFXRVpysgiXatCbVGiLq4fmk-ShJCmEauCN5bzf1VOKR1Ha_OfS26h-k8KXtL7di7baAXGO-j8EVd8qK_JVXA_Lrl0ehINVtJ6Kz6T5gJ3XkfPrZt-s42rQgd4RKmaYe0iz5vKncy4EcI4bnUuhWZGpi5omc-VN5xSp0FTLq22qSeeKOOYA-EJ30W1cTGGPZS43FNHmabx6qZkrqlTzHrDQEmQBvZRPQo5nCy4LIZL-Q7-GD9F6-3-fWfYue3eHaKNuK0RLcHYEarNpnM4Rmv2YzZ6n56UZ_EFeLWJpw |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2023+5th+International+Conference+on+Data-driven+Optimization+of+Complex+Systems+%28DOCS%29&rft.atitle=Q-Learning+Based+Adaptive+Flow+Control&rft.au=Zhang%2C+Xiaoping&rft.au=Wang%2C+Yunlan&rft.au=Zhao%2C+Tianhai&rft.au=Hu%2C+Shuai&rft.date=2023-09-22&rft.pub=IEEE&rft.spage=1&rft.epage=6&rft_id=info:doi/10.1109%2FDOCS60977.2023.10294952&rft.externalDocID=10294952 |