Dynamic Routing Algorithm with Q-learning for Internet of things with Delayed Estimator
With the popularity of the Internet of things (IoT), tremendous objects are connected to the network, making the network topology complex. Mechanical engineering achieves intelligent identification, positioning, tracking, monitoring and management of engineering machinery through the IoT, where info...
Uloženo v:
| Vydáno v: | IOP conference series. Earth and environmental science Ročník 234; číslo 1; s. 12048 - 12055 |
|---|---|
| Hlavní autoři: | , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Bristol
IOP Publishing
08.03.2019
|
| Témata: | |
| ISSN: | 1755-1307, 1755-1315, 1755-1315 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | With the popularity of the Internet of things (IoT), tremendous objects are connected to the network, making the network topology complex. Mechanical engineering achieves intelligent identification, positioning, tracking, monitoring and management of engineering machinery through the IoT, where information exchange and communication require novel intelligent routing algorithms as traditional routing algorithms are unfit for current network environment. Q-routing implemented a dynamic adjustment which was based on the network environment by combining the Q-learning algorithm. However, Q-routing is a highly random network environment and leads to a decline in performance because of overestimation of values. To solve the problem, we propose an algorithm called Delayed Q-routing (DQ-routing), which uses two sets of value functions to carry out random delayed updates so as to reduce the overestimation of the value function and improve the rate of convergence. The experiments indicate that DQ-routing algorithm gets well performance in several problems. |
|---|---|
| AbstractList | With the popularity of the Internet of things (IoT), tremendous objects are connected to the network, making the network topology complex. Mechanical engineering achieves intelligent identification, positioning, tracking, monitoring and management of engineering machinery through the IoT, where information exchange and communication require novel intelligent routing algorithms as traditional routing algorithms are unfit for current network environment. Q-routing implemented a dynamic adjustment which was based on the network environment by combining the Q-learning algorithm. However, Q-routing is a highly random network environment and leads to a decline in performance because of overestimation of values. To solve the problem, we propose an algorithm called Delayed Q-routing (DQ-routing), which uses two sets of value functions to carry out random delayed updates so as to reduce the overestimation of the value function and improve the rate of convergence. The experiments indicate that DQ-routing algorithm gets well performance in several problems. |
| Author | Chen, Haiyan Feng, Renjun Wang, Fang |
| Author_xml | – sequence: 1 givenname: Fang surname: Wang fullname: Wang, Fang email: grammynow@aliyun.com organization: State Grid Jiangsu Electric Power Limited Company Suzhou Power Supply Branch , China – sequence: 2 givenname: Renjun surname: Feng fullname: Feng, Renjun email: frj1989@126.com organization: State Grid Jiangsu Electric Power Limited Company Suzhou Power Supply Branch , China – sequence: 3 givenname: Haiyan surname: Chen fullname: Chen, Haiyan email: 13372175016@189.cn organization: State Grid Jiangsu Electric Power Limited Company Suzhou Power Supply Branch , China |
| BookMark | eNqFkF1LwzAUhoNMcJv-BSl4401t0jRLCt6MrepgIH7hZcjSZMvokplmyP69LZWJIuwmCTnPe87hGYCedVYBcIngDYKMJYgSEiOMSJLiLEEJRCnM2AnoHwq9wxvSMzCo6zWEI5rhvA_ep3srNkZGz24XjF1G42rpvAmrTfTZnNFTXCnhbVvRzkczG5S3KkROR2HV_NYdNlWV2KsyKupgNiI4fw5OtahqdfF9D8HbXfE6eYjnj_ezyXgeS5ymLJaMSLWAcrFAQueZQExTiKjCuYRUM5xnLMeihEgorEstFMmwYlplkqY5hgoPwVXXd-vdx07Vga_dzttmJE8JoQQTmqcNNeoo6V1de6X51jd7-j1HkLcSeeuHt654I5Ej3klsgrd_gtIEEYyzwQtTHY9fd3Hjtj-LFcXLL4xvS92g6T_okf5f0_yVPA |
| CitedBy_id | crossref_primary_10_2478_cait_2021_0042 crossref_primary_10_1016_j_adhoc_2022_102790 crossref_primary_10_1109_TNSE_2021_3085514 crossref_primary_10_3390_s23063024 |
| Cites_doi | 10.1007/978-3-642-27645-3_7 10.1007/978-3-642-41569-2_13 10.1016/j.jnca.2011.03.031 10.1016/j.pmcj.2013.06.004 10.1016/j.compeleceng.2014.07.010 10.1109/EIConRusNW.2016.7448188 10.1109/MCAS.2009.933854 10.1016/j.physa.2013.03.059 10.2200/S00268ED1V01Y201005AIM009 10.1016/j.chb.2016.04.023 10.1109/TC.2011.239 10.1016/j.asoc.2015.12.028 10.1016/j.jcss.2007.08.009 |
| ContentType | Journal Article |
| Copyright | Published under licence by IOP Publishing Ltd 2019. This work is published under http://creativecommons.org/licenses/by/3.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| Copyright_xml | – notice: Published under licence by IOP Publishing Ltd – notice: 2019. This work is published under http://creativecommons.org/licenses/by/3.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| DBID | O3W TSCCA AAYXX CITATION ABUWG AEUYN AFKRA ATCPS AZQEC BENPR BHPHI CCPQU DWQXO GNUQQ HCIFZ PATMY PHGZM PHGZT PIMPY PKEHL PQEST PQQKQ PQUKI PRINS PYCSY |
| DOI | 10.1088/1755-1315/234/1/012048 |
| DatabaseName | Institute of Physics Open Access Journal Titles IOPscience (Open Access) CrossRef ProQuest Central (Alumni) ProQuest One Sustainability ProQuest Central UK/Ireland ProQuest Agricultural & Environmental Science & Pollution Managment ProQuest Central Essentials ProQuest Central Natural Science Collection ProQuest One Community College ProQuest Central Korea ProQuest Central Student SciTech Premium Collection Environmental Science Database ProQuest Central Premium ProQuest One Academic ProQuest Publicly Available Content ProQuest One Academic Middle East (New) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China Environmental Science Collection |
| DatabaseTitle | CrossRef Publicly Available Content Database ProQuest Central Student ProQuest One Academic Middle East (New) ProQuest Central Essentials ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Central China ProQuest Central Environmental Science Collection ProQuest One Sustainability ProQuest One Academic UKI Edition Natural Science Collection ProQuest Central Korea Agricultural & Environmental Science Collection Environmental Science Database ProQuest Central (New) ProQuest One Academic ProQuest One Academic (New) |
| DatabaseTitleList | Publicly Available Content Database |
| Database_xml | – sequence: 1 dbid: O3W name: Institute of Physics Journals Open Access url: http://iopscience.iop.org/ sourceTypes: Enrichment Source Publisher – sequence: 2 dbid: PIMPY name: Publicly Available Content Database url: http://search.proquest.com/publiccontent sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Geology |
| DocumentTitleAlternate | Dynamic Routing Algorithm with Q-learning for Internet of things with Delayed Estimator |
| EISSN | 1755-1315 |
| ExternalDocumentID | 10_1088_1755_1315_234_1_012048 EES_234_1_012048 |
| GroupedDBID | 1JI 2WC 4.4 5B3 5GY 5VS AAFWJ AAJIO AAJKP ABHWH ACAFW ACHIP AEFHF AEJGL AFKRA AFYNE AHSEE AIYBF AKPSB ALMA_UNASSIGNED_HOLDINGS ASPBG ATCPS ATQHT AVWKF AZFZN BENPR BHPHI CCPQU CEBXE CJUJL CRLBU CS3 DU5 EDWGO EJD EQZZN HCIFZ IJHAN IOP IZVLO KNG KQ8 N5L O3W OK1 PATMY PIMPY PJBAE PYCSY RIN SY9 T37 TR2 TSCCA W28 AAYXX AEINN AEUYN AFFHD CITATION PHGZM PHGZT ABUWG AZQEC DWQXO GNUQQ PKEHL PQEST PQQKQ PQUKI PRINS |
| ID | FETCH-LOGICAL-c3228-c85ceb0cbb1af94a18f7017e39c07f8394893ad01ae3fdfae543e8fe4c72930e3 |
| IEDL.DBID | O3W |
| ISSN | 1755-1307 1755-1315 |
| IngestDate | Sun Nov 30 04:20:50 EST 2025 Sat Nov 29 03:52:15 EST 2025 Tue Nov 18 22:22:49 EST 2025 Wed Aug 21 03:33:33 EDT 2024 Thu Jan 07 13:52:08 EST 2021 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Language | English |
| License | Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. http://iopscience.iop.org/info/page/text-and-data-mining http://creativecommons.org/licenses/by/3.0 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c3228-c85ceb0cbb1af94a18f7017e39c07f8394893ad01ae3fdfae543e8fe4c72930e3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| OpenAccessLink | https://iopscience.iop.org/article/10.1088/1755-1315/234/1/012048 |
| PQID | 2557535792 |
| PQPubID | 4998669 |
| PageCount | 8 |
| ParticipantIDs | crossref_primary_10_1088_1755_1315_234_1_012048 crossref_citationtrail_10_1088_1755_1315_234_1_012048 iop_journals_10_1088_1755_1315_234_1_012048 proquest_journals_2557535792 |
| PublicationCentury | 2000 |
| PublicationDate | 2019-03-08 |
| PublicationDateYYYYMMDD | 2019-03-08 |
| PublicationDate_xml | – month: 03 year: 2019 text: 2019-03-08 day: 08 |
| PublicationDecade | 2010 |
| PublicationPlace | Bristol |
| PublicationPlace_xml | – name: Bristol |
| PublicationTitle | IOP conference series. Earth and environmental science |
| PublicationTitleAlternate | IOP Conf. Ser.: Earth Environ. Sci |
| PublicationYear | 2019 |
| Publisher | IOP Publishing |
| Publisher_xml | – name: IOP Publishing |
| References | Nordrum A. (4) 2016 Wang C. (2) 2015; 33 12 14 15 17 Farahnakian F. (11) 2011 19 Neu G. (20) 2010 1 3 5 6 Sutton (13) 2005; 16 7 8 9 Puterman M. L. (16) 2014 Taylor M. E. (18) 2009; 10 10 |
| References_xml | – start-page: 1804 year: 2010 ident: 20 publication-title: Advances in Neural Information Processing Systems – ident: 15 doi: 10.1007/978-3-642-27645-3_7 – volume: 10 start-page: 1633 year: 2009 ident: 18 publication-title: Journal of Machine Learning Research – ident: 3 doi: 10.1007/978-3-642-41569-2_13 – ident: 6 doi: 10.1016/j.jnca.2011.03.031 – start-page: 1 year: 2011 ident: 11 publication-title: Networked Embedded Systems for Enterprise Applications (NESEA), 2011 IEEE 2nd International Conference on IEEE – ident: 8 doi: 10.1016/j.pmcj.2013.06.004 – volume: 16 start-page: 285 issn: 0885-6125 year: 2005 ident: 13 publication-title: Machine Learning – ident: 9 doi: 10.1016/j.compeleceng.2014.07.010 – ident: 12 doi: 10.1109/EIConRusNW.2016.7448188 – ident: 14 doi: 10.1109/MCAS.2009.933854 – ident: 1 doi: 10.1016/j.physa.2013.03.059 – start-page: 100 year: 2014 ident: 16 publication-title: Markov decision processes: discrete stochastic dynamic programming – ident: 19 doi: 10.2200/S00268ED1V01Y201005AIM009 – ident: 5 doi: 10.1016/j.chb.2016.04.023 – ident: 7 doi: 10.1109/TC.2011.239 – volume: 33 start-page: 2955 issn: 0733-8724 year: 2015 ident: 2 publication-title: Journal of Lightwave Technology – ident: 10 doi: 10.1016/j.asoc.2015.12.028 – start-page: 18 year: 2016 ident: 4 publication-title: Popular internet of things forecast of 50 billion devices by 2020 is outdated – ident: 17 doi: 10.1016/j.jcss.2007.08.009 |
| SSID | ssj0067439 |
| Score | 2.1494436 |
| Snippet | With the popularity of the Internet of things (IoT), tremendous objects are connected to the network, making the network topology complex. Mechanical... |
| SourceID | proquest crossref iop |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 12048 |
| SubjectTerms | Algorithms Internet of Things Machine learning Mechanical engineering Network topologies Topology |
| SummonAdditionalLinks | – databaseName: ProQuest Central dbid: BENPR link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1ZS8QwEA6e4Iu3uF5E8E1CN5t2mzyJR9UHWbz1LaQ5VFh3V7cK_nsnbeqBoA--NpNSOsnMl8nMfAhtgcdqO6cMsW1nSJwzRpRINMkptyn4HBOXyePXJ2mnw29vxWkIuA1DWmVtE0tDbfrax8gjgL6ArJNUtHYGT8SzRvnb1UChMYrGfacyWOfje1nn9Ly2xT7DXpQlkUlCwFqndY0wHPvCM5pELRZHNPJVpJ4F6It7Gn3oD37Y6NLxHM7895Nn0XSAnHi3WiNzaMT25tHkUUnp-7aAbg4qUnrsk4PAkeHd7h28pbh_xD5Ii89IYJa4wwBwcRVCtAXuO1yUpJ-V2IHtqjdrcAY249Gf5BfR1WF2uX9MAt0C0bCrOdE80TZv6jynyolYUe5S2K-WCd1MHQAp36dGmSZVljnjlE1iZrmzsQaAzpqWLaGxXr9nlxEGXJCKlJlc5SrWMeXCtFsATR2cDgHw0QZK6r8sdehF7ikxurK8E-dceu1Irx0J2pFUVtppoOhj3qDqxvHnjG1Qogwbc_in9OY36Sy7-DYuB8Y10Fqt7U_BT1Wv_D68iqYAcokyi42vobHi-cWuown9WjwMnzfC8n0Hlz_wOg priority: 102 providerName: ProQuest |
| Title | Dynamic Routing Algorithm with Q-learning for Internet of things with Delayed Estimator |
| URI | https://iopscience.iop.org/article/10.1088/1755-1315/234/1/012048 https://www.proquest.com/docview/2557535792 |
| Volume | 234 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIOP databaseName: Institute of Physics Journals Open Access customDbUrl: eissn: 1755-1315 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0067439 issn: 1755-1307 databaseCode: O3W dateStart: 20080501 isFulltext: true titleUrlDefault: http://iopscience.iop.org/ providerName: IOP Publishing – providerCode: PRVPQU databaseName: Environmental Science Database customDbUrl: eissn: 1755-1315 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0067439 issn: 1755-1307 databaseCode: PATMY dateStart: 20080501 isFulltext: true titleUrlDefault: http://search.proquest.com/environmentalscience providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Central customDbUrl: eissn: 1755-1315 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0067439 issn: 1755-1307 databaseCode: BENPR dateStart: 20080501 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest – providerCode: PRVPQU databaseName: Publicly Available Content Database customDbUrl: eissn: 1755-1315 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0067439 issn: 1755-1307 databaseCode: PIMPY dateStart: 20080501 isFulltext: true titleUrlDefault: http://search.proquest.com/publiccontent providerName: ProQuest |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV3db9MwED_tA6S9jA2YVhiTkXhDJnHtNPbjYBkgQVc-N54sxx9jUtdWXYa0_56zkzJVE5qQeEuUu8S62He_s-8D4AVarEEIxlE_CI6KmnNqVGFpzaQv0eY4kYLHv38oh0N5eqpGXTRhyoWZzjrV_wov20LBrQi7gDiZocErKOOsyPpcZCyL6Z9CrsI6l2jMcUof85OFMo4h9irlRCaevFwkCf_1PUv2aRXHcEtJJ8tz9OA_jHkLNjvYSQ5ahm1Y8ZOHcP9taut7_QhODtvG9CQGCKExIwfjs-n8vPl5QeJGLflEu-4SZwRBLmm3EX1DpoE0qfFnS3box-baO1Kh3riI3vxj-HZUfX3zjnYtF6jFlS2plYX1dW7rmpmghGEylLhmPVc2LwOCqVirxricGc-DC8YXgnsZvLAI0nnu-Q6sTaYTvwsEsUGpSu5qUxthBZPKDfoITwN6iAj6WA-KhaC17eqRx7YYY53OxaXUUWg6Ck2j0DTTrdB6kP3hm7UVOe7keIn_RXeL8_JO6udL1FX1Zem5nrnQg73FnLghROcMfb-iVP0n__TBp7CBKEylwDa5B2vN_Mo_g3v2V3N-Od-H9dfVcPR5P01rvBu9_zj68RsoD-6N |
| linkProvider | IOP Publishing |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1Lb9QwEB61WxBceCMWChgJTsjKOk429gGhit3SVberRRRoT8bxo6y0L7oBtH-K38g4D0qFRE89cE3sKIk_z3wez_gDeI4eq-u9ttR1vaVJzjnVMjU0Z8Jl6HNsUiaPfxxmo5E4OpLjDfjZ1MKEtMrGJpaG2i5MiJFHSH2RWaeZjF8vv9KgGhV2VxsJjQoW-279A5dsq1eDHo7vizje7R--2aO1qgA1CF5BjUiNyzsmz5n2MtFM-Axh6bg0ncwjXwjHsWjbYdpxb712acKd8C4xyEN5x3F87iZsJQHsLdgaDw7Gx43tDxn9sizBTFOK3iFrapJxmVlfY2kU8yRiUahaDapDf7jDzcli-ZdPKB3d7s3_7Rfdghs1pSY71Ry4DRtufgeuvi0li9d34VNvPdeziSEh-QkdNdmZnuBbF19mJAShyTtaK2ecECTwpAqRuoIsPClKUdOqWc9N9dpZ0kebOAuRinvw4VK-6j605ou5ewAEeU8mM25znevEJExI242Rentc_SKhZW1Im1FVpj5rPUh-TFW55y-ECmhQAQ0K0aCYqtDQhuh3v2V12siFPV4iaFRteFYXtn52rnW___7cfbW0vg3bDbrOGp5B6-G_bz-Fa3uHB0M1HIz2H8F1pJeyzNgT29AqTr-5x3DFfC8mq9Mn9dQh8PmyofgLWHhOqg |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV3db9MwED9tYyBeYMAQhcGMtDdkEtdOYz9ONAW0qds0YHuzHH-MSV1bdQFp_z1nJx2qEJqQeIuUu8Q623e_s-8DYA8t1iAE46gfBEdFzTk1qrC0ZtKXaHOcSMHj3w7L8Vien6vjNahuc2Fm8071v8fHtlBwK8IuIE5maPAKyjgrsj4XGcti-qeQ2dyFdbgXq5XExX3Ez5YKOYbZq5QXmfjycpko_NdvrdiodRzHH4o6WZ_R4_807i141MFPst8yPYE1P30K9z-m9r43z-Bs2DaoJzFQCI0a2Z9czBaXzfcrEg9syQntukxcEAS7pD1O9A2ZBdKkBqAt2dBPzI13pEL9cRW9-m34Oqq-fPhEu9YL1OIOl9TKwvo6t3XNTFDCMBlK3LueK5uXAUFVrFljXM6M58EF4wvBvQxeWATrPPf8OWxMZ1P_AghihFKV3NWmNsIKJpUb9BGmBvQUEfyxHhRLYWvb1SWP7TEmOt2PS6mj4HQUnEbBaaZbwfUgu-Wbt5U57uR4h3Oju016fSf12xXqqjpdea9x4nqws1wXvwnRSUMfsChV_-U__XAXHhwPR_rw8_jgFTxEYKZSrJvcgY1m8cO_hk37s7m8XrxJq_sXg0vwqQ |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Dynamic+Routing+Algorithm+with+Q-learning+for+Internet+of+things+with+Delayed+Estimator&rft.jtitle=IOP+conference+series.+Earth+and+environmental+science&rft.au=Wang%2C+Fang&rft.au=Feng%2C+Renjun&rft.au=Chen%2C+Haiyan&rft.date=2019-03-08&rft.pub=IOP+Publishing&rft.issn=1755-1307&rft.eissn=1755-1315&rft.volume=234&rft.issue=1&rft_id=info:doi/10.1088%2F1755-1315%2F234%2F1%2F012048&rft.externalDocID=EES_234_1_012048 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1755-1307&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1755-1307&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1755-1307&client=summon |