Dynamic Routing Algorithm with Q-learning for Internet of things with Delayed Estimator

With the popularity of the Internet of things (IoT), tremendous objects are connected to the network, making the network topology complex. Mechanical engineering achieves intelligent identification, positioning, tracking, monitoring and management of engineering machinery through the IoT, where info...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IOP conference series. Earth and environmental science Ročník 234; číslo 1; s. 12048 - 12055
Hlavní autori: Wang, Fang, Feng, Renjun, Chen, Haiyan
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Bristol IOP Publishing 08.03.2019
Predmet:
Algorithms
Internet of Things
Machine learning
Mechanical engineering
Network topologies
Topology
ISSN:1755-1307, 1755-1315, 1755-1315
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Abstract With the popularity of the Internet of things (IoT), tremendous objects are connected to the network, making the network topology complex. Mechanical engineering achieves intelligent identification, positioning, tracking, monitoring and management of engineering machinery through the IoT, where information exchange and communication require novel intelligent routing algorithms as traditional routing algorithms are unfit for current network environment. Q-routing implemented a dynamic adjustment which was based on the network environment by combining the Q-learning algorithm. However, Q-routing is a highly random network environment and leads to a decline in performance because of overestimation of values. To solve the problem, we propose an algorithm called Delayed Q-routing (DQ-routing), which uses two sets of value functions to carry out random delayed updates so as to reduce the overestimation of the value function and improve the rate of convergence. The experiments indicate that DQ-routing algorithm gets well performance in several problems.
AbstractList With the popularity of the Internet of things (IoT), tremendous objects are connected to the network, making the network topology complex. Mechanical engineering achieves intelligent identification, positioning, tracking, monitoring and management of engineering machinery through the IoT, where information exchange and communication require novel intelligent routing algorithms as traditional routing algorithms are unfit for current network environment. Q-routing implemented a dynamic adjustment which was based on the network environment by combining the Q-learning algorithm. However, Q-routing is a highly random network environment and leads to a decline in performance because of overestimation of values. To solve the problem, we propose an algorithm called Delayed Q-routing (DQ-routing), which uses two sets of value functions to carry out random delayed updates so as to reduce the overestimation of the value function and improve the rate of convergence. The experiments indicate that DQ-routing algorithm gets well performance in several problems.
Author Chen, Haiyan
Feng, Renjun
Wang, Fang
Author_xml – sequence: 1
  givenname: Fang
  surname: Wang
  fullname: Wang, Fang
  email: grammynow@aliyun.com
  organization: State Grid Jiangsu Electric Power Limited Company Suzhou Power Supply Branch , China
– sequence: 2
  givenname: Renjun
  surname: Feng
  fullname: Feng, Renjun
  email: frj1989@126.com
  organization: State Grid Jiangsu Electric Power Limited Company Suzhou Power Supply Branch , China
– sequence: 3
  givenname: Haiyan
  surname: Chen
  fullname: Chen, Haiyan
  email: 13372175016@189.cn
  organization: State Grid Jiangsu Electric Power Limited Company Suzhou Power Supply Branch , China
BookMark eNqFkF1LwzAUhoNMcJv-BSl4401t0jRLCt6MrepgIH7hZcjSZMvokplmyP69LZWJIuwmCTnPe87hGYCedVYBcIngDYKMJYgSEiOMSJLiLEEJRCnM2AnoHwq9wxvSMzCo6zWEI5rhvA_ep3srNkZGz24XjF1G42rpvAmrTfTZnNFTXCnhbVvRzkczG5S3KkROR2HV_NYdNlWV2KsyKupgNiI4fw5OtahqdfF9D8HbXfE6eYjnj_ezyXgeS5ymLJaMSLWAcrFAQueZQExTiKjCuYRUM5xnLMeihEgorEstFMmwYlplkqY5hgoPwVXXd-vdx07Vga_dzttmJE8JoQQTmqcNNeoo6V1de6X51jd7-j1HkLcSeeuHt654I5Ej3klsgrd_gtIEEYyzwQtTHY9fd3Hjtj-LFcXLL4xvS92g6T_okf5f0_yVPA
CitedBy_id crossref_primary_10_2478_cait_2021_0042
crossref_primary_10_1016_j_adhoc_2022_102790
crossref_primary_10_1109_TNSE_2021_3085514
crossref_primary_10_3390_s23063024
Cites_doi 10.1007/978-3-642-27645-3_7
10.1007/978-3-642-41569-2_13
10.1016/j.jnca.2011.03.031
10.1016/j.pmcj.2013.06.004
10.1016/j.compeleceng.2014.07.010
10.1109/EIConRusNW.2016.7448188
10.1109/MCAS.2009.933854
10.1016/j.physa.2013.03.059
10.2200/S00268ED1V01Y201005AIM009
10.1016/j.chb.2016.04.023
10.1109/TC.2011.239
10.1016/j.asoc.2015.12.028
10.1016/j.jcss.2007.08.009
ContentType Journal Article
Copyright Published under licence by IOP Publishing Ltd
2019. This work is published under http://creativecommons.org/licenses/by/3.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml – notice: Published under licence by IOP Publishing Ltd
– notice: 2019. This work is published under http://creativecommons.org/licenses/by/3.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID O3W
TSCCA
AAYXX
CITATION
ABUWG
AEUYN
AFKRA
ATCPS
AZQEC
BENPR
BHPHI
CCPQU
DWQXO
GNUQQ
HCIFZ
PATMY
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQQKQ
PQUKI
PRINS
PYCSY
DOI 10.1088/1755-1315/234/1/012048
DatabaseName Institute of Physics Open Access Journal Titles
IOPscience (Open Access)
CrossRef
ProQuest Central (Alumni)
ProQuest One Sustainability
ProQuest Central UK/Ireland
Agricultural & Environmental Science Collection
ProQuest Central Essentials
ProQuest Central
Natural Science Collection
ProQuest One Community College
ProQuest Central Korea
ProQuest Central Student
SciTech Premium Collection
Environmental Science Database
ProQuest Central Premium
ProQuest One Academic (New)
Publicly Available Content Database
ProQuest One Academic Middle East (New)
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Academic (retired)
ProQuest One Academic UKI Edition
ProQuest Central China
Environmental Science Collection
DatabaseTitle CrossRef
Publicly Available Content Database
ProQuest Central Student
ProQuest One Academic Middle East (New)
ProQuest Central Essentials
ProQuest One Academic Eastern Edition
ProQuest Central (Alumni Edition)
SciTech Premium Collection
ProQuest One Community College
ProQuest Central China
ProQuest Central
Environmental Science Collection
ProQuest One Sustainability
ProQuest One Academic UKI Edition
Natural Science Collection
ProQuest Central Korea
Agricultural & Environmental Science Collection
Environmental Science Database
ProQuest Central (New)
ProQuest One Academic
ProQuest One Academic (New)
DatabaseTitleList
Publicly Available Content Database
Database_xml – sequence: 1
  dbid: O3W
  name: Institute of Physics Open Access Journal Titles
  url: http://iopscience.iop.org/
  sourceTypes:
    Enrichment Source
    Publisher
– sequence: 2
  dbid: PIMPY
  name: Publicly Available Content Database
  url: http://search.proquest.com/publiccontent
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Geology
DocumentTitleAlternate Dynamic Routing Algorithm with Q-learning for Internet of things with Delayed Estimator
EISSN 1755-1315
ExternalDocumentID 10_1088_1755_1315_234_1_012048
EES_234_1_012048
GroupedDBID 1JI
2WC
4.4
5B3
5GY
5VS
AAFWJ
AAJIO
AAJKP
ABHWH
ACAFW
ACHIP
AEFHF
AEJGL
AFKRA
AFYNE
AHSEE
AIYBF
AKPSB
ALMA_UNASSIGNED_HOLDINGS
ASPBG
ATCPS
ATQHT
AVWKF
AZFZN
BENPR
BHPHI
CCPQU
CEBXE
CJUJL
CRLBU
CS3
DU5
EDWGO
EJD
EQZZN
HCIFZ
IJHAN
IOP
IZVLO
KNG
KQ8
N5L
O3W
OK1
PATMY
PIMPY
PJBAE
PYCSY
RIN
SY9
T37
TR2
TSCCA
W28
AAYXX
AEINN
AEUYN
AFFHD
CITATION
PHGZM
PHGZT
ABUWG
AZQEC
DWQXO
GNUQQ
PKEHL
PQEST
PQQKQ
PQUKI
PRINS
ID FETCH-LOGICAL-c3228-c85ceb0cbb1af94a18f7017e39c07f8394893ad01ae3fdfae543e8fe4c72930e3
IEDL.DBID BENPR
ISSN 1755-1307
1755-1315
IngestDate Sun Nov 30 04:20:50 EST 2025
Sat Nov 29 03:52:15 EST 2025
Tue Nov 18 22:22:49 EST 2025
Wed Aug 21 03:33:33 EDT 2024
Thu Jan 07 13:52:08 EST 2021
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 1
Language English
License Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
http://iopscience.iop.org/info/page/text-and-data-mining
http://creativecommons.org/licenses/by/3.0
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c3228-c85ceb0cbb1af94a18f7017e39c07f8394893ad01ae3fdfae543e8fe4c72930e3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
OpenAccessLink https://www.proquest.com/docview/2557535792?pq-origsite=%requestingapplication%
PQID 2557535792
PQPubID 4998669
PageCount 8
ParticipantIDs crossref_primary_10_1088_1755_1315_234_1_012048
crossref_citationtrail_10_1088_1755_1315_234_1_012048
iop_journals_10_1088_1755_1315_234_1_012048
proquest_journals_2557535792
PublicationCentury 2000
PublicationDate 2019-03-08
PublicationDateYYYYMMDD 2019-03-08
PublicationDate_xml – month: 03
  year: 2019
  text: 2019-03-08
  day: 08
PublicationDecade 2010
PublicationPlace Bristol
PublicationPlace_xml – name: Bristol
PublicationTitle IOP conference series. Earth and environmental science
PublicationTitleAlternate IOP Conf. Ser.: Earth Environ. Sci
PublicationYear 2019
Publisher IOP Publishing
Publisher_xml – name: IOP Publishing
References Nordrum A. (4) 2016
Wang C. (2) 2015; 33
12
14
15
17
Farahnakian F. (11) 2011
19
Neu G. (20) 2010
1
3
5
6
Sutton (13) 2005; 16
7
8
9
Puterman M. L. (16) 2014
Taylor M. E. (18) 2009; 10
10
References_xml – start-page: 1804
  year: 2010
  ident: 20
  publication-title: Advances in Neural Information Processing Systems
– ident: 15
  doi: 10.1007/978-3-642-27645-3_7
– volume: 10
  start-page: 1633
  year: 2009
  ident: 18
  publication-title: Journal of Machine Learning Research
– ident: 3
  doi: 10.1007/978-3-642-41569-2_13
– ident: 6
  doi: 10.1016/j.jnca.2011.03.031
– start-page: 1
  year: 2011
  ident: 11
  publication-title: Networked Embedded Systems for Enterprise Applications (NESEA), 2011 IEEE 2nd International Conference on IEEE
– ident: 8
  doi: 10.1016/j.pmcj.2013.06.004
– volume: 16
  start-page: 285
  issn: 0885-6125
  year: 2005
  ident: 13
  publication-title: Machine Learning
– ident: 9
  doi: 10.1016/j.compeleceng.2014.07.010
– ident: 12
  doi: 10.1109/EIConRusNW.2016.7448188
– ident: 14
  doi: 10.1109/MCAS.2009.933854
– ident: 1
  doi: 10.1016/j.physa.2013.03.059
– start-page: 100
  year: 2014
  ident: 16
  publication-title: Markov decision processes: discrete stochastic dynamic programming
– ident: 19
  doi: 10.2200/S00268ED1V01Y201005AIM009
– ident: 5
  doi: 10.1016/j.chb.2016.04.023
– ident: 7
  doi: 10.1109/TC.2011.239
– volume: 33
  start-page: 2955
  issn: 0733-8724
  year: 2015
  ident: 2
  publication-title: Journal of Lightwave Technology
– ident: 10
  doi: 10.1016/j.asoc.2015.12.028
– start-page: 18
  year: 2016
  ident: 4
  publication-title: Popular internet of things forecast of 50 billion devices by 2020 is outdated
– ident: 17
  doi: 10.1016/j.jcss.2007.08.009
SSID ssj0067439
Score 2.149548
Snippet With the popularity of the Internet of things (IoT), tremendous objects are connected to the network, making the network topology complex. Mechanical...
SourceID proquest
crossref
iop
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 12048
SubjectTerms Algorithms
Internet of Things
Machine learning
Mechanical engineering
Network topologies
Topology
SummonAdditionalLinks – databaseName: Institute of Physics Open Access Journal Titles
  dbid: O3W
  link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV3dS8MwEA9uKvjitzidEsE3qU2bdk0fh9v0Qabix_YW0jSZg32xVWH_vZe0U4aICL4VeteG630lvbsfQucpoSplYEi-FNQJVCKdWNaIE2kI5jQg0rMoES-3UbvNut34vqgmtL0w40nh-i_hMh8UnIuwKIhjLgS80PGoF7o-DVzPNe2fASuhVcogmINK39HOwhmbEvvY9kRaHhItmoR_fM5SfCrBGr45aRt5Wlv_sOZttFmknbieM-ygFTXaRevXFtZ3voc6jRyYHpsCIQhmuD7ojaf97HWIzUEtfnAKdIkehiQX58eIKsNjjTML_JmTNdRAzFWKm-A3hmY3v4-eW82nqxungFxwJFg2cyQLpUqITBJP6DgQHtMR2KyisSSRhmTKzKoRKfGEojrVQoUBVUyrQEKSTomiB6g8Go_UIcKh8IE1YWDgkAZIX8S1VNQUJGAi0klIKihcCJrLYh65gcUYcPtfnDFuhMaN0DgIjXs8F1oFuZ98k3wix68cF_BdeGGcs1-pz5aom83Hpft8kuoKqi504osQNmew9wuj2D_60wuP0QZkYbEtbGNVVM6mb-oErcn3rD-bnlp1_gBVu-sw
  priority: 102
  providerName: IOP Publishing
Title Dynamic Routing Algorithm with Q-learning for Internet of things with Delayed Estimator
URI https://iopscience.iop.org/article/10.1088/1755-1315/234/1/012048
https://www.proquest.com/docview/2557535792
Volume 234
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIOP
  databaseName: Institute of Physics Open Access Journal Titles
  customDbUrl:
  eissn: 1755-1315
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0067439
  issn: 1755-1307
  databaseCode: O3W
  dateStart: 20080501
  isFulltext: true
  titleUrlDefault: http://iopscience.iop.org/
  providerName: IOP Publishing
– providerCode: PRVPQU
  databaseName: Environmental Science Database
  customDbUrl:
  eissn: 1755-1315
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0067439
  issn: 1755-1307
  databaseCode: PATMY
  dateStart: 20080501
  isFulltext: true
  titleUrlDefault: http://search.proquest.com/environmentalscience
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: Proquest Central
  customDbUrl:
  eissn: 1755-1315
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0067439
  issn: 1755-1307
  databaseCode: BENPR
  dateStart: 20080501
  isFulltext: true
  titleUrlDefault: https://www.proquest.com/central
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: Publicly Available Content Database
  customDbUrl:
  eissn: 1755-1315
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0067439
  issn: 1755-1307
  databaseCode: PIMPY
  dateStart: 20080501
  isFulltext: true
  titleUrlDefault: http://search.proquest.com/publiccontent
  providerName: ProQuest
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LT9wwEB6xUKReCn2gbnnIlXqrrDwcb5wTWiCUSu12-6DQk-U4NiAtu8tuQOLfM04cKEKCA8ck4yjKeGY-j8fzAXwqQ2ZKgYYUa8VoYgpNM90LaWoxmLMk1FHNEvH3WzoYiOPjbOgTbnNfVtn6xNpRlxPtcuQBQl9E1jzN4u3pBXWsUW531VNodGDJdSrDeb60kw-Gv1pf7Crss_pIJOcUvXXanhHGZZ-_F_EgZkkQBe4UqWMB-i88dc4m0wc-ug48-yvP_eRVeOUhJ-k3c-Q1LJjxG1j-UlP6Xr-Fo72GlJ644iAMZKQ_OsG3VKfnxCVpyU_qmSVOCAJc0qQQTUUmllQ16WcjtmdG6tqUJEefce5W8u_gcD__s3tAPd0C1WjVgmrBtSlCXRSRslmiImFTtFfDMh2mFoGU61OjyjBShtnSKsMTZoQ1iUaAzkLD1mBxPBmb90C4inFoIdC4EQLoWGW9UvUMgi-V2oKHXeDtX5ba9yJ3lBgjWe-JCyGddqTTjkTtyEg22ulCcDtu2nTjeHLEZ1Si9IY5f1L64z3pPP9977mclrYLG6227wTvVP3h8cfr8BIhV1ZXsYkNWKxml2YTXuir6mw-2_LTdws6P9gRXg2_fh_-uwHI5PLV
linkProvider ProQuest
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1bTxQxFD6BRaMvCl7iKmhJ9Mk0c-l0p_NACGEX2bBs1ogKT7XTaXGTvcmOkP1T_EZO5yISE3nigdfp6WRm-vWcb057-gG8z3xmMoETKdSK0cikmia65dPYYjBnka-DQiXiWy_u98XxcTJYgsu6FsZtq6x9YuGos6l2OXIPqS8yax4n4fbsF3WqUW51tZbQKGFxYBYX-Ms23-q2cXw_hOFe52h3n1aqAlQjeAXVgmuT-jpNA2WTSAXCxghLwxLtxxb5gjuORWV-oAyzmVWGR8wIayKNPJT5huF9l2ElcmBvwMqgezg4qX2_29GfFCWYnFOMDnFdk4y_mdW1gHshi7zAc1WrTnXor3C4PJzO_okJRaDbe3rfPtEqPKkoNdkp58AaLJnJM3j4qZAsXjyH7-3FRI2HmrjNTxioyc7oFJ86_zkmLglNPtNKOeOUIIEnZYrU5GRqSV6ImpZmbTNSC5ORDvrEsctUvICvd_JWL6ExmU7MKyBchdg1Fei8kOLoUCWtTLUMkksV25T7TeD1qEpdnbXuJD9GsljzF0I6NEiHBolokIEs0dAE70-_WXnayK09PiJoZOV45rdab96w7nS-3GiXs8w2Yb1G17XhNbRe_7_5HTzaPzrsyV63f_AGHiO9TIode2IdGvnZb7MBD_R5Ppyfva2mDoEfdw3FK-wSTpk
linkToPdf http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV3dT9swED_xMRAvsPEhCt3mSXtDIU6cNM4jouk2reo6ja83y3HsglTaqgQk_nvOTlpUoQkh8RYpd4l19t39bN8HwPeCMl1wVKRQSeZFOldeqlrUSww6cxZRFbguERfdpNfjV1dpfwmyeS7MeFKb_mN8rAoFVyKsA-K4jw4v9gIWxH7IIj_wbfpnxP1JYZZh1VYrsYv7D7ucGWQbZp-6vEjHR5NZovB_v7Xgo5ZxHC8MtfM-na13GvdH2KzhJzmpmD7Bkh5tw9oP1973cQcu21WDemIDhdCpkZPhYDy9Ka9viT2wJX-9usvEgCDYJdVxoi7J2JDSNQCtyNp6KB91QTK0H7d2V78L553s7PSnV7de8BRqOPcUj5XOqcrzQJo0kgE3CequZqmiiUFQZWvWyIIGUjNTGKnjiGludKQQrDOq2R6sjMYjvQ8kliGy5hwVHeGACmXaKmRLIxCTiclj2oB4Jmyh6rrktj3GULj7cc6FFZywghMoOBGISnAN8Od8k6oyx6scRzg3olbSu1epvy1QZ9m_hfcCJ64Bzdm6eCbETRruAeMkDQ_e9MOvsN5vd0T3V-_3IWwgMEtdrBtvwko5vdef4YN6KG_upl_c6n4CeyPwmA
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Dynamic+Routing+Algorithm+with+Q-learning+for+Internet+of+things+with+Delayed+Estimator&rft.jtitle=IOP+conference+series.+Earth+and+environmental+science&rft.au=Wang%2C+Fang&rft.au=Feng%2C+Renjun&rft.au=Chen%2C+Haiyan&rft.date=2019-03-08&rft.issn=1755-1315&rft.eissn=1755-1315&rft.volume=234&rft.spage=12048&rft_id=info:doi/10.1088%2F1755-1315%2F234%2F1%2F012048&rft.externalDBID=n%2Fa&rft.externalDocID=10_1088_1755_1315_234_1_012048
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1755-1307&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1755-1307&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1755-1307&client=summon