Virtual Network Function Embedding under Nodal Outage Using Deep Q-Learning
With the emergence of various types of applications such as delay-sensitive applications, future communication networks are expected to be increasingly complex and dynamic. Network Function Virtualization (NFV) provides the necessary support towards efficient management of such complex networks, by...
Uloženo v:
| Vydáno v: | Future internet Ročník 13; číslo 3; s. 82 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
MDPI AG
01.03.2021
|
| Témata: | |
| ISSN: | 1999-5903, 1999-5903 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | With the emergence of various types of applications such as delay-sensitive applications, future communication networks are expected to be increasingly complex and dynamic. Network Function Virtualization (NFV) provides the necessary support towards efficient management of such complex networks, by virtualizing network functions and placing them on shared commodity servers. However, one of the critical issues in NFV is the resource allocation for the highly complex services; moreover, this problem is classified as an NP-Hard problem. To solve this problem, our work investigates the potential of Deep Reinforcement Learning (DRL) as a swift yet accurate approach (as compared to integer linear programming) for deploying Virtualized Network Functions (VNFs) under several Quality-of-Service (QoS) constraints such as latency, memory, CPU, and failure recovery requirements. More importantly, the failure recovery requirements are focused on the node-outage problem where outage can be either due to a disaster or unavailability of network topology information (e.g., due to proprietary and ownership issues). In DRL, we adopt a Deep Q-Learning (DQL) based algorithm where the primary network estimates the action-value function Q, as well as the predicted Q, highly causing divergence in Q-value’s updates. This divergence increases for the larger-scale action and state-space causing inconsistency in learning, resulting in an inaccurate output. Thus, to overcome this divergence, our work has adopted a well-known approach, i.e., introducing Target Neural Networks and Experience Replay algorithms in DQL. The constructed model is simulated for two real network topologies—Netrail Topology and BtEurope Topology—with various capacities of the nodes (e.g., CPU core, VNFs per Core), links (e.g., bandwidth and latency), several VNF Forwarding Graph (VNF-FG) complexities, and different degrees of the nodal outage from 0% to 50%. We can conclude from our work that, with the increase in network density or nodal capacity or VNF-FG’s complexity, the model took extremely high computation time to execute the desirable results. Moreover, with the rise in complexity of the VNF-FG, the resources decline much faster. In terms of the nodal outage, our model provided almost 70–90% Service Acceptance Rate (SAR) even with a 50% nodal outage for certain combinations of scenarios. |
|---|---|
| AbstractList | With the emergence of various types of applications such as delay-sensitive applications, future communication networks are expected to be increasingly complex and dynamic. Network Function Virtualization (NFV) provides the necessary support towards efficient management of such complex networks, by virtualizing network functions and placing them on shared commodity servers. However, one of the critical issues in NFV is the resource allocation for the highly complex services; moreover, this problem is classified as an NP-Hard problem. To solve this problem, our work investigates the potential of Deep Reinforcement Learning (DRL) as a swift yet accurate approach (as compared to integer linear programming) for deploying Virtualized Network Functions (VNFs) under several Quality-of-Service (QoS) constraints such as latency, memory, CPU, and failure recovery requirements. More importantly, the failure recovery requirements are focused on the node-outage problem where outage can be either due to a disaster or unavailability of network topology information (e.g., due to proprietary and ownership issues). In DRL, we adopt a Deep Q-Learning (DQL) based algorithm where the primary network estimates the action-value function Q, as well as the predicted Q, highly causing divergence in Q-value’s updates. This divergence increases for the larger-scale action and state-space causing inconsistency in learning, resulting in an inaccurate output. Thus, to overcome this divergence, our work has adopted a well-known approach, i.e., introducing Target Neural Networks and Experience Replay algorithms in DQL. The constructed model is simulated for two real network topologies—Netrail Topology and BtEurope Topology—with various capacities of the nodes (e.g., CPU core, VNFs per Core), links (e.g., bandwidth and latency), several VNF Forwarding Graph (VNF-FG) complexities, and different degrees of the nodal outage from 0% to 50%. We can conclude from our work that, with the increase in network density or nodal capacity or VNF-FG’s complexity, the model took extremely high computation time to execute the desirable results. Moreover, with the rise in complexity of the VNF-FG, the resources decline much faster. In terms of the nodal outage, our model provided almost 70–90% Service Acceptance Rate (SAR) even with a 50% nodal outage for certain combinations of scenarios. |
| Author | Ahmadi, Hamed Nag, Avishek Chetty, Swarna Bindu Sharma, Sachin |
| Author_xml | – sequence: 1 givenname: Swarna Bindu surname: Chetty fullname: Chetty, Swarna Bindu – sequence: 2 givenname: Hamed orcidid: 0000-0001-5508-8757 surname: Ahmadi fullname: Ahmadi, Hamed – sequence: 3 givenname: Sachin orcidid: 0000-0002-8358-2258 surname: Sharma fullname: Sharma, Sachin – sequence: 4 givenname: Avishek orcidid: 0000-0003-1702-1492 surname: Nag fullname: Nag, Avishek |
| BookMark | eNptkE1Lw0AQhhepYK29-AtyFqKTbDabPUpttVhaBOs1THYnZWubLZsE8d-bWFERB4b5euc5vOdsULmKGLuM4JpzBTeljThwgCw-YcNIKRUKBXzwqz9j47reQhdcxWkqh-zxxfqmxV2wpObN-ddg1la6sa4KpvuCjLHVJmgrQz5YOtPJVm2DGwrWdX-4IzoET-GC0FfdfMFOS9zVNP6qI7aeTZ8nD-FidT-f3C5CzTlvwgJBSVEoTEGjSbtEBUqAzgpleEYglNE6kVnfYElCcg4ySWRKEsqY-IjNj1zjcJsfvN2jf88d2vxz4fwmR99YvaNcxHFRlpHGIuEJosooQlPEQkmKuMhMx7o6srR3de2p_OZFkPeu5j-udmL4I9a2wd6txqPd_ffyAQqKewM |
| CitedBy_id | crossref_primary_10_1016_j_comcom_2022_08_005 crossref_primary_10_3390_fi15020078 crossref_primary_10_1109_ACCESS_2022_3215744 |
| Cites_doi | 10.1007/s00521-019-04376-6 10.1109/TNSM.2018.2867827 10.1109/ANTS50601.2020.9342803 10.1109/TNSM.2019.2947905 10.1109/TNSM.2018.2876623 10.23919/INM.2017.7987271 10.1109/COMST.2015.2477041 10.1109/JSAC.2011.111002 10.1109/MNET.001.1900287 10.1109/TNET.2018.2890631 10.1109/COMST.2019.2916583 10.1016/j.comnet.2018.01.028 10.1109/MCOM.2019.1900271 10.1109/JSAC.2018.2815318 10.1109/NETSOFT.2015.7116120 10.1109/TNSM.2016.2598420 10.1038/nature14236 10.1109/INM.2015.7140281 10.1109/JSAC.2019.2906744 10.1109/JSAC.2017.2760162 |
| ContentType | Journal Article |
| DBID | AAYXX CITATION DOA |
| DOI | 10.3390/fi13030082 |
| DatabaseName | CrossRef DOAJ Open Access Full Text |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | CrossRef |
| Database_xml | – sequence: 1 dbid: DOA name: DOAJ Open Access Full Text url: https://www.doaj.org/ sourceTypes: Open Website |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1999-5903 |
| ExternalDocumentID | oai_doaj_org_article_522bff1cab434aa98e1adb2597e1358d 10_3390_fi13030082 |
| GroupedDBID | -DT .4I 5VS 7WY 8FE 8FG 8FL AADQD AAFWJ AAKPC AAYXX ABDBF ABUWG ACIHN ADBBV ADMLS AEAQA AFFHD AFKRA AFPKN AFZYC ALMA_UNASSIGNED_HOLDINGS ARAPS AZQEC BCNDV BENPR BEZIV BGLVJ BPHCQ CCPQU CITATION DWQXO E3Z EAP EBS EJD ESX FRNLG GNUQQ GROUPED_DOAJ HCIFZ IAO K60 K6V K6~ K7- KQ8 M0C MODMG M~E OK1 P62 PHGZM PHGZT PIMPY PQBIZ PQBZA PQGLB PQQKQ PROAC RNS TR2 |
| ID | FETCH-LOGICAL-c333t-ba0975b9a60cad6cada90950c8b9d38e059dcc478059dafe5733074476e70f2e3 |
| IEDL.DBID | DOA |
| ISICitedReferencesCount | 5 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000633683400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1999-5903 |
| IngestDate | Fri Oct 03 12:53:18 EDT 2025 Sat Nov 29 07:08:05 EST 2025 Tue Nov 18 22:16:36 EST 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 3 |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c333t-ba0975b9a60cad6cada90950c8b9d38e059dcc478059dafe5733074476e70f2e3 |
| ORCID | 0000-0001-5508-8757 0000-0002-8358-2258 0000-0003-1702-1492 |
| OpenAccessLink | https://doaj.org/article/522bff1cab434aa98e1adb2597e1358d |
| ParticipantIDs | doaj_primary_oai_doaj_org_article_522bff1cab434aa98e1adb2597e1358d crossref_primary_10_3390_fi13030082 crossref_citationtrail_10_3390_fi13030082 |
| PublicationCentury | 2000 |
| PublicationDate | 2021-03-01 |
| PublicationDateYYYYMMDD | 2021-03-01 |
| PublicationDate_xml | – month: 03 year: 2021 text: 2021-03-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationTitle | Future internet |
| PublicationYear | 2021 |
| Publisher | MDPI AG |
| Publisher_xml | – name: MDPI AG |
| References | Gupta (ref_21) 2018; 133 Yuan (ref_15) 2020; 32 Saad (ref_1) 2019; 34 Knight (ref_22) 2011; 29 Quang (ref_7) 2018; 16 Mijumbi (ref_2) 2015; 18 Sciancalepore (ref_16) 2018; 15 ref_14 Quang (ref_5) 2019; 16 ref_24 Herrera (ref_4) 2016; 13 ref_10 Jang (ref_13) 2017; 35 Mnih (ref_20) 2015; 518 Letaief (ref_3) 2019; 57 ref_19 Nejad (ref_6) 2018; 36 Agarwal (ref_12) 2019; 27 ref_17 ref_9 ref_8 Dehury (ref_11) 2019; 37 Luong (ref_18) 2019; 21 (ref_23) 1959; 6 |
| References_xml | – volume: 32 start-page: 1995 year: 2020 ident: ref_15 article-title: A Q-learning-based approach for virtual network embedding in data center publication-title: Neural Comput. Appl. doi: 10.1007/s00521-019-04376-6 – volume: 15 start-page: 1292 year: 2018 ident: ref_16 article-title: z-TORCH: An automated NFV orchestration and monitoring solution publication-title: IEEE Trans. Netw. Serv. Manag. doi: 10.1109/TNSM.2018.2867827 – ident: ref_17 doi: 10.1109/ANTS50601.2020.9342803 – volume: 16 start-page: 1318 year: 2019 ident: ref_5 article-title: A deep reinforcement learning approach for VNF Forwarding Graph Embedding publication-title: IEEE Trans. Netw. Serv. Manag. doi: 10.1109/TNSM.2019.2947905 – volume: 16 start-page: 98 year: 2018 ident: ref_7 article-title: Single and multi-domain adaptive allocation algorithms for vnf forwarding graph embedding publication-title: IEEE Trans. Netw. Serv. Manag. doi: 10.1109/TNSM.2018.2876623 – ident: ref_9 doi: 10.23919/INM.2017.7987271 – ident: ref_24 – volume: 18 start-page: 236 year: 2015 ident: ref_2 article-title: Network function virtualization: State-of-the-art and research challenges publication-title: IEEE Commun. Surv. Tutor. doi: 10.1109/COMST.2015.2477041 – ident: ref_10 – volume: 29 start-page: 1765 year: 2011 ident: ref_22 article-title: The Internet Topology Zoo publication-title: Sel. Areas Commun. IEEE J. doi: 10.1109/JSAC.2011.111002 – volume: 34 start-page: 134 year: 2019 ident: ref_1 article-title: A vision of 6G wireless systems: Applications, trends, technologies, and open research problems publication-title: IEEE Netw. doi: 10.1109/MNET.001.1900287 – volume: 27 start-page: 433 year: 2019 ident: ref_12 article-title: VNF placement and resource allocation for the support of vertical services in 5G networks publication-title: IEEE/ACM Trans. Netw. doi: 10.1109/TNET.2018.2890631 – volume: 21 start-page: 3133 year: 2019 ident: ref_18 article-title: Applications of deep reinforcement learning in communications and networking: A survey publication-title: IEEE Commun. Surv. Tutor. doi: 10.1109/COMST.2019.2916583 – volume: 133 start-page: 1 year: 2018 ident: ref_21 article-title: On service-chaining strategies using virtual network functions in operator networks publication-title: Comput. Netw. doi: 10.1016/j.comnet.2018.01.028 – volume: 57 start-page: 84 year: 2019 ident: ref_3 article-title: The roadmap to 6G: AI empowered wireless networks publication-title: IEEE Commun. Mag. doi: 10.1109/MCOM.2019.1900271 – volume: 36 start-page: 542 year: 2018 ident: ref_6 article-title: vSPACE: VNF simultaneous placement, admission control and embedding publication-title: IEEE J. Sel. Areas Commun. doi: 10.1109/JSAC.2018.2815318 – ident: ref_14 doi: 10.1109/NETSOFT.2015.7116120 – ident: ref_19 – volume: 13 start-page: 518 year: 2016 ident: ref_4 article-title: Resource allocation in NFV: A comprehensive survey publication-title: IEEE Trans. Netw. Serv. Manag. doi: 10.1109/TNSM.2016.2598420 – volume: 518 start-page: 529 year: 2015 ident: ref_20 article-title: Human-level control through deep reinforcement learning publication-title: Nature doi: 10.1038/nature14236 – ident: ref_8 doi: 10.1109/INM.2015.7140281 – volume: 37 start-page: 1029 year: 2019 ident: ref_11 article-title: DYVINE: Fitness-based dynamic virtual network embedding in cloud computing publication-title: IEEE J. Sel. Areas Commun. doi: 10.1109/JSAC.2019.2906744 – volume: 35 start-page: 2532 year: 2017 ident: ref_13 article-title: Joint optimization of service function placement and flow distribution for service function chaining publication-title: IEEE J. Sel. Areas Commun. doi: 10.1109/JSAC.2017.2760162 – volume: 6 start-page: 18 year: 1959 ident: ref_23 article-title: On random graphs I publication-title: Publ. Math. Debr. |
| SSID | ssj0000392667 |
| Score | 2.1862338 |
| Snippet | With the emergence of various types of applications such as delay-sensitive applications, future communication networks are expected to be increasingly complex... |
| SourceID | doaj crossref |
| SourceType | Open Website Enrichment Source Index Database |
| StartPage | 82 |
| SubjectTerms | deep Q-learning network function virtualization virtual network function forwarding graph embedding (VNF-FGE) |
| Title | Virtual Network Function Embedding under Nodal Outage Using Deep Q-Learning |
| URI | https://doaj.org/article/522bff1cab434aa98e1adb2597e1358d |
| Volume | 13 |
| WOSCitedRecordID | wos000633683400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: DOAJ Open Access Full Text customDbUrl: eissn: 1999-5903 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000392667 issn: 1999-5903 databaseCode: DOA dateStart: 20090101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 1999-5903 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000392667 issn: 1999-5903 databaseCode: M~E dateStart: 20090101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre – providerCode: PRVPQU databaseName: ABI/INFORM Global (OCUL) customDbUrl: eissn: 1999-5903 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000392667 issn: 1999-5903 databaseCode: M0C dateStart: 20090101 isFulltext: true titleUrlDefault: https://search.proquest.com/abiglobal providerName: ProQuest – providerCode: PRVPQU databaseName: Advanced Technologies & Aerospace Database customDbUrl: eissn: 1999-5903 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000392667 issn: 1999-5903 databaseCode: P5Z dateStart: 20090101 isFulltext: true titleUrlDefault: https://search.proquest.com/hightechjournals providerName: ProQuest – providerCode: PRVPQU databaseName: Computer Science Database customDbUrl: eissn: 1999-5903 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000392667 issn: 1999-5903 databaseCode: K7- dateStart: 20090101 isFulltext: true titleUrlDefault: http://search.proquest.com/compscijour providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest ABI/INFORM Collection customDbUrl: eissn: 1999-5903 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000392667 issn: 1999-5903 databaseCode: 7WY dateStart: 20090101 isFulltext: true titleUrlDefault: https://www.proquest.com/abicomplete providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Central (subscription) customDbUrl: eissn: 1999-5903 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000392667 issn: 1999-5903 databaseCode: BENPR dateStart: 20090101 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Publicly Available Content customDbUrl: eissn: 1999-5903 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000392667 issn: 1999-5903 databaseCode: PIMPY dateStart: 20090101 isFulltext: true titleUrlDefault: http://search.proquest.com/publiccontent providerName: ProQuest |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LS8NAEF6ketCD-MT6KAt68bA0yW72cfTRooixgko9hX1KQdsSU3-_u0msFQQvHnIZhhC-DfN9A7PfAHCCfZzKWCEnqUWEGIe4dQZJH9VEUq2tqZZNsCzjw6EYLKz6CjNhtT1wDVzX6wPlXKylIphIKbiNpVFetDMb45SbUH0jJhaaqaoGe9qnlNV-pNj39V03CtU6MN4PBlow6q8Ypb8B1hspCM_qT9gES3a8BdYWDAK3wc3TqAgXPGBWT2vDvqehACXsvSlrAu_AcAusgNnE-LS7WenrA6zmAOCltVN4jxoH1Zcd8NjvPVxcoWb9AdIY4xIpGQmWKiFppKWh_pHCC6JIcyUM5tYLI6M1CUsJhJHOBmdDLwgIo5ZFLrF4F7TGk7HdAzDRXMuUaitiQSSXQqXYEWaUiR3XSdIGp1-Q5LrxBg8rKl5z3yME-PJv-NrgeJ47rR0xfs06D8jOM4KLdRXwZ5s3Z5v_dbb7__GSA7CahDmUam7sELTKYmaPwIr-KEfvRaf6bTpgeXB9O3j-BE3Vy7Q |
| linkProvider | Directory of Open Access Journals |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Virtual+Network+Function+Embedding+under+Nodal+Outage+Using+Deep+Q-Learning&rft.jtitle=Future+internet&rft.au=Chetty%2C+Swarna+Bindu&rft.au=Ahmadi%2C+Hamed&rft.au=Sharma%2C+Sachin&rft.au=Nag%2C+Avishek&rft.date=2021-03-01&rft.issn=1999-5903&rft.eissn=1999-5903&rft.volume=13&rft.issue=3&rft.spage=82&rft_id=info:doi/10.3390%2Ffi13030082&rft.externalDBID=n%2Fa&rft.externalDocID=10_3390_fi13030082 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1999-5903&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1999-5903&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1999-5903&client=summon |