NFVdeep adaptive online service function chain deployment with deep reinforcement learning

With the evolution of network function virtualization (NFV), diverse network services can be flexibly offered as service function chains (SFCs) consisted of different virtual network functions (VNFs). However, network state and traffic typically exhibit unpredictable variations due to stochastically...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings of the International Symposium on Quality of Service s. 1 - 10
Hlavní autoři: Xiao, Yikai, Zhang, Qixia, Liu, Fangming, Wang, Jia, Zhao, Miao, Zhang, Zhongxing, Zhang, Jiaxing
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: New York, NY, USA ACM 24.06.2019
Edice:ACM Other Conferences
Témata:
Computing methodologies > Machine learning
Deep Reinforcement Learning
Network Function Virtualization (NFV)
Networks > Network algorithms > Control path algorithms > Network resources allocation
Networks > Network components > Middle boxes > network appliances
Networks > Network properties > Network dynamics
QoS-Aware Resource Management
Service Function Chain
service function chain
deep reinforcement learning
network function virtualization (NFV)
QoS-aware resource management
ISBN:9781450367783, 145036778X
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Abstract With the evolution of network function virtualization (NFV), diverse network services can be flexibly offered as service function chains (SFCs) consisted of different virtual network functions (VNFs). However, network state and traffic typically exhibit unpredictable variations due to stochastically arriving requests with different quality of service (QoS) requirements. Thus, an adaptive online SFC deployment approach is needed to handle the real-time network variations and various service requests. In this paper, we firstly introduce a Markov decision process (MDP) model to capture the dynamic network state transitions. In order to jointly minimize the operation cost of NFV providers and maximize the total throughput of requests, we propose NFVdeep, an adaptive, online, deep reinforcement learning approach to automatically deploy SFCs for requests with different QoS requirements. Specifically, we use a serialization-and-backtracking method to effectively deal with large discrete action space. We also adopt a policy gradient based method to improve the training efficiency and convergence to optimality. Extensive experimental results demonstrate that NFVdeep converges fast in the training process and responds rapidly to arriving requests especially in large, frequently transferred network state space. Consequently, NFVdeep surpasses the state-of-the-art methods by 32.59% higher accepted throughput and 33.29% lower operation cost on average.
AbstractList With the evolution of network function virtualization (NFV), diverse network services can be flexibly offered as service function chains (SFCs) consisted of different virtual network functions (VNFs). However, network state and traffic typically exhibit unpredictable variations due to stochastically arriving requests with different quality of service (QoS) requirements. Thus, an adaptive online SFC deployment approach is needed to handle the real-time network variations and various service requests. In this paper, we firstly introduce a Markov decision process (MDP) model to capture the dynamic network state transitions. In order to jointly minimize the operation cost of NFV providers and maximize the total throughput of requests, we propose NFVdeep, an adaptive, online, deep reinforcement learning approach to automatically deploy SFCs for requests with different QoS requirements. Specifically, we use a serialization-and-backtracking method to effectively deal with large discrete action space. We also adopt a policy gradient based method to improve the training efficiency and convergence to optimality. Extensive experimental results demonstrate that NFVdeep converges fast in the training process and responds rapidly to arriving requests especially in large, frequently transferred network state space. Consequently, NFVdeep surpasses the state-of-the-art methods by 32.59% higher accepted throughput and 33.29% lower operation cost on average.
Author Xiao, Yikai
Zhang, Zhongxing
Zhang, Jiaxing
Wang, Jia
Zhao, Miao
Zhang, Qixia
Liu, Fangming
Author_xml – sequence: 1
  givenname: Yikai
  surname: Xiao
  fullname: Xiao, Yikai
  organization: Huazhong University of Science and Technology, China
– sequence: 2
  givenname: Qixia
  surname: Zhang
  fullname: Zhang, Qixia
  organization: Huazhong University of Science and Technology, China
– sequence: 3
  givenname: Fangming
  surname: Liu
  fullname: Liu, Fangming
  email: fmliu@hust.edu.cn
  organization: Huazhong University of Science and Technology, China
– sequence: 4
  givenname: Jia
  surname: Wang
  fullname: Wang, Jia
  organization: The Hong Kong Polytechnic University, Hong Kong
– sequence: 5
  givenname: Miao
  surname: Zhao
  fullname: Zhao, Miao
  organization: The Hong Kong Polytechnic University, Hong Kong
– sequence: 6
  givenname: Zhongxing
  surname: Zhang
  fullname: Zhang, Zhongxing
  organization: Huazhong University of Science and Technology, China
– sequence: 7
  givenname: Jiaxing
  surname: Zhang
  fullname: Zhang, Jiaxing
  organization: Huazhong University of Science and Technology, China
BookMark eNqNj7tOxDAQRY0A8VhSU_ADNAkeTzy2S7RiAWkFDdBafoylALtZJTT8PUGbiorq6s65Gumci6Ntv2UhLkE2AK2-QVSkrG6mdFLTgaicsROQSMZYPPzTT0U1ju9SSmUNtBLPxMnT6i0z7y7EcQmfI1dzLsTr6u5l-VCvn-8fl7frOiDYr5pURBswFR0SuQLaFCqxzUXBdLDFZGATgySXQDERJJucAYwpqYyYcSGu9n87Zva7oduE4ds7SZawnej1noa08bHvP0YP0v-a-tnUz6bTtPnn1Meh44I_JmhP1A
ContentType Conference Proceeding
Copyright 2019 ACM
Copyright_xml – notice: 2019 ACM
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1145/3326285.3329056
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList

Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 9781450367783
145036778X
EndPage 10
ExternalDocumentID 9068634
Genre orig-research
GroupedDBID 6IE
6IF
6IL
6IN
ABLEC
ACM
ADPZR
ALMA_UNASSIGNED_HOLDINGS
APO
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
GUFHI
IEGSK
OCL
RIE
RIL
AAWTH
LHSKQ
ID FETCH-LOGICAL-a318t-62b38a3cf5ac69f157f6fb4df21ac68f7d1e7ba069c12e661c8c9713bcc2d33d3
IEDL.DBID RIE
ISBN 9781450367783
145036778X
ISICitedReferencesCount 156
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000709368800021&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Tue Nov 04 05:03:12 EST 2025
Wed Jan 31 06:47:06 EST 2024
IsPeerReviewed false
IsScholarly true
Keywords service function chain
deep reinforcement learning
network function virtualization (NFV)
QoS-aware resource management
Language English
License Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org
LinkModel DirectLink
MeetingName IWQoS '19: IEEE/ACM International Symposium on Quality of Service
MergedId FETCHMERGED-LOGICAL-a318t-62b38a3cf5ac69f157f6fb4df21ac68f7d1e7ba069c12e661c8c9713bcc2d33d3
PageCount 10
ParticipantIDs acm_books_10_1145_3326285_3329056
ieee_primary_9068634
acm_books_10_1145_3326285_3329056_brief
PublicationCentury 2000
PublicationDate 20190624
2019-June
PublicationDateYYYYMMDD 2019-06-24
2019-06-01
PublicationDate_xml – month: 06
  year: 2019
  text: 20190624
  day: 24
PublicationDecade 2010
PublicationPlace New York, NY, USA
PublicationPlace_xml – name: New York, NY, USA
PublicationSeriesTitle ACM Other Conferences
PublicationTitle Proceedings of the International Symposium on Quality of Service
PublicationTitleAbbrev IWQoS
PublicationYear 2019
Publisher ACM
Publisher_xml – name: ACM
SSID ssj0002871403
Score 2.5017133
Snippet With the evolution of network function virtualization (NFV), diverse network services can be flexibly offered as service function chains (SFCs) consisted of...
SourceID ieee
acm
SourceType Publisher
StartPage 1
SubjectTerms Computing methodologies -- Machine learning
Deep Reinforcement Learning
Network Function Virtualization (NFV)
Networks -- Network algorithms -- Control path algorithms -- Network resources allocation
Networks -- Network components -- Middle boxes -- network appliances
Networks -- Network properties -- Network dynamics
QoS-Aware Resource Management
Service Function Chain
Subtitle adaptive online service function chain deployment with deep reinforcement learning
Title NFVdeep
URI https://ieeexplore.ieee.org/document/9068634
WOSCitedRecordID wos000709368800021&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3LS8MwHA7b8OBJZRPniwiCF7u1eTSpN5kWT2OIysBDyeMX3cGt7OHfb9KWiSCIp6YhNPC16e-RfN8PoUvNwL9ZRqJEWR4xI2ikLNWRBBPk4lRMrK2KTYjxWE6n2aSFrrdcGACoDp_BIDSrvXy7MJuQKhtmgc9AWRu1hRA1V2ubTwmeP4tpo96TMD6k3jMhkg_8NYtDgeq2Mh8_iqhUNiTf-9_s-6j3TcbDk62ZOUAtmHfR6zh_sQDlDb61qgy_LFyLhuJm8ePcG6wAOh69--Af30Go7BtmwCH16u-hxI9Q6aaaKkWIG6nVtx56zu-fRg9RUychUn5FrqOUaCoVNY4rk2Yu4cKlTjPrSOI7pBM2AaFVnGYmIeANspEm88GpNoZYSi09RJ35Yg5HCMeOJQKs5cpKpon17qSWgmgnleYWXB9deBCLEACsiprTzIsG6KIBuo-u_hxT6OUsPK0bYC7KWlijaBA-_r37BO16byWrz2mdos56uYEztGM-17PV8rz6Gr4AgFmy5w
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bS8MwGA3bFPRJZRPnNYLgi93aXNbUN5mOiXMMmTLwoeTyRffgNnbx95u0ZSII4lPTEBo4bfpdknM-hC4UA_dmGQkiaXjAdEwDaagKBGgvFydDYkxWbCLu98VolAxK6GrNhQGA7PAZNHwz28s3U73yqbJm4vkMlJXRBmeMRDlba51R8b4_C2mh3xMx3qTONyGCN9w1CX2J6rLUHz_KqGRWpLPzv_l3Ue2bjocHa0Ozh0owqaLXfufFAMyu8Y2RM__TwrlsKC6WP-44k-Vhx-13F_7jW_C1ff0M2Cdf3T3M8BNkyqk6SxLiQmz1rYaeO3fDdjcoKiUE0q3JZdAiigpJteVStxIb8di2rGLGksh1CBubCGIlw1aiIwLOJGuhExeeKq2JodTQfVSZTCdwgHBoWRSDMVwawRQxzqFUIibKCqm4AVtH5w7E1IcAizRnNfO0ADotgK6jyz_HpGo-9k-repjTWS6tkRYIH_7efYa2usPHXtq77z8coW3nuyT5qa1jVFnOV3CCNvXncryYn2Zfxhf9DrYu
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+of+the+International+Symposium+on+Quality+of+Service&rft.atitle=NFVdeep&rft.au=Xiao%2C+Yikai&rft.au=Zhang%2C+Qixia&rft.au=Liu%2C+Fangming&rft.au=Wang%2C+Jia&rft.series=ACM+Other+Conferences&rft.date=2019-06-24&rft.pub=ACM&rft.isbn=9781450367783&rft.spage=1&rft.epage=10&rft_id=info:doi/10.1145%2F3326285.3329056
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=9781450367783/lc.gif&client=summon&freeimage=true
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=9781450367783/mc.gif&client=summon&freeimage=true
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=9781450367783/sc.gif&client=summon&freeimage=true