Deep Reinforcement Learning for Cooperative Content Caching in Vehicular Edge Computing and Networks

In this article, we propose a cooperative edge caching scheme, a new paradigm to jointly optimize the content placement and content delivery in the vehicular edge computing and networks, with the aid of the flexible trilateral cooperations among a macro-cell station, roadside units, and smart vehicl...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE internet of things journal Jg. 7; H. 1; S. 247 - 257
Hauptverfasser:	Qiao, Guanhua, Leng, Supeng, Maharjan, Sabita, Zhang, Yan, Ansari, Nirwan
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	Piscataway IEEE 01.01.2020 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:	Base stations Caching Computational modeling Computer simulation Content delivery content placement Cooperative caching cooperative edge caching deep deterministic policy gradient (DDPG) double time-scale Markov decision process (DTS-MDP) Edge computing Indexes Integer programming Intelligent vehicles Internet of Things Linear programming Machine learning Markov processes Mixed integer Optimization Placement Roadsides vehicular edge computing and networks
ISSN:	2327-4662, 2327-4662
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Abstract	In this article, we propose a cooperative edge caching scheme, a new paradigm to jointly optimize the content placement and content delivery in the vehicular edge computing and networks, with the aid of the flexible trilateral cooperations among a macro-cell station, roadside units, and smart vehicles. We formulate the joint optimization problem as a double time-scale Markov decision process (DTS-MDP), based on the fact that the time-scale of content timeliness changes less frequently as compared to the vehicle mobility and network states during the content delivery process. At the beginning of the large time-scale, the content placement/updating decision can be obtained according to the content popularity, vehicle driving paths, and resource availability. On the small time-scale, the joint vehicle scheduling and bandwidth allocation scheme is designed to minimize the content access cost while satisfying the constraint on content delivery latency. To solve the long-term mixed integer linear programming (LT-MILP) problem, we propose a nature-inspired method based on the deep deterministic policy gradient (DDPG) framework to obtain a suboptimal solution with a low computation complexity. The simulation results demonstrate that the proposed cooperative caching system can reduce the system cost, as well as the content delivery latency, and improve content hit ratio, as compared to the noncooperative and random edge caching schemes.
AbstractList	In this article, we propose a cooperative edge caching scheme, a new paradigm to jointly optimize the content placement and content delivery in the vehicular edge computing and networks, with the aid of the flexible trilateral cooperations among a macro-cell station, roadside units, and smart vehicles. We formulate the joint optimization problem as a double time-scale Markov decision process (DTS-MDP), based on the fact that the time-scale of content timeliness changes less frequently as compared to the vehicle mobility and network states during the content delivery process. At the beginning of the large time-scale, the content placement/updating decision can be obtained according to the content popularity, vehicle driving paths, and resource availability. On the small time-scale, the joint vehicle scheduling and bandwidth allocation scheme is designed to minimize the content access cost while satisfying the constraint on content delivery latency. To solve the long-term mixed integer linear programming (LT-MILP) problem, we propose a nature-inspired method based on the deep deterministic policy gradient (DDPG) framework to obtain a suboptimal solution with a low computation complexity. The simulation results demonstrate that the proposed cooperative caching system can reduce the system cost, as well as the content delivery latency, and improve content hit ratio, as compared to the noncooperative and random edge caching schemes.
Author	Qiao, Guanhua Ansari, Nirwan Zhang, Yan Maharjan, Sabita Leng, Supeng
Author_xml	– sequence: 1 givenname: Guanhua surname: Qiao fullname: Qiao, Guanhua email: qghuestc@126.com organization: School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, China – sequence: 2 givenname: Supeng orcidid: 0000-0003-0049-5982 surname: Leng fullname: Leng, Supeng email: spleng@uestc.edu.cn organization: School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, China – sequence: 3 givenname: Sabita orcidid: 0000-0002-4616-8488 surname: Maharjan fullname: Maharjan, Sabita email: sabita@simula.no organization: Center for Resilient Networks and Applications, Simula Metropolitan Center for Digital Engineering, Oslo, Norway – sequence: 4 givenname: Yan orcidid: 0000-0002-8561-5092 surname: Zhang fullname: Zhang, Yan email: yanzhang@ieee.org organization: Department of Informatics, University of Oslo, Oslo, Norway – sequence: 5 givenname: Nirwan orcidid: 0000-0001-8541-3565 surname: Ansari fullname: Ansari, Nirwan email: nirwan.ansari@njit.edu organization: Department of Electrical and Computer Engineering, Advanced Networking Laboratory, New Jersey Institute of Technology, Newark, NJ, USA
BookMark	eNp9kFFLwzAQx4NMcM59APGl4HNn0jRp8yh16mQ4kOlrqMl1y9zSmqaK396UDREffMqR-_3uuP8pGtjaAkLnBE8IweLqYbZYThJMxCQRKeMpPkLDhCZZnHKeDH7VJ2jcthuMcdAYEXyI9A1AEz2BsVXtFOzA-mgOpbPGrqLwFRV13YArvfmAUFvfA0Wp1n3f2OgF1kZ129JFU73qiV3T-b5XWh09gv-s3Vt7ho6rctvC-PCO0PPtdFncx_PF3ay4nseKMuFjygUI4DgXmuiKUk0xZYA55jpjimgKIlOKKFUxzHQFVJMKXgORJlrRIIzQ5X5u4-r3DlovN3XnbFgpE0pzzgkVJFDZnlKublsHlVTGhwPDca40W0mw7FOVfaqyT1UeUg0m-WM2zuxK9_Wvc7F3DAD88HmeCZZR-g2QT4Vy
CODEN	IITJAU
CitedBy_id	crossref_primary_10_1145_3664613 crossref_primary_10_1016_j_future_2024_107704 crossref_primary_10_1109_TITS_2020_3015210 crossref_primary_10_1109_JIOT_2024_3465553 crossref_primary_10_1155_2020_8837751 crossref_primary_10_1109_ACCESS_2022_3143815 crossref_primary_10_1109_JSYST_2024_3442958 crossref_primary_10_1007_s10586_024_04305_w crossref_primary_10_1109_JIOT_2024_3401219 crossref_primary_10_1109_JIOT_2024_3366009 crossref_primary_10_1109_TMC_2024_3367543 crossref_primary_10_1109_COMST_2024_3442149 crossref_primary_10_1109_JIOT_2021_3081480 crossref_primary_10_1109_MNET_011_2000663 crossref_primary_10_1109_TITS_2023_3250320 crossref_primary_10_1109_TNET_2023_3278032 crossref_primary_10_1186_s13677_023_00472_0 crossref_primary_10_1016_j_sysarc_2021_102000 crossref_primary_10_1016_j_vehcom_2021_100398 crossref_primary_10_1109_TMC_2024_3403851 crossref_primary_10_1016_j_vehcom_2021_100397 crossref_primary_10_1016_j_vehcom_2021_100399 crossref_primary_10_1109_TMM_2024_3366399 crossref_primary_10_3390_app12199987 crossref_primary_10_1109_TNSM_2021_3123959 crossref_primary_10_1016_j_dcan_2024_12_006 crossref_primary_10_1109_TITS_2020_3017172 crossref_primary_10_1109_TSC_2021_3106260 crossref_primary_10_1109_JIOT_2024_3388297 crossref_primary_10_1016_j_vehcom_2021_100391 crossref_primary_10_1155_2020_8842694 crossref_primary_10_1007_s00500_023_09354_8 crossref_primary_10_1109_TVT_2021_3132961 crossref_primary_10_1007_s11518_024_5604_1 crossref_primary_10_1016_j_adhoc_2023_103386 crossref_primary_10_1016_j_jnca_2024_103886 crossref_primary_10_1109_TWC_2023_3272348 crossref_primary_10_1145_3653975 crossref_primary_10_1145_3447032 crossref_primary_10_1109_JIOT_2021_3098003 crossref_primary_10_1016_j_comnet_2023_110075 crossref_primary_10_1016_j_neucom_2023_126258 crossref_primary_10_1109_TWC_2024_3467285 crossref_primary_10_1109_JIOT_2024_3506713 crossref_primary_10_3390_electronics11142130 crossref_primary_10_1109_JSEN_2024_3436612 crossref_primary_10_1109_TMC_2021_3123135 crossref_primary_10_1109_TVT_2022_3210570 crossref_primary_10_1109_ACCESS_2020_2985731 crossref_primary_10_1049_itr2_12255 crossref_primary_10_1109_TITS_2025_3549493 crossref_primary_10_1109_JSAC_2024_3365893 crossref_primary_10_1109_JIOT_2021_3078701 crossref_primary_10_1109_TPDS_2022_3222509 crossref_primary_10_3390_informatics7040043 crossref_primary_10_1109_JIOT_2020_2997720 crossref_primary_10_1109_TCOMM_2022_3161021 crossref_primary_10_3390_rs15112801 crossref_primary_10_1109_JIOT_2022_3161571 crossref_primary_10_1109_TMC_2022_3193499 crossref_primary_10_1016_j_comcom_2022_12_021 crossref_primary_10_1109_TVT_2021_3092346 crossref_primary_10_1109_TIV_2023_3344478 crossref_primary_10_1007_s10586_022_03803_z crossref_primary_10_1109_JIOT_2021_3089280 crossref_primary_10_1016_j_adhoc_2023_103313 crossref_primary_10_3390_computers12030063 crossref_primary_10_1016_j_comnet_2024_110395 crossref_primary_10_3390_app13020780 crossref_primary_10_1109_TVT_2024_3384952 crossref_primary_10_1016_j_vehcom_2025_100912 crossref_primary_10_1109_JIOT_2021_3094507 crossref_primary_10_1109_TCOMM_2020_3044298 crossref_primary_10_1109_TII_2022_3186039 crossref_primary_10_1109_MCOMSTD_001_2000045 crossref_primary_10_1109_TWC_2024_3462760 crossref_primary_10_3390_s23010499 crossref_primary_10_1109_ACCESS_2023_3297280 crossref_primary_10_1109_ACCESS_2022_3140719 crossref_primary_10_1109_TVT_2021_3133291 crossref_primary_10_3390_s23249687 crossref_primary_10_1109_TVT_2023_3234336 crossref_primary_10_1016_j_jestch_2024_101699 crossref_primary_10_1109_JIOT_2023_3264281 crossref_primary_10_1109_TITS_2022_3224395 crossref_primary_10_1109_TVT_2020_3029109 crossref_primary_10_1109_JSAC_2021_3087232 crossref_primary_10_1109_TWC_2022_3204738 crossref_primary_10_1109_TIV_2023_3323217 crossref_primary_10_1109_TVT_2021_3121096 crossref_primary_10_1016_j_tra_2020_08_005 crossref_primary_10_1016_j_eswa_2023_122834 crossref_primary_10_1109_TNSM_2021_3136439 crossref_primary_10_1155_2021_3759631 crossref_primary_10_1109_JIOT_2023_3324381 crossref_primary_10_1109_JIOT_2022_3229009 crossref_primary_10_1109_JIOT_2021_3061467 crossref_primary_10_1155_2022_3501329 crossref_primary_10_1016_j_comnet_2022_108797 crossref_primary_10_1109_LCOMM_2022_3208941 crossref_primary_10_1109_TVT_2023_3244966 crossref_primary_10_1109_TITS_2020_3017474 crossref_primary_10_1109_JIOT_2022_3156100 crossref_primary_10_1109_TNSE_2022_3153172 crossref_primary_10_1109_JIOT_2023_3235661 crossref_primary_10_1016_j_adhoc_2025_103888 crossref_primary_10_1016_j_comnet_2024_110761 crossref_primary_10_1016_j_vehcom_2023_100689 crossref_primary_10_1007_s11042_023_15590_9 crossref_primary_10_3390_s22186995 crossref_primary_10_1016_j_tcs_2024_114462 crossref_primary_10_1109_TNSE_2024_3433544 crossref_primary_10_1016_j_future_2024_107679 crossref_primary_10_1109_TCC_2023_3234543 crossref_primary_10_1109_JSTSP_2022_3221271 crossref_primary_10_1109_JIOT_2021_3084912 crossref_primary_10_3390_pr10091838 crossref_primary_10_1016_j_sysarc_2024_103306 crossref_primary_10_1016_j_jnca_2020_102966 crossref_primary_10_1109_TNSM_2022_3198074 crossref_primary_10_1109_JIOT_2023_3317971 crossref_primary_10_1016_j_simpat_2023_102810 crossref_primary_10_1109_JIOT_2024_3389656 crossref_primary_10_1155_2022_6138434 crossref_primary_10_1109_JSYST_2021_3137011 crossref_primary_10_3390_s23031732 crossref_primary_10_1007_s00607_024_01396_6 crossref_primary_10_1109_TITS_2023_3263239 crossref_primary_10_1109_TNSM_2021_3086721 crossref_primary_10_1145_3448099 crossref_primary_10_1109_JIOT_2022_3176289 crossref_primary_10_1109_MWC_012_2100587 crossref_primary_10_1109_TVT_2023_3277191 crossref_primary_10_1109_TPDS_2024_3368763 crossref_primary_10_1109_JIOT_2020_3000300 crossref_primary_10_1109_TITS_2023_3264553 crossref_primary_10_1002_dac_70188 crossref_primary_10_1016_j_vehcom_2024_100860 crossref_primary_10_1109_JIOT_2023_3289994 crossref_primary_10_1007_s12083_021_01252_w crossref_primary_10_1109_TITS_2022_3184973 crossref_primary_10_1109_ACCESS_2020_2964029 crossref_primary_10_1016_j_comnet_2022_109513 crossref_primary_10_1109_ACCESS_2025_3559465 crossref_primary_10_1109_JIOT_2023_3292374 crossref_primary_10_1007_s12083_022_01369_6 crossref_primary_10_1109_MCE_2024_3358025 crossref_primary_10_1109_TITS_2021_3131240 crossref_primary_10_1109_TCE_2024_3454270 crossref_primary_10_1145_3461841 crossref_primary_10_1109_JIOT_2022_3151667 crossref_primary_10_1109_TVT_2023_3285073 crossref_primary_10_1631_FITEE_2400025 crossref_primary_10_1016_j_simpat_2024_103025 crossref_primary_10_1109_ACCESS_2020_3030943 crossref_primary_10_1109_TVT_2021_3077072 crossref_primary_10_1016_j_comcom_2024_107928 crossref_primary_10_1109_JIOT_2025_3545418 crossref_primary_10_1109_TVT_2022_3226764 crossref_primary_10_1109_COMST_2022_3149714 crossref_primary_10_1109_ACCESS_2020_3024683 crossref_primary_10_1109_JIOT_2023_3279048 crossref_primary_10_1109_TITS_2024_3392596 crossref_primary_10_1109_JIOT_2021_3109003 crossref_primary_10_1016_j_comcom_2023_09_005 crossref_primary_10_1109_TITS_2021_3087833 crossref_primary_10_1109_TSC_2023_3335412 crossref_primary_10_1109_TGCN_2022_3146145 crossref_primary_10_1109_TCCN_2020_3002253 crossref_primary_10_1016_j_jnca_2024_104039 crossref_primary_10_1109_JIOT_2023_3349255 crossref_primary_10_1109_TMC_2024_3481276 crossref_primary_10_1049_tje2_12250 crossref_primary_10_1109_TVT_2024_3357086 crossref_primary_10_1109_JIOT_2020_2981557 crossref_primary_10_1109_TITS_2020_3030496 crossref_primary_10_1109_ACCESS_2020_3011705 crossref_primary_10_1109_JIOT_2023_3265434 crossref_primary_10_1109_TSC_2022_3190375 crossref_primary_10_3390_fi14070192 crossref_primary_10_1109_JIOT_2021_3052542 crossref_primary_10_1109_JIOT_2021_3110913 crossref_primary_10_1088_1742_6596_2732_1_012001 crossref_primary_10_1145_3718361 crossref_primary_10_1007_s11280_022_01011_8 crossref_primary_10_1109_COMST_2023_3338153 crossref_primary_10_1109_JIOT_2021_3086623 crossref_primary_10_1016_j_icte_2024_05_012 crossref_primary_10_1109_MIC_2023_3284693 crossref_primary_10_1109_TCSS_2021_3068369 crossref_primary_10_1016_j_jnca_2024_104000 crossref_primary_10_1016_j_suscom_2025_101114 crossref_primary_10_1109_JIOT_2021_3087719 crossref_primary_10_1109_TITS_2020_3002712 crossref_primary_10_1109_TSC_2022_3203700 crossref_primary_10_1109_JIOT_2021_3138434 crossref_primary_10_1109_JIOT_2022_3209256 crossref_primary_10_1109_TITS_2022_3171071 crossref_primary_10_1109_TVT_2024_3479290 crossref_primary_10_3390_electronics12102294 crossref_primary_10_1007_s12559_024_10380_3 crossref_primary_10_1007_s10791_024_09476_z crossref_primary_10_1109_TVT_2024_3473738 crossref_primary_10_3390_sym16070906 crossref_primary_10_1109_JIOT_2021_3068427 crossref_primary_10_1109_JSAC_2020_3036946 crossref_primary_10_1109_TGCN_2021_3127923 crossref_primary_10_1109_MNET_021_2000292 crossref_primary_10_1016_j_jksuci_2023_101813 crossref_primary_10_1109_TCE_2025_3571029 crossref_primary_10_1109_TNSE_2023_3292570
Cites_doi	10.1109/SPAWC.2016.7536874 10.1109/JIOT.2019.2903191 10.1109/TITS.2011.2171951 10.1109/TVT.2017.2784562 10.1109/TMC.2016.2597851 10.1109/JSTSP.2017.2787979 10.1109/MWC.2018.1700303 10.1109/ACCESS.2017.2693440 10.1109/TVT.2015.2479942 10.1016/j.mechatronics.2017.10.010 10.1109/TSP.2011.2165211 10.1109/COMST.2018.2841349 10.1587/transcom.2017CQP0014 10.1109/MCOM.2014.6736746 10.1109/GLOCOM.2016.7842121 10.1109/MNET.2018.1800097 10.1109/GLOCOM.2016.7841723 10.1109/JIOT.2018.2866947 10.1109/ICC.2018.8422220 10.1109/TVT.2015.2480711 10.1109/TCOMM.2014.2337890 10.1007/978-1-4615-6331-0 10.1109/ACCESS.2017.2714191 10.1109/JSAC.2016.2545413 10.3390/s16070974 10.3390/jsan5040017
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020
DBID	97E RIA RIE AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D
DOI	10.1109/JIOT.2019.2945640
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE/IET Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Computer and Information Systems Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Advanced Technologies Database with Aerospace ProQuest Computer Science Collection Computer and Information Systems Abstracts Professional
DatabaseTitleList	Computer and Information Systems Abstracts
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Computer Science
EISSN	2327-4662
EndPage	257
ExternalDocumentID	10_1109_JIOT_2019_2945640 8879573
Genre	orig-research
GrantInformation_xml	– fundername: Fundamental Research Funds for the Central Universities grantid: ZYGX2016Z011 funderid: 10.13039/501100012226 – fundername: Science and Technology Program of Sichuan Province, China grantid: 2019YFH0007 – fundername: National Basic Research Program of China (973 Program); National Key Research and Development Program of China grantid: 2018YFC0807101 funderid: 10.13039/501100012166 – fundername: EU H2020 Project COSAFE grantid: MSCA-RISE-2018-824019
GroupedDBID	0R~ 4.4 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABJNI ABQJQ ABVLG AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD IFIPE IPLJI JAVBF M43 OCL PQQKQ RIA RIE AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c359t-369e9e6089d1df33d3035e0606d75c1d3e97cc1ccf505dfe3d1feb5e042dc3f33
IEDL.DBID	RIE
ISICitedReferencesCount	269
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000508181000019&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	2327-4662
IngestDate	Sun Oct 05 00:18:34 EDT 2025 Sat Nov 29 06:16:48 EST 2025 Tue Nov 18 22:20:16 EST 2025 Wed Aug 27 02:40:58 EDT 2025
IsPeerReviewed	false
IsScholarly	true
Issue	1
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c359t-369e9e6089d1df33d3035e0606d75c1d3e97cc1ccf505dfe3d1feb5e042dc3f33
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0003-0049-5982 0000-0002-4616-8488 0000-0002-8561-5092 0000-0001-8541-3565
PQID	2338661391
PQPubID	2040421
PageCount	11
ParticipantIDs	crossref_citationtrail_10_1109_JIOT_2019_2945640 crossref_primary_10_1109_JIOT_2019_2945640 proquest_journals_2338661391 ieee_primary_8879573
PublicationCentury	2000
PublicationDate	2020-Jan. 2020-1-00 20200101
PublicationDateYYYYMMDD	2020-01-01
PublicationDate_xml	– month: 01 year: 2020 text: 2020-Jan.
PublicationDecade	2020
PublicationPlace	Piscataway
PublicationPlace_xml	– name: Piscataway
PublicationTitle	IEEE internet of things journal
PublicationTitleAbbrev	JIoT
PublicationYear	2020
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref12 lillicrap (ref33) 2015 ref15 ref36 ref14 ref31 ref10 ref2 ref1 ref17 csaba (ref32) 2010 ref16 (ref34) 2016 ref19 (ref24) 2018 ref18 silver (ref30) 2014 zhang (ref28) 2018; e101 b ref26 ref25 konda (ref29) 1999; 13 zhang (ref20) 2018 ref22 ref21 (ref35) 2016 ref27 ref8 ref7 kleinberg (ref11) 2004 ref9 ref4 ref3 ref6 ref5 breslau (ref23) 2014
References_xml	– ident: ref7 doi: 10.1109/SPAWC.2016.7536874 – ident: ref2 doi: 10.1109/JIOT.2019.2903191 – start-page: 126 year: 2014 ident: ref23 article-title: Web caching and Zipf-like distribution: Evidence and implications publication-title: Proc IEEE Comput Commun Soc (INFOCOM) – year: 2015 ident: ref33 article-title: Continuous control with deep reinforcement learning publication-title: arXiv preprint arXiv 1509 02971 – ident: ref26 doi: 10.1109/TITS.2011.2171951 – start-page: 1 year: 2010 ident: ref32 article-title: Algorithms for reinforcement learning publication-title: Synthesis Lectures on Artificial Intelligence and Machine Learning – ident: ref10 doi: 10.1109/TVT.2017.2784562 – ident: ref13 doi: 10.1109/TMC.2016.2597851 – ident: ref22 doi: 10.1109/JSTSP.2017.2787979 – ident: ref3 doi: 10.1109/MWC.2018.1700303 – year: 2016 ident: ref35 publication-title: Reimplementation of DDPG(Continuous Control With Deep Reinforcement Learning) Based on OpenAI Gym + Tensorflow – ident: ref9 doi: 10.1109/ACCESS.2017.2693440 – ident: ref27 doi: 10.1109/TVT.2015.2479942 – ident: ref21 doi: 10.1016/j.mechatronics.2017.10.010 – ident: ref36 doi: 10.1109/TSP.2011.2165211 – ident: ref4 doi: 10.1109/COMST.2018.2841349 – volume: e101 b start-page: 1625 year: 2018 ident: ref28 article-title: A deep reinforcement learning based approach for cost- and energy-aware multi-flow mobile data offloading publication-title: IEICE Trans Commun doi: 10.1587/transcom.2017CQP0014 – ident: ref8 doi: 10.1109/MCOM.2014.6736746 – year: 2016 ident: ref34 publication-title: DC Data Sets – ident: ref15 doi: 10.1109/GLOCOM.2016.7842121 – ident: ref31 doi: 10.1109/MNET.2018.1800097 – year: 2018 ident: ref24 publication-title: Geohash Algorithm – year: 2004 ident: ref11 publication-title: Algorithm Design – ident: ref14 doi: 10.1109/GLOCOM.2016.7841723 – ident: ref16 doi: 10.1109/JIOT.2018.2866947 – ident: ref19 doi: 10.1109/ICC.2018.8422220 – ident: ref18 doi: 10.1109/TVT.2015.2480711 – start-page: 387 year: 2014 ident: ref30 article-title: Determinstic policy gradient algorithms publication-title: Proc Int Conf Mach Learn (ICML) – ident: ref25 doi: 10.1109/TCOMM.2014.2337890 – year: 2018 ident: ref20 article-title: Cache-enabled dynamic rate allocation via deep self-transfer reinforcement learning publication-title: arXiv 1893 11334v1 – volume: 13 start-page: 1008 year: 1999 ident: ref29 article-title: Actor-critic algorithm publication-title: Proc NIPS – ident: ref12 doi: 10.1007/978-1-4615-6331-0 – ident: ref17 doi: 10.1109/ACCESS.2017.2714191 – ident: ref6 doi: 10.1109/JSAC.2016.2545413 – ident: ref5 doi: 10.3390/s16070974 – ident: ref1 doi: 10.3390/jsan5040017
SSID	ssj0001105196
Score	2.5945418
Snippet	In this article, we propose a cooperative edge caching scheme, a new paradigm to jointly optimize the content placement and content delivery in the vehicular...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	247
SubjectTerms	Base stations Caching Computational modeling Computer simulation Content delivery content placement Cooperative caching cooperative edge caching deep deterministic policy gradient (DDPG) double time-scale Markov decision process (DTS-MDP) Edge computing Indexes Integer programming Intelligent vehicles Internet of Things Linear programming Machine learning Markov processes Mixed integer Optimization Placement Roadsides vehicular edge computing and networks
Title	Deep Reinforcement Learning for Cooperative Content Caching in Vehicular Edge Computing and Networks
URI	https://ieeexplore.ieee.org/document/8879573 https://www.proquest.com/docview/2338661391
Volume	7
WOSCitedRecordID	wos000508181000019&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 2327-4662 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001105196 issn: 2327-4662 databaseCode: RIE dateStart: 20140101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NS8NAEB20ePDit1itsgdPYjTJptnsUaqiHqqIireQ7Ey0IGlpq7_f2c22IorgLSSzJOTtx7zZ2TcAh6Uy7IWnZYAJxkHSxYznQe7LmMnKFJo5iWmKTah-P3t-1ncLcDw_C0NELvmMTuyl28vHoXm3obLTzFbGVnIRFpVSzVmtr3hKZJ2R1G9cRqE-vbm-fbC5W_ok1lY0Jfy29LhaKj8mYLeqXK7-73vWYMV7j-KsgXsdFqjegNVZZQbhB-om4DnRSNyTk0U1LgIovJLqi-BbojccjqgR_RZOoIoNek1epRjU4oleBy5BVVzgi7WwL7DPihpFv8kcn2zB4-XFQ-8q8PUUAiO7ehrIVJOmNMw0RlhJibx8dSlkCoOKIUFJWhkTGVOxW4QVSYwqKtkiidFIbrANrXpY0w6IJJSE7PtQIctEVUVmuikzt7hMmM4VYdKGcParc-PFxm3Ni7fckY5Q5xad3KKTe3TacDRvMmqUNv4y3rRwzA09Em3ozPDM_Vic5DGzcPZCpI52f2-1B8uxZdEusNKB1nT8TvuwZD6mg8n4wHWzT0fe0dM
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT9tAEB5BigQX3lVDge6hJ4TB9q4fe6xSELSQIhQQN8veGdNIlRORhN_f2fUmCIEq9WbZs7Llbx_zzc5-A_C1ygx74WkVoMI4UAnmPA9yX8Zc1qbUzElMW2wi6_fzhwd9swTHi7MwROSSz-jEXrq9fByZmQ2Vnea2MnYml-FDolQctae1XiIqkXVHUr91GYX69Mflr4HN3tInsbayKeGrxcdVU3kzBbt15Xzj_75oE9a9_yi-tYBvwRI127Axr80g_FDdAfxONBa35IRRjYsBCq-l-ij4luiNRmNqZb-Fk6hig16bWSmGjbin30OXoirO8NFa2BfYZ2WDot_mjk924e78bNC7CHxFhcDIRE8DmWrSlIa5xghrKZEXsIRCJjGYMSgoSWfGRMbU7BhhTRKjmiq2UDEayQ0-QqcZNfQJhAolIXs_VMpKZXWZmyRl7hZXigldGaouhPNfXRgvN26rXvwpHO0IdWHRKSw6hUenC0eLJuNWa-NfxjsWjoWhR6IL-3M8Cz8aJ0XMPJz9EKmjvfdbfYHVi8H1VXF12f_5GdZiy6ldmGUfOtOnGR3AinmeDidPh67L_QVYRNUa
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Deep+Reinforcement+Learning+for+Cooperative+Content+Caching+in+Vehicular+Edge+Computing+and+Networks&rft.jtitle=IEEE+internet+of+things+journal&rft.au=Qiao%2C+Guanhua&rft.au=Leng%2C+Supeng&rft.au=Maharjan%2C+Sabita&rft.au=Zhang%2C+Yan&rft.date=2020-01-01&rft.issn=2327-4662&rft.eissn=2327-4662&rft.volume=7&rft.issue=1&rft.spage=247&rft.epage=257&rft_id=info:doi/10.1109%2FJIOT.2019.2945640&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_JIOT_2019_2945640
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2327-4662&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2327-4662&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2327-4662&client=summon