Efficient approximate dynamic programming based on design and analysis of computer experiments for infinite-horizon optimization
•A sequential sampling algorithm for the infinite-horizon approximate dynamic programming is proposed.•A new stopping criterion to effectively identify an optimally equivalent value function is given.•The extrapolation issue of approximate value function built by MARS is explored and discussed. The...
Saved in:
| Published in: | Computers & operations research Vol. 124; p. 105032 |
|---|---|
| Main Authors: | , , , , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
New York
Elsevier Ltd
01.12.2020
Pergamon Press Inc |
| Subjects: | |
| ISSN: | 0305-0548, 0305-0548 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | •A sequential sampling algorithm for the infinite-horizon approximate dynamic programming is proposed.•A new stopping criterion to effectively identify an optimally equivalent value function is given.•The extrapolation issue of approximate value function built by MARS is explored and discussed.
The approximate dynamic programming (ADP) method based on the design and analysis of computer experiments (DACE) approach has been demonstrated as an effective method to solve multistage decision-making problems in the literature. However, this method is still not efficient for infinite-horizon optimization considering the required large volume of sampling in the state space and high-quality value function identification. Therefore, we propose a sequential sampling algorithm and embed it into a DACE-based ADP method to obtain a high-quality value function approximation. Considering the limitations of the traditional stopping criterion (Bellman error bound), we further propose a 45-degree line stopping criterion to terminate value iteration early by identifying an optimally equivalent value function. A comparison of the computational results with those of other three existing policies indicates that the proposed sampling algorithm and stopping criterion can determine a high-quality ADP policy. Finally, we discuss the extrapolation issue of the value function approximated by multivariate adaptive regression splines, the results of which further demonstrate the quality of the ADP policy generated in this study. |
|---|---|
| AbstractList | The approximate dynamic programming (ADP) method based on the design and analysis of computer experiments (DACE) approach has been demonstrated as an effective method to solve multistage decision-making problems in the literature. However, this method is still not efficient for infinite-horizon optimization considering the required large volume of sampling in the state space and high-quality value function identification. Therefore, we propose a sequential sampling algorithm and embed it into a DACE-based ADP method to obtain a high-quality value function approximation. Considering the limitations of the traditional stopping criterion (Bellman error bound), we further propose a 45-degree line stopping criterion to terminate value iteration early by identifying an optimally equivalent value function. A comparison of the computational results with those of other three existing policies indicates that the proposed sampling algorithm and stopping criterion can determine a high-quality ADP policy. Finally, we discuss the extrapolation issue of the value function approximated by multivariate adaptive regression splines, the results of which further demonstrate the quality of the ADP policy generated in this study. •A sequential sampling algorithm for the infinite-horizon approximate dynamic programming is proposed.•A new stopping criterion to effectively identify an optimally equivalent value function is given.•The extrapolation issue of approximate value function built by MARS is explored and discussed. The approximate dynamic programming (ADP) method based on the design and analysis of computer experiments (DACE) approach has been demonstrated as an effective method to solve multistage decision-making problems in the literature. However, this method is still not efficient for infinite-horizon optimization considering the required large volume of sampling in the state space and high-quality value function identification. Therefore, we propose a sequential sampling algorithm and embed it into a DACE-based ADP method to obtain a high-quality value function approximation. Considering the limitations of the traditional stopping criterion (Bellman error bound), we further propose a 45-degree line stopping criterion to terminate value iteration early by identifying an optimally equivalent value function. A comparison of the computational results with those of other three existing policies indicates that the proposed sampling algorithm and stopping criterion can determine a high-quality ADP policy. Finally, we discuss the extrapolation issue of the value function approximated by multivariate adaptive regression splines, the results of which further demonstrate the quality of the ADP policy generated in this study. |
| ArticleNumber | 105032 |
| Author | Kulvanitchaiyanunt, Asama Liu, Feng Rosenberger, Jay M. Zhou, Yuan Chen, Victoria C.P. Chen, Ying |
| Author_xml | – sequence: 1 givenname: Ying surname: Chen fullname: Chen, Ying email: yingchen@hit.edu.cn organization: Department of Management Science and Engineering, Harbin Institute of Technology, Harbin, Heilongjiang 150000, China – sequence: 2 givenname: Feng orcidid: 0000-0002-5225-8199 surname: Liu fullname: Liu, Feng email: fliu0@mgh.harvard.edu organization: School of Systems and Enterprises, Stevens Institute of Technology, Hoboken, NJ 07030, USA – sequence: 3 givenname: Jay M. orcidid: 0000-0003-4038-1402 surname: Rosenberger fullname: Rosenberger, Jay M. email: jrosenbe@uta.edu organization: Department of Industrial, Manufacturing & Systems Engineering, The University of Texas at Arlington, Arlington, TX 76019, USA – sequence: 4 givenname: Victoria C.P. surname: Chen fullname: Chen, Victoria C.P. email: vchen@uta.edu organization: Department of Industrial, Manufacturing & Systems Engineering, The University of Texas at Arlington, Arlington, TX 76019, USA – sequence: 5 givenname: Asama surname: Kulvanitchaiyanunt fullname: Kulvanitchaiyanunt, Asama email: asama.kulvanitchaiyanunt@mavs.uta.edu organization: Coraline Co., Ltd, Bangkok, Thailand – sequence: 6 givenname: Yuan surname: Zhou fullname: Zhou, Yuan email: yuan.zhou@uta.edu organization: Department of Industrial, Manufacturing & Systems Engineering, The University of Texas at Arlington, Arlington, TX 76019, USA |
| BookMark | eNp9kU1LRCEUhiUKqqkf0E5ofafj_Y5WEX1B0KbW4uhxOsNcvakTTat-ek7TIlokiEc974vv4yHbdd4hYycCpgJEe7aYah-mJZSbfQNVucMOoIKmgKbud3_V--wwxgXk0ZXigH1eW0ua0CWuxjH4dxpUQm7WTg2keT6ZBzUM5OZ8piIa7h03GGnuuHImT7VcR4rcW679MK4SBo7vIwYasmfk1gdOzpKjhMWLD_SRDfyYaKAPlci7I7Zn1TLi8c86Yc83109Xd8XD4-391eVDoaumS4VpEYStbA9YW9AzPWtLBW3foTBgDWrRV5jvOttqjeeqaZu67s_rWkBfQjerJux065sjva4wJrnwq5CfH2XZQJNhVJXIXWLbpYOPMaCVY06iwloKkBvQciEzaLkBLbegs6b7o9GUvrOloGj5r_Jiq8Qc_I0wyLj5Co2GAuokjad_1F-E0p0m |
| CitedBy_id | crossref_primary_10_1016_j_fmre_2021_09_011 crossref_primary_10_1007_s10479_025_06561_4 crossref_primary_10_1016_j_fmre_2021_06_010 crossref_primary_10_1109_TASE_2021_3119930 crossref_primary_10_1016_j_ejor_2023_01_049 crossref_primary_10_1016_j_ejco_2024_100089 crossref_primary_10_1007_s12555_021_0120_2 crossref_primary_10_1016_j_ress_2023_109594 |
| Cites_doi | 10.1029/2008WR007024 10.1016/j.ejor.2005.01.022 10.1080/07408170500232495 10.1287/opre.1080.0576 10.1007/s10589-007-9054-8 10.1287/opre.2013.1174 10.1016/j.ejor.2017.03.050 10.1007/s10589-013-9614-z 10.1007/s10287-015-0242-1 10.1080/07408179408966604 10.1145/2897824.2925881 10.1145/352958.352982 10.1016/j.cor.2005.02.043 10.1109/TNN.2004.824413 10.1111/j.1442-9993.1993.tb00438.x 10.1007/BF01386213 10.1029/93WR02277 10.1109/TCYB.2017.2660533 10.1287/opre.47.1.38 10.1109/ACC.2009.5160611 10.1287/msom.2017.0681 10.1145/1273496.1273589 10.1016/0041-5553(67)90144-9 10.1016/j.csda.2014.07.015 10.1016/j.cor.2012.11.016 10.1109/TIA.2017.2766230 10.1016/j.cor.2013.09.006 10.1109/TCYB.2015.2492242 |
| ContentType | Journal Article |
| Copyright | 2020 Elsevier Ltd Copyright Pergamon Press Inc. Dec 2020 |
| Copyright_xml | – notice: 2020 Elsevier Ltd – notice: Copyright Pergamon Press Inc. Dec 2020 |
| DBID | AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D |
| DOI | 10.1016/j.cor.2020.105032 |
| DatabaseName | CrossRef Computer and Information Systems Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Computer and Information Systems Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Advanced Technologies Database with Aerospace ProQuest Computer Science Collection Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Computer and Information Systems Abstracts |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Computer Science Business |
| EISSN | 0305-0548 |
| ExternalDocumentID | 10_1016_j_cor_2020_105032 S0305054820301490 |
| GroupedDBID | --K --M -~X .DC .~1 0R~ 186 1B1 1OL 1RT 1~. 1~5 29F 4.4 457 4G. 5GY 5VS 6J9 7-5 71M 8P~ 9JN 9JO AAAKF AAAKG AABNK AACTN AAEDT AAEDW AAFJI AAIKJ AAKOC AALRI AAOAW AAQXK AARIN AAXKI AAXUO AAYFN AAYOK ABAOU ABBOA ABDPE ABEFU ABFNM ABFRF ABJNI ABMAC ABMMH ABUCO ABXDB ACDAQ ACGFO ACGFS ACNCT ACNNM ACRLP ACZNC ADBBV ADEZE ADGUI ADJOM ADMUD AEBSH AEFWE AEHXG AEKER AENEX AFFNX AFJKZ AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHZHX AI. AIALX AIEXJ AIGVJ AIKHN AITUG AJOXV AKRWK ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOMHK AOUOD APLSM ARUGR ASPBG AVARZ AVWKF AXJTR AZFZN BKOJK BKOMP BLXMC CS3 DU5 EBS EFJIC EJD EO8 EO9 EP2 EP3 FDB FEDTE FGOYB FIRID FNPLU FYGXN G-Q GBLVA GBOLZ HAMUX HVGLF HZ~ H~9 IHE J1W KOM LY1 M41 MHUIS MO0 MS~ O-L O9- OAUVE OZT P-8 P-9 P2P PC. PQQKQ PRBVW Q38 R2- RIG ROL RPZ RXW SDF SDG SDP SDS SES SEW SPC SPCBC SSB SSD SSO SSV SSW SSZ T5K TAE TN5 U5U UAO UPT VH1 WUQ XPP ZMT ~02 ~G- 9DU AATTM AAYWO AAYXX ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO AEIPS AEUPX AFPUW AGQPQ AIGII AIIUN AKBMS AKYEP ANKPU APXCP CITATION EFKBS EFLBG ~HD 7SC 8FD AGCQF JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c357t-d6e01f3f80e4f0cbcb62a0687e1d0fdec183e0e47f6cce9a565448944108207b3 |
| ISICitedReferencesCount | 9 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000571468600013&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0305-0548 |
| IngestDate | Wed Aug 13 11:34:45 EDT 2025 Sat Nov 29 07:20:22 EST 2025 Tue Nov 18 22:04:17 EST 2025 Tue Dec 03 03:45:22 EST 2024 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Extrapolation Approximate dynamic programming State space sampling Stopping criterion |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c357t-d6e01f3f80e4f0cbcb62a0687e1d0fdec183e0e47f6cce9a565448944108207b3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0002-5225-8199 0000-0003-4038-1402 |
| PQID | 2505721331 |
| PQPubID | 45870 |
| ParticipantIDs | proquest_journals_2505721331 crossref_primary_10_1016_j_cor_2020_105032 crossref_citationtrail_10_1016_j_cor_2020_105032 elsevier_sciencedirect_doi_10_1016_j_cor_2020_105032 |
| PublicationCentury | 2000 |
| PublicationDate | 2020-12-01 |
| PublicationDateYYYYMMDD | 2020-12-01 |
| PublicationDate_xml | – month: 12 year: 2020 text: 2020-12-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | Computers & operations research |
| PublicationYear | 2020 |
| Publisher | Elsevier Ltd Pergamon Press Inc |
| Publisher_xml | – name: Elsevier Ltd – name: Pergamon Press Inc |
| References | Peng, Berseth, van de Panne (b0155) 2016; 35 Chen, Ruppert, Shoemaker (b0045) 1999; 47 Halton (b0095) 1960; 2 Sarikprueck, Lee, Kulvanitchaiyanunt, Chen, Rosenberger (b0175) 2017; 54 Fan, Tarun, Chen (b0065) 2013; 40 Cervellera, Muselli (b0025) 2007; 38 Kossmann, Stocker (b0110) 2000; 25 Birge, Louveaux (b0015) 1997 Farahmand, A. M., Ghavamzadeh, M., Szepesvari, C., Mannor, S. (2009) Regularized Fitted Q-Iteration for planning in continuous-space Markovian decision problems. 2009 American Control Conference, 10–12, June, St, Louis, MO, USA. Powell (b0160) 2011 Tilmant, Pinte, Goor (b0190) 2008; 44 Martinez, Shih, Chen, Kim (b0130) 2015; 81 Cervellara, Gaggero, Maccio (b0020) 2014; 43 Parr, R., Painter-Wakefield, C., Li, L., Littman, M., 2007. Analyzing feature generation for value-function approximation. In: Proceedings of the 24th International Conference on Machine Learning, Corvallis, OR. Sobol (b0180) 1967; 7 Zéphyr, Lang, Lamond, Cote (b0210) 2017; 262 Niederreiter (b0140) 1992 Papavasiliou, Shmuel (b0145) 2013; 61 Ernst, Geurts, Wehenkel (b0060) 2005; 6 Yang, Chen, Chang, Sattler, Wen (b0200) 2009; 57 Cervellera, Chen, Wen (b0030) 2006; 171 Cervellera, Maccio (b8000) 2017; 47 Cervellera, Muselli (b0035) 2004; 15 Zéphyr, Pascal, Lamond (b0205) 2015; 12 Bellman (b0005) 1957 Gnecco, Sanguineti (b0085) 2016; 1 Zoppoli, Sanguineti, Gnecco, Parisini (b0215) 2019 Munos, Szepesvari (b0135) 2008; 9 Friedman (b0075) 1991; 19 Bertsekas (b8010) 2017 Kutner, Nachtshim, Neter, Li (b0115) 2004 Heath, Jackson (b0100) 1994; 26 Bertsekas, Tsitsiklis (b0010) 1996 Chen, Tsui, Barton, Meckesheimer (b0050) 2006; 38 Clarke (b0055) 1993; 18 Gaggero, Gnecco, Sanguineti (b0080) 2014; 58 Tejada-Guibert, Johnson, Stedinger (b0185) 1993; 29 Hadley, Whitin (b0090) 1963 Lee, Lee (b0120) 2004; 2 Wei, Liu, Liu (b0195) 2016; 46 Cervellera, Wen, Chen (b0040) 2007; 34 Long, Nohdurft, Spinler (b0125) 2018; 20 Sacks, Welch, Mitchell, Wynn (b0165) 1989; 4 Niederreiter (10.1016/j.cor.2020.105032_b0140) 1992 Yang (10.1016/j.cor.2020.105032_b0200) 2009; 57 Sacks (10.1016/j.cor.2020.105032_b0165) 1989; 4 Tejada-Guibert (10.1016/j.cor.2020.105032_b0185) 1993; 29 Cervellera (10.1016/j.cor.2020.105032_b0035) 2004; 15 Papavasiliou (10.1016/j.cor.2020.105032_b0145) 2013; 61 Sobol (10.1016/j.cor.2020.105032_b0180) 1967; 7 Zéphyr (10.1016/j.cor.2020.105032_b0210) 2017; 262 Cervellera (10.1016/j.cor.2020.105032_b0040) 2007; 34 Munos (10.1016/j.cor.2020.105032_b0135) 2008; 9 Bellman (10.1016/j.cor.2020.105032_b0005) 1957 Ernst (10.1016/j.cor.2020.105032_b0060) 2005; 6 Tilmant (10.1016/j.cor.2020.105032_b0190) 2008; 44 Zoppoli (10.1016/j.cor.2020.105032_b0215) 2019 Wei (10.1016/j.cor.2020.105032_b0195) 2016; 46 Zéphyr (10.1016/j.cor.2020.105032_b0205) 2015; 12 Chen (10.1016/j.cor.2020.105032_b0050) 2006; 38 Bertsekas (10.1016/j.cor.2020.105032_b8010) 2017 10.1016/j.cor.2020.105032_b0070 Cervellara (10.1016/j.cor.2020.105032_b0020) 2014; 43 Chen (10.1016/j.cor.2020.105032_b0045) 1999; 47 Fan (10.1016/j.cor.2020.105032_b0065) 2013; 40 Powell (10.1016/j.cor.2020.105032_b0160) 2011 Peng (10.1016/j.cor.2020.105032_b0155) 2016; 35 Kossmann (10.1016/j.cor.2020.105032_b0110) 2000; 25 Hadley (10.1016/j.cor.2020.105032_b0090) 1963 Martinez (10.1016/j.cor.2020.105032_b0130) 2015; 81 10.1016/j.cor.2020.105032_b0150 Cervellera (10.1016/j.cor.2020.105032_b8000) 2017; 47 Gaggero (10.1016/j.cor.2020.105032_b0080) 2014; 58 Lee (10.1016/j.cor.2020.105032_b0120) 2004; 2 Sarikprueck (10.1016/j.cor.2020.105032_b0175) 2017; 54 Birge (10.1016/j.cor.2020.105032_b0015) 1997 Cervellera (10.1016/j.cor.2020.105032_b0030) 2006; 171 Heath (10.1016/j.cor.2020.105032_b0100) 1994; 26 Kutner (10.1016/j.cor.2020.105032_b0115) 2004 Cervellera (10.1016/j.cor.2020.105032_b0025) 2007; 38 Friedman (10.1016/j.cor.2020.105032_b0075) 1991; 19 Halton (10.1016/j.cor.2020.105032_b0095) 1960; 2 Bertsekas (10.1016/j.cor.2020.105032_b0010) 1996 Clarke (10.1016/j.cor.2020.105032_b0055) 1993; 18 Gnecco (10.1016/j.cor.2020.105032_b0085) 2016; 1 Long (10.1016/j.cor.2020.105032_b0125) 2018; 20 |
| References_xml | – volume: 58 start-page: 31 year: 2014 end-page: 85 ident: b0080 article-title: Approximate dynamic programming for stochastic N-stage optimization with application to optimal consumption under uncertainty publication-title: Comput. Optim. Appl. – volume: 1 start-page: 1 year: 2016 end-page: 16 ident: b0085 article-title: Neural approximations of the solutions to a class of stochastic optimal control problems publication-title: J. NeuroTechnol. – year: 1997 ident: b0015 article-title: Introduction to Stochastic Programming – volume: 171 start-page: 1139 year: 2006 end-page: 1151 ident: b0030 article-title: Optimization of a large-scale water reservoir network by stochastic dynamic programming with efficient state space discretization publication-title: Euro. J. Oper. Res. – volume: 46 start-page: 840 year: 2016 end-page: 853 ident: b0195 article-title: Value iteration adaptive dynamic programming for optimal control of discrete-time nonlinear systems publication-title: IEEE Trans. Cybern. – volume: 47 start-page: 3355 year: 2017 end-page: 3366 ident: b8000 article-title: A novel approach for sampling in approximate dynamic programming based on F-discrepancy publication-title: IEEE Transactions on Cybernetics – year: 2004 ident: b0115 article-title: Applied Linear Regression Models – volume: 12 start-page: 539 year: 2015 end-page: 557 ident: b0205 article-title: Controlled approximation of the value function in stochastic dynamic programming for multi-reservoir systems publication-title: CMS – volume: 43 start-page: 108 year: 2014 end-page: 115 ident: b0020 article-title: Low-discrepancy sampling for approximate dynamic programming with local approximators publication-title: Comput. Oper. Res. – volume: 26 start-page: 17 year: 1994 end-page: 30 ident: b0100 article-title: Modelling the evolution of demand forecasts with application to safety stock analysis in production/distribution systems publication-title: IIE Trans. – volume: 81 start-page: 89 year: 2015 end-page: 106 ident: b0130 article-title: A Convex version of multivariate adaptive regression splines publication-title: Comput. Stat. Data Anal. – volume: 57 start-page: 484 year: 2009 end-page: 498 ident: b0200 article-title: A decision-making framework for ozone pollution control publication-title: Oper. Res. – volume: 19 start-page: 1 year: 1991 end-page: 67 ident: b0075 article-title: Multivariate adaptive regression splines publication-title: Ann. Stat. – volume: 34 start-page: 70 year: 2007 end-page: 90 ident: b0040 article-title: Neural network and regression spline value function approximations for stochastic dynamic programming publication-title: Comp. Oper. Res. – volume: 2 start-page: 84 year: 1960 end-page: 90 ident: b0095 article-title: On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals publication-title: Numer. Math. – volume: 61 start-page: 578 year: 2013 end-page: 592 ident: b0145 article-title: Multiarea stochastic unit commitment for high wind penetration in a transmission constrained network publication-title: Oper. Res. – volume: 7 start-page: 784 year: 1967 end-page: 802 ident: b0180 article-title: The distribution of points in a cube and the approximate evaluation of integrals publication-title: USSR Comput. Math. Math. Phys. – year: 1957 ident: b0005 article-title: Dynamic Programming – volume: 15 start-page: 533 year: 2004 end-page: 543 ident: b0035 article-title: Deterministic design for neural network learning: an approach based on discrepancy publication-title: IEEE Trans. Neural Networks – volume: 40 start-page: 1076 year: 2013 end-page: 1084 ident: b0065 article-title: Adaptive value function approximation for continuous-state stochastic dynamic programming publication-title: Comput. Oper. Res. – volume: 25 start-page: 43 year: 2000 end-page: 82 ident: b0110 article-title: Iterative dynamic programming: a new class of query optimization algorithms publication-title: ACM Trans. Database Syst.. – volume: 29 start-page: 3969 year: 1993 end-page: 3980 ident: b0185 article-title: Comparison of two approaches for implementing multi-reservoir operating policies derived using dynamic programming publication-title: Water Resour. Res. – year: 1996 ident: b0010 article-title: Neuro-dynamic Programming – year: 2017 ident: b8010 article-title: Dynamic programming and optimal control – volume: 35 start-page: 81 year: 2016 ident: b0155 article-title: Terrain-adaptive locomotion skills using deep reinforcement learning publication-title: ACM Trans. Graphics (TOG) – volume: 38 start-page: 417 year: 2007 end-page: 443 ident: b0025 article-title: Efficient sampling in approximate dynamic programming algorithms publication-title: Comput. Optim. Appl. – reference: Farahmand, A. M., Ghavamzadeh, M., Szepesvari, C., Mannor, S. (2009) Regularized Fitted Q-Iteration for planning in continuous-space Markovian decision problems. 2009 American Control Conference, 10–12, June, St, Louis, MO, USA. – volume: 2 start-page: 263 year: 2004 end-page: 278 ident: b0120 article-title: Approximate dynamic programming strategies and their applicability for process control: a review and future directions publication-title: Int. J. Control Autom. Syst. – year: 2011 ident: b0160 article-title: Approximate Dynamic Programming: Solving the Curses of Dimensionality – volume: 18 start-page: 117 year: 1993 end-page: 143 ident: b0055 article-title: Non-parametric multivariate analyses of changes in community structure publication-title: Austral Ecol. – year: 2019 ident: b0215 article-title: Neural Approximations for Optimal Control and Decision, Communications and Control Engineering – volume: 6 start-page: 503 year: 2005 end-page: 556 ident: b0060 article-title: Tree-based batch mode reinforcement learning publication-title: J. Mach. Learn. Res. – reference: Parr, R., Painter-Wakefield, C., Li, L., Littman, M., 2007. Analyzing feature generation for value-function approximation. In: Proceedings of the 24th International Conference on Machine Learning, Corvallis, OR. – volume: 44 start-page: W12431 year: 2008 ident: b0190 article-title: Assessing marginal water values in multipurpose multireservoir systems via stochastic programming publication-title: Water Resour. Res. – volume: 262 start-page: 586 year: 2017 end-page: 601 ident: b0210 article-title: Approximate stochastic dynamic programming for hydroelectric production planning publication-title: Eur. J. Oper. Res. – volume: 54 start-page: 977 year: 2017 end-page: 986 ident: b0175 article-title: Bounds for optimal control of a regional plug-In electric vehicle charging station system publication-title: IEEE Trans. Ind. Appl. – volume: 20 start-page: 181 year: 2018 end-page: 198 ident: b0125 article-title: Spatial resource allocation for emerging epidemics: a comparison of greedy, myopic, and dynamic policies publication-title: Manuf. Serv. Oper. Manage. – year: 1963 ident: b0090 article-title: Analysis of Inventory Systems – volume: 47 start-page: 38 year: 1999 end-page: 53 ident: b0045 article-title: Applying experimental design and regression splines to high-dimensional continuous-state stochastic dynamic programming publication-title: Oper. Res. – volume: 9 start-page: 815 year: 2008 end-page: 857 ident: b0135 article-title: Finite-time bounds for fitted value iteration publication-title: J. Mach. Learn. Res. – year: 1992 ident: b0140 article-title: Random Number Generation and Quasi-Monte Carlo Methods – volume: 4 start-page: 409 year: 1989 end-page: 423 ident: b0165 article-title: Design and analysis of computer experiments publication-title: Stat. Sci. – volume: 38 start-page: 273 year: 2006 end-page: 291 ident: b0050 article-title: A review on design, modeling, and applications of computer experiments publication-title: IIE Trans. – volume: 44 start-page: W12431 year: 2008 ident: 10.1016/j.cor.2020.105032_b0190 article-title: Assessing marginal water values in multipurpose multireservoir systems via stochastic programming publication-title: Water Resour. Res. doi: 10.1029/2008WR007024 – year: 2011 ident: 10.1016/j.cor.2020.105032_b0160 – year: 1957 ident: 10.1016/j.cor.2020.105032_b0005 – volume: 171 start-page: 1139 year: 2006 ident: 10.1016/j.cor.2020.105032_b0030 article-title: Optimization of a large-scale water reservoir network by stochastic dynamic programming with efficient state space discretization publication-title: Euro. J. Oper. Res. doi: 10.1016/j.ejor.2005.01.022 – volume: 2 start-page: 263 issue: 3 year: 2004 ident: 10.1016/j.cor.2020.105032_b0120 article-title: Approximate dynamic programming strategies and their applicability for process control: a review and future directions publication-title: Int. J. Control Autom. Syst. – volume: 38 start-page: 273 year: 2006 ident: 10.1016/j.cor.2020.105032_b0050 article-title: A review on design, modeling, and applications of computer experiments publication-title: IIE Trans. doi: 10.1080/07408170500232495 – volume: 57 start-page: 484 year: 2009 ident: 10.1016/j.cor.2020.105032_b0200 article-title: A decision-making framework for ozone pollution control publication-title: Oper. Res. doi: 10.1287/opre.1080.0576 – volume: 38 start-page: 417 year: 2007 ident: 10.1016/j.cor.2020.105032_b0025 article-title: Efficient sampling in approximate dynamic programming algorithms publication-title: Comput. Optim. Appl. doi: 10.1007/s10589-007-9054-8 – volume: 61 start-page: 578 issue: 3 year: 2013 ident: 10.1016/j.cor.2020.105032_b0145 article-title: Multiarea stochastic unit commitment for high wind penetration in a transmission constrained network publication-title: Oper. Res. doi: 10.1287/opre.2013.1174 – volume: 262 start-page: 586 issue: 2 year: 2017 ident: 10.1016/j.cor.2020.105032_b0210 article-title: Approximate stochastic dynamic programming for hydroelectric production planning publication-title: Eur. J. Oper. Res. doi: 10.1016/j.ejor.2017.03.050 – volume: 58 start-page: 31 year: 2014 ident: 10.1016/j.cor.2020.105032_b0080 article-title: Approximate dynamic programming for stochastic N-stage optimization with application to optimal consumption under uncertainty publication-title: Comput. Optim. Appl. doi: 10.1007/s10589-013-9614-z – volume: 12 start-page: 539 issue: 4 year: 2015 ident: 10.1016/j.cor.2020.105032_b0205 article-title: Controlled approximation of the value function in stochastic dynamic programming for multi-reservoir systems publication-title: CMS doi: 10.1007/s10287-015-0242-1 – year: 2019 ident: 10.1016/j.cor.2020.105032_b0215 – volume: 26 start-page: 17 issue: 3 year: 1994 ident: 10.1016/j.cor.2020.105032_b0100 article-title: Modelling the evolution of demand forecasts with application to safety stock analysis in production/distribution systems publication-title: IIE Trans. doi: 10.1080/07408179408966604 – volume: 4 start-page: 409 issue: 4 year: 1989 ident: 10.1016/j.cor.2020.105032_b0165 article-title: Design and analysis of computer experiments publication-title: Stat. Sci. – volume: 35 start-page: 81 issue: 4 year: 2016 ident: 10.1016/j.cor.2020.105032_b0155 article-title: Terrain-adaptive locomotion skills using deep reinforcement learning publication-title: ACM Trans. Graphics (TOG) doi: 10.1145/2897824.2925881 – volume: 25 start-page: 43 issue: 1 year: 2000 ident: 10.1016/j.cor.2020.105032_b0110 article-title: Iterative dynamic programming: a new class of query optimization algorithms publication-title: ACM Trans. Database Syst.. doi: 10.1145/352958.352982 – year: 1992 ident: 10.1016/j.cor.2020.105032_b0140 – volume: 34 start-page: 70 year: 2007 ident: 10.1016/j.cor.2020.105032_b0040 article-title: Neural network and regression spline value function approximations for stochastic dynamic programming publication-title: Comp. Oper. Res. doi: 10.1016/j.cor.2005.02.043 – volume: 15 start-page: 533 year: 2004 ident: 10.1016/j.cor.2020.105032_b0035 article-title: Deterministic design for neural network learning: an approach based on discrepancy publication-title: IEEE Trans. Neural Networks doi: 10.1109/TNN.2004.824413 – volume: 18 start-page: 117 issue: 1 year: 1993 ident: 10.1016/j.cor.2020.105032_b0055 article-title: Non-parametric multivariate analyses of changes in community structure publication-title: Austral Ecol. doi: 10.1111/j.1442-9993.1993.tb00438.x – volume: 2 start-page: 84 year: 1960 ident: 10.1016/j.cor.2020.105032_b0095 article-title: On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals publication-title: Numer. Math. doi: 10.1007/BF01386213 – volume: 29 start-page: 3969 year: 1993 ident: 10.1016/j.cor.2020.105032_b0185 article-title: Comparison of two approaches for implementing multi-reservoir operating policies derived using dynamic programming publication-title: Water Resour. Res. doi: 10.1029/93WR02277 – volume: 47 start-page: 3355 issue: 10 year: 2017 ident: 10.1016/j.cor.2020.105032_b8000 article-title: A novel approach for sampling in approximate dynamic programming based on F-discrepancy publication-title: IEEE Transactions on Cybernetics doi: 10.1109/TCYB.2017.2660533 – volume: 47 start-page: 38 year: 1999 ident: 10.1016/j.cor.2020.105032_b0045 article-title: Applying experimental design and regression splines to high-dimensional continuous-state stochastic dynamic programming publication-title: Oper. Res. doi: 10.1287/opre.47.1.38 – ident: 10.1016/j.cor.2020.105032_b0070 doi: 10.1109/ACC.2009.5160611 – volume: 20 start-page: 181 issue: 2 year: 2018 ident: 10.1016/j.cor.2020.105032_b0125 article-title: Spatial resource allocation for emerging epidemics: a comparison of greedy, myopic, and dynamic policies publication-title: Manuf. Serv. Oper. Manage. doi: 10.1287/msom.2017.0681 – ident: 10.1016/j.cor.2020.105032_b0150 doi: 10.1145/1273496.1273589 – volume: 6 start-page: 503 year: 2005 ident: 10.1016/j.cor.2020.105032_b0060 article-title: Tree-based batch mode reinforcement learning publication-title: J. Mach. Learn. Res. – year: 1963 ident: 10.1016/j.cor.2020.105032_b0090 – volume: 7 start-page: 784 issue: 4 year: 1967 ident: 10.1016/j.cor.2020.105032_b0180 article-title: The distribution of points in a cube and the approximate evaluation of integrals publication-title: USSR Comput. Math. Math. Phys. doi: 10.1016/0041-5553(67)90144-9 – volume: 81 start-page: 89 year: 2015 ident: 10.1016/j.cor.2020.105032_b0130 article-title: A Convex version of multivariate adaptive regression splines publication-title: Comput. Stat. Data Anal. doi: 10.1016/j.csda.2014.07.015 – volume: 40 start-page: 1076 issue: 4 year: 2013 ident: 10.1016/j.cor.2020.105032_b0065 article-title: Adaptive value function approximation for continuous-state stochastic dynamic programming publication-title: Comput. Oper. Res. doi: 10.1016/j.cor.2012.11.016 – volume: 1 start-page: 1 year: 2016 ident: 10.1016/j.cor.2020.105032_b0085 article-title: Neural approximations of the solutions to a class of stochastic optimal control problems publication-title: J. NeuroTechnol. – volume: 54 start-page: 977 issue: 2 year: 2017 ident: 10.1016/j.cor.2020.105032_b0175 article-title: Bounds for optimal control of a regional plug-In electric vehicle charging station system publication-title: IEEE Trans. Ind. Appl. doi: 10.1109/TIA.2017.2766230 – volume: 43 start-page: 108 year: 2014 ident: 10.1016/j.cor.2020.105032_b0020 article-title: Low-discrepancy sampling for approximate dynamic programming with local approximators publication-title: Comput. Oper. Res. doi: 10.1016/j.cor.2013.09.006 – year: 2017 ident: 10.1016/j.cor.2020.105032_b8010 – year: 2004 ident: 10.1016/j.cor.2020.105032_b0115 – year: 1996 ident: 10.1016/j.cor.2020.105032_b0010 – volume: 19 start-page: 1 issue: 1 year: 1991 ident: 10.1016/j.cor.2020.105032_b0075 article-title: Multivariate adaptive regression splines publication-title: Ann. Stat. – volume: 46 start-page: 840 issue: 3 year: 2016 ident: 10.1016/j.cor.2020.105032_b0195 article-title: Value iteration adaptive dynamic programming for optimal control of discrete-time nonlinear systems publication-title: IEEE Trans. Cybern. doi: 10.1109/TCYB.2015.2492242 – volume: 9 start-page: 815 issue: May year: 2008 ident: 10.1016/j.cor.2020.105032_b0135 article-title: Finite-time bounds for fitted value iteration publication-title: J. Mach. Learn. Res. – year: 1997 ident: 10.1016/j.cor.2020.105032_b0015 |
| SSID | ssj0000721 |
| Score | 2.3921785 |
| Snippet | •A sequential sampling algorithm for the infinite-horizon approximate dynamic programming is proposed.•A new stopping criterion to effectively identify an... The approximate dynamic programming (ADP) method based on the design and analysis of computer experiments (DACE) approach has been demonstrated as an effective... |
| SourceID | proquest crossref elsevier |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 105032 |
| SubjectTerms | Algorithms Approximate dynamic programming Approximation Criteria Decision making Design analysis Dynamic programming Extrapolation Horizon Iterative methods Operations research Optimization Regression analysis Sequential sampling Spline functions State space sampling Stopping criterion |
| Title | Efficient approximate dynamic programming based on design and analysis of computer experiments for infinite-horizon optimization |
| URI | https://dx.doi.org/10.1016/j.cor.2020.105032 https://www.proquest.com/docview/2505721331 |
| Volume | 124 |
| WOSCitedRecordID | wos000571468600013&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 0305-0548 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000721 issn: 0305-0548 databaseCode: AIEXJ dateStart: 19950101 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV3LbtNAFB2FFiFY8CggCgXNgm6ILDke22Mvq5AIqhKySFFYjfy4o7pqnZCkVWDFr_Fn3HklblQiWLCxonFsRznH956ZuQ9C3gJkUcp830sDCR6-idzLmIxwlhJHOUsgCYNcN5vgg0EyHqfDVuuXy4W5vuB1nSyX6fS_Qo1jCLZKnf0HuFc3xQH8jKDjEWHH418B39NFIXTouKoXvqxQk0K7NJ3nXTzWpVohUB6sVLsFpQ7jsHVb10VKCtvxodEGQFdvUBFcldKq3tlkVv1QihMNz6XN6GzKXdczYq4ZNpnCzEbe2RpDq7Xork0S-eo8qYoRqq60tIb1kOoDqSPSDNGOs-_tT5u3-FKpfYgqa3eHzSWNYDM8ZIi3Ua2WTBTKzYBQZZ88lJnGZsMtY86qB2HDLndu9RZm4eIcwVaVYQPd9Ni3y603KnMPPov-6cmJGPXGo0PWn37zVNsytb1_yN4bCt0huwGPUvQNu0cfe-PjtRzgOvlv9SPd1roOMtx48p_E0YZM0Npn9Jg8tJMWemTI9oS0oN4j91zOxB555HCm1lXskQeNQpdPyc8VKWmDlNSSkjZISTUp6aSmhpQUSUkdKelEUkdK2iAlRVLSTVLSJimfkdN-b9T94NneH17BIr7wyhj8jmQy8SGUfpEXeRxkfpxw6JS-LKFAVwR4jsu4KCDNcF4ShkmK4l5pWp6z52SnntTwgtCAQyBjVsa8zEOeQ5bERQoS0EZlMgqyfeK7P10UtjC-6s9yIVwE5LlAnITCSRic9sm71SVTUxVm25dDh6SwstbIVYE83HbZgUNdWPMyF2rCgnxirPNy--lX5P76zTogO4vZFbwmd4vrRTWfvbEc_Q0GxNE3 |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Efficient+approximate+dynamic+programming+based+on+design+and+analysis+of+computer+experiments+for+infinite-horizon+optimization&rft.jtitle=Computers+%26+operations+research&rft.au=Chen%2C+Ying&rft.au=Liu%2C+Feng&rft.au=Rosenberger%2C+Jay+M&rft.au=Chen%2C+Victoria+CP&rft.date=2020-12-01&rft.pub=Pergamon+Press+Inc&rft.issn=0305-0548&rft.eissn=0305-0548&rft.volume=124&rft.spage=1&rft_id=info:doi/10.1016%2Fj.cor.2020.105032&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0305-0548&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0305-0548&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0305-0548&client=summon |