On discounted approximations of undiscounted stochastic games and Markov decision processes with limited randomness
It is shown that the discount factor needed to solve an undiscounted mean payoff stochastic game to optimality is exponentially close to 1, even in one-player games with a single random node and polynomially bounded rewards and transition probabilities. For the class of the so-called irreducible gam...
Uloženo v:
| Vydáno v: | Operations research letters Ročník 41; číslo 4; s. 357 - 362 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Elsevier B.V
01.07.2013
|
| Témata: | |
| ISSN: | 0167-6377, 1872-7468 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | It is shown that the discount factor needed to solve an undiscounted mean payoff stochastic game to optimality is exponentially close to 1, even in one-player games with a single random node and polynomially bounded rewards and transition probabilities. For the class of the so-called irreducible games with perfect information and a constant number of random nodes, we obtain a pseudo-polynomial algorithm using discounts. |
|---|---|
| AbstractList | It is shown that the discount factor needed to solve an undiscounted mean payoff stochastic game to optimality is exponentially close to 1, even in one-player games with a single random node and polynomially bounded rewards and transition probabilities. For the class of the so-called irreducible games with perfect information and a constant number of random nodes, we obtain a pseudo-polynomial algorithm using discounts. |
| Author | Makino, Kazuhisa Gurvich, Vladimir Boros, Endre Elbassioni, Khaled |
| Author_xml | – sequence: 1 givenname: Endre surname: Boros fullname: Boros, Endre email: boros@rutcor.rutgers.edu organization: RUTCOR, Rutgers University, 640 Bartholomew Road, Piscataway NJ 08854-8003, United States – sequence: 2 givenname: Khaled surname: Elbassioni fullname: Elbassioni, Khaled email: elbassio@mpi-inf.mpg.de, kelbassioni@masdar.ac.ae organization: Masdar Institute of Science and Technology, Abu Dhabi, United Arab Emirates – sequence: 3 givenname: Vladimir surname: Gurvich fullname: Gurvich, Vladimir email: gurvich@rutcor.rutgers.edu organization: RUTCOR, Rutgers University, 640 Bartholomew Road, Piscataway NJ 08854-8003, United States – sequence: 4 givenname: Kazuhisa surname: Makino fullname: Makino, Kazuhisa email: makino@kurims.kyoto-u.ac.jp organization: Research Institute for Mathematical Sciences (RIMS), Kyoto University, Kyoto 606-8502, Japan |
| BookMark | eNp9kE1PAjEQhhuDiYD-AG_9A7tO2WW7G0-G-JVguOi5Ke1UiktL2oL67y3qwXjgNId3njczz4gMnHdIyCWDkgFrrtalD305AVaVUJcAzQkZspZPCl437YAM8w4vmorzMzKKcQ0AvGXtkMSFo9pG5XcuoaZyuw3-w25kst5F6g3duT9xTF6tZExW0Ve5wUil0_RJhje_pxqVjZmiuUFhjDl9t2lFe7uxBzbkXb9xOTknp0b2ES9-55i83N0-zx6K-eL-cXYzL9Sk46loWAdTaPVyamRtOg2M56uNRETDtKwMNkslNVeSSaiNaauWdV0HinXLqWx0NSbsp1cFH2NAI7YhvxY-BQNxsCbWIlsTB2sCapGtZYb_Y5RN3zZSkLY_Sl7_kJhf2lsMIiqLTqG2AVUS2tsj9Be3w44X |
| CitedBy_id | crossref_primary_10_1016_j_ic_2019_03_005 crossref_primary_10_1007_s13235_016_0199_x |
| Cites_doi | 10.1287/moor.24.4.817 10.1016/0304-3975(95)00188-3 10.1287/mnsc.12.5.359 10.1016/0041-5553(88)90012-2 10.1007/s13235-013-0075-x 10.1137/1011093 10.1007/BF01768705 10.1016/0012-365X(78)90011-0 10.1016/0890-5401(92)90048-K 10.1016/0022-247X(76)90178-5 10.1007/s00453-007-0175-3 10.1007/978-3-642-13036-6_26 |
| ContentType | Journal Article |
| Copyright | 2013 Elsevier B.V. |
| Copyright_xml | – notice: 2013 Elsevier B.V. |
| DBID | AAYXX CITATION |
| DOI | 10.1016/j.orl.2013.04.006 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Sciences (General) |
| EISSN | 1872-7468 |
| EndPage | 362 |
| ExternalDocumentID | 10_1016_j_orl_2013_04_006 S0167637713000515 |
| GroupedDBID | --K --M -~X .DC .~1 0R~ 123 1B1 1OL 1RT 1~. 1~5 29N 4.4 457 4G. 4R4 5VS 7-5 71M 8P~ 9JN 9JO AAAKF AAAKG AACTN AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AARIN AAXUO ABAOU ABFNM ABJNI ABMAC ABUCO ABXDB ABYKQ ACAZW ACDAQ ACGFS ACNCT ACRLP ADBBV ADEZE ADGUI ADIYS ADMUD AEBSH AEKER AENEX AFFNX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AIEXJ AIGVJ AIKHN AITUG AJBFU AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ APLSM ARUGR ASPBG AVWKF AXJTR AZFZN BKOJK BLXMC CS3 DU5 EBS EFJIC EFLBG EJD EO8 EO9 EP2 EP3 FDB FEDTE FGOYB FIRID FNPLU FYGXN G-Q GBLVA HAMUX HVGLF HZ~ IHE J1W KOM LY1 M41 MHUIS MO0 MS~ N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 R2- RIG ROL RPZ SDF SDG SDS SES SEW SPC SPCBC SSB SSD SSW SSZ T5K TN5 WH7 WUQ XPP XSW ~G- 9DU AATTM AAXKI AAYWO AAYXX ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP CITATION EFKBS ~HD |
| ID | FETCH-LOGICAL-c297t-6190508db5fa4f9d017007faeeef1da3fe6bcad7ca1a04ff83819990c19b5a6d3 |
| ISICitedReferencesCount | 1 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000321318300009&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0167-6377 |
| IngestDate | Sat Nov 29 07:26:30 EST 2025 Tue Nov 18 21:01:57 EST 2025 Fri Feb 23 02:30:14 EST 2024 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 4 |
| Keywords | Discounted stochastic games Zero-sum stochastic games Markov decision processes Pseudo-polynomial algorithms Saddle point |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c297t-6190508db5fa4f9d017007faeeef1da3fe6bcad7ca1a04ff83819990c19b5a6d3 |
| PageCount | 6 |
| ParticipantIDs | crossref_primary_10_1016_j_orl_2013_04_006 crossref_citationtrail_10_1016_j_orl_2013_04_006 elsevier_sciencedirect_doi_10_1016_j_orl_2013_04_006 |
| PublicationCentury | 2000 |
| PublicationDate | 2013-07-01 |
| PublicationDateYYYYMMDD | 2013-07-01 |
| PublicationDate_xml | – month: 07 year: 2013 text: 2013-07-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationTitle | Operations research letters |
| PublicationYear | 2013 |
| Publisher | Elsevier B.V |
| Publisher_xml | – name: Elsevier B.V |
| References | Condon (br000020) 1992; 96 Liggett, Lippman (br000065) 1969; 4 Moulin (br000085) 1976; 45 Gillette (br000035) 1957; vol. 39 Mine, Osaki (br000075) 1970 Gimbert, Horn (br000040) 2008; vol. 4962 Andersson, Miltersen (br000005) 2009; vol. 5878 Hoffman, Karp (br000055) 1966; 12 Miltersen (br000070) 2011 Zwick, Paterson (br000095) 1996; 158 E. Boros, K. Elbassioni, V. Gurvich, K. Makino, A pumping algorithm for ergodic stochastic mean payoff games with perfect information, in: Proc. 14th IPCO, 2010, pp. 341–354. Karp (br000060) 1978; 23 Gurvich, Karzanov, Khachiyan (br000045) 1988; 28 Condon (br000025) 1993; vol. 13 Eherenfeucht, Mycielski (br000030) 1979; 8 Pisaruk (br000090) 1999; 24 Boros, Elbassioni, Gurvich, Makino (br000015) 2013 Halman (br000050) 2007; 49 Moulin (br000080) 1976; 5 Halman (10.1016/j.orl.2013.04.006_br000050) 2007; 49 Moulin (10.1016/j.orl.2013.04.006_br000080) 1976; 5 Eherenfeucht (10.1016/j.orl.2013.04.006_br000030) 1979; 8 Mine (10.1016/j.orl.2013.04.006_br000075) 1970 Moulin (10.1016/j.orl.2013.04.006_br000085) 1976; 45 Condon (10.1016/j.orl.2013.04.006_br000020) 1992; 96 Boros (10.1016/j.orl.2013.04.006_br000015) 2013 Gimbert (10.1016/j.orl.2013.04.006_br000040) 2008; vol. 4962 Miltersen (10.1016/j.orl.2013.04.006_br000070) 2011 Gurvich (10.1016/j.orl.2013.04.006_br000045) 1988; 28 Liggett (10.1016/j.orl.2013.04.006_br000065) 1969; 4 Karp (10.1016/j.orl.2013.04.006_br000060) 1978; 23 Pisaruk (10.1016/j.orl.2013.04.006_br000090) 1999; 24 10.1016/j.orl.2013.04.006_br000010 Andersson (10.1016/j.orl.2013.04.006_br000005) 2009; vol. 5878 Zwick (10.1016/j.orl.2013.04.006_br000095) 1996; 158 Condon (10.1016/j.orl.2013.04.006_br000025) 1993; vol. 13 Gillette (10.1016/j.orl.2013.04.006_br000035) 1957; vol. 39 Hoffman (10.1016/j.orl.2013.04.006_br000055) 1966; 12 |
| References_xml | – year: 2013 ident: br000015 article-title: On canonical forms for zero-sum stochastic mean payoff games publication-title: Dynamic Games and Applications – volume: vol. 13 year: 1993 ident: br000025 article-title: An algorithm for simple stochastic games publication-title: Advances in Computational Complexity Theory – volume: 28 start-page: 85 year: 1988 end-page: 91 ident: br000045 article-title: Cyclic games and an algorithm to find minimax cycle means in directed graphs publication-title: The USSR Computational Mathematics and Mathematical Physics – volume: vol. 5878 start-page: 112 year: 2009 end-page: 121 ident: br000005 article-title: The complexity of solving stochastic games on graphs publication-title: Proc. 20th ISAAC – volume: 4 start-page: 604 year: 1969 end-page: 607 ident: br000065 article-title: Stochastic games with perfect information and time-average payoff publication-title: SIAM Review – volume: 45 year: 1976 ident: br000085 article-title: Prolongement des jeux à deux joueurs de somme nulle, Bulletin de la Société Mathématique de France publication-title: Memoire – volume: 24 start-page: 817 year: 1999 end-page: 828 ident: br000090 article-title: Mean cost cyclical games publication-title: Mathematics of Operations Research – volume: 96 start-page: 203 year: 1992 end-page: 224 ident: br000020 article-title: Complexity of stochastic games publication-title: Information and Computation – volume: vol. 39 start-page: 179 year: 1957 end-page: 187 ident: br000035 article-title: Stochastic games with zero stop probabilities publication-title: Contribution to the Theory of Games III – volume: 8 start-page: 109 year: 1979 end-page: 113 ident: br000030 article-title: Positional strategies for mean payoff games publication-title: International Journal of Game Theory – volume: 49 start-page: 37 year: 2007 end-page: 50 ident: br000050 article-title: Simple stochastic games, parity games, mean payoff games and discounted payoff games are all LP-type problems publication-title: Algorithmica – year: 1970 ident: br000075 article-title: Markovian Decision Process – volume: 158 start-page: 343 year: 1996 end-page: 359 ident: br000095 article-title: The complexity of mean payoff games on graphs publication-title: Theoretical Computer Science – reference: E. Boros, K. Elbassioni, V. Gurvich, K. Makino, A pumping algorithm for ergodic stochastic mean payoff games with perfect information, in: Proc. 14th IPCO, 2010, pp. 341–354. – volume: 23 start-page: 309 year: 1978 end-page: 311 ident: br000060 article-title: A characterization of the minimum cycle mean in a digraph publication-title: Discrete Mathematics – volume: vol. 4962 start-page: 5 year: 2008 end-page: 19 ident: br000040 article-title: Simple stochastic games with few random vertices are easy to solve publication-title: Proc. 11th FoSSaCS – volume: 5 start-page: 490 year: 1976 end-page: 507 ident: br000080 article-title: Extension of two person zero sum games publication-title: Journal of Mathematical Analysis and Applications – year: 2011 ident: br000070 article-title: Discounted stochastic games poorly approximate undiscounted ones, Manuscript, Tech. Rep. – volume: 12 start-page: 359 year: 1966 end-page: 370 ident: br000055 article-title: On non-terminating stochastic games publication-title: Management Science – volume: 45 year: 1976 ident: 10.1016/j.orl.2013.04.006_br000085 article-title: Prolongement des jeux à deux joueurs de somme nulle, Bulletin de la Société Mathématique de France publication-title: Memoire – volume: 24 start-page: 817 issue: 4 year: 1999 ident: 10.1016/j.orl.2013.04.006_br000090 article-title: Mean cost cyclical games publication-title: Mathematics of Operations Research doi: 10.1287/moor.24.4.817 – volume: 158 start-page: 343 issue: 1–2 year: 1996 ident: 10.1016/j.orl.2013.04.006_br000095 article-title: The complexity of mean payoff games on graphs publication-title: Theoretical Computer Science doi: 10.1016/0304-3975(95)00188-3 – volume: 12 start-page: 359 year: 1966 ident: 10.1016/j.orl.2013.04.006_br000055 article-title: On non-terminating stochastic games publication-title: Management Science doi: 10.1287/mnsc.12.5.359 – volume: 28 start-page: 85 year: 1988 ident: 10.1016/j.orl.2013.04.006_br000045 article-title: Cyclic games and an algorithm to find minimax cycle means in directed graphs publication-title: The USSR Computational Mathematics and Mathematical Physics doi: 10.1016/0041-5553(88)90012-2 – volume: vol. 39 start-page: 179 year: 1957 ident: 10.1016/j.orl.2013.04.006_br000035 article-title: Stochastic games with zero stop probabilities – year: 2013 ident: 10.1016/j.orl.2013.04.006_br000015 article-title: On canonical forms for zero-sum stochastic mean payoff games publication-title: Dynamic Games and Applications doi: 10.1007/s13235-013-0075-x – volume: vol. 13 year: 1993 ident: 10.1016/j.orl.2013.04.006_br000025 article-title: An algorithm for simple stochastic games – year: 2011 ident: 10.1016/j.orl.2013.04.006_br000070 – volume: 4 start-page: 604 year: 1969 ident: 10.1016/j.orl.2013.04.006_br000065 article-title: Stochastic games with perfect information and time-average payoff publication-title: SIAM Review doi: 10.1137/1011093 – year: 1970 ident: 10.1016/j.orl.2013.04.006_br000075 – volume: 8 start-page: 109 year: 1979 ident: 10.1016/j.orl.2013.04.006_br000030 article-title: Positional strategies for mean payoff games publication-title: International Journal of Game Theory doi: 10.1007/BF01768705 – volume: 23 start-page: 309 year: 1978 ident: 10.1016/j.orl.2013.04.006_br000060 article-title: A characterization of the minimum cycle mean in a digraph publication-title: Discrete Mathematics doi: 10.1016/0012-365X(78)90011-0 – volume: 96 start-page: 203 year: 1992 ident: 10.1016/j.orl.2013.04.006_br000020 article-title: Complexity of stochastic games publication-title: Information and Computation doi: 10.1016/0890-5401(92)90048-K – volume: 5 start-page: 490 issue: 2 year: 1976 ident: 10.1016/j.orl.2013.04.006_br000080 article-title: Extension of two person zero sum games publication-title: Journal of Mathematical Analysis and Applications doi: 10.1016/0022-247X(76)90178-5 – volume: 49 start-page: 37 issue: 1 year: 2007 ident: 10.1016/j.orl.2013.04.006_br000050 article-title: Simple stochastic games, parity games, mean payoff games and discounted payoff games are all LP-type problems publication-title: Algorithmica doi: 10.1007/s00453-007-0175-3 – volume: vol. 5878 start-page: 112 year: 2009 ident: 10.1016/j.orl.2013.04.006_br000005 article-title: The complexity of solving stochastic games on graphs – ident: 10.1016/j.orl.2013.04.006_br000010 doi: 10.1007/978-3-642-13036-6_26 – volume: vol. 4962 start-page: 5 year: 2008 ident: 10.1016/j.orl.2013.04.006_br000040 article-title: Simple stochastic games with few random vertices are easy to solve |
| SSID | ssj0007818 |
| Score | 1.9980191 |
| Snippet | It is shown that the discount factor needed to solve an undiscounted mean payoff stochastic game to optimality is exponentially close to 1, even in one-player... |
| SourceID | crossref elsevier |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 357 |
| SubjectTerms | Discounted stochastic games Markov decision processes Pseudo-polynomial algorithms Saddle point Zero-sum stochastic games |
| Title | On discounted approximations of undiscounted stochastic games and Markov decision processes with limited randomness |
| URI | https://dx.doi.org/10.1016/j.orl.2013.04.006 |
| Volume | 41 |
| WOSCitedRecordID | wos000321318300009&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1872-7468 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0007818 issn: 0167-6377 databaseCode: AIEXJ dateStart: 19950201 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Lj9MwELaqXQ5wQOwCYnnJBw5AFZSHU8fHFSoCBFsOC-otmsQ27W43qdKHKv4V_5Bx7DxYdhF74BJVaW2nna-e8fjzN4S8iGIdyzDMPJYL7bEIwEs0Z54OIQsC7UNsdWY_8ZOTZDoVXwaDn81ZmO2CF0Wy24nlfzU13kNjm6OzNzB32ynewNdodLyi2fH6T4afFGbXpa4BgcFkrRm-m190jLdN0XsbQ798Bkarefjd8GUd76I6L7dD6crvDJf2MIFyJ-EW9kzUEL2cLC-KhsLhItzJUlVuMCckhC3qM0NdQr6sLLlvbNiUHXsEPaoZz_ILZui5ZMsO2pgZrc4AfVuAxAdoOcWfTTmt0hJDfmxm8xX0ExmmqARvEhkut4lz9ihyVV3c5MyCHghZb6aNrK61c9qRndL_8Ac2NXH2pqzMNlMQ1bK2_hXa25d8YstUbEhwZyl2kZouUp-ltcj7fshjgb5g__jDePqxdf88qZPK7bdpttJrUuGl57g6GOoFOKf3yF23MqHHFlEHZKCKQ3Knp1d5SA6cJ1jRl06u_NV9spoUtEMU_R1wtNS0DzjaAY7WgKMIImoBRxvA0RZw1ACOOsDRDnAPyNd349O37z1XysPLQ8HXHi7TfVwKyCzWwLSQRrXJ5xqUUjqQEGk1ynKQPIcAfKZ1YhIJGCjlgchiGMnoIdkrykI9IjT3Q6kgywLBcHXvCwCtBcQxhyQG7idHxG9-0zR3Ovem3MoivdaWR-R122RpRV7-9mHWGCp1UaqNPlME3fXNHt9kjCfkdvcPeUr21tVGPSO38u16vqqeO8T9AghJu-M |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=On+discounted+approximations+of+undiscounted+stochastic+games+and+Markov+decision+processes+with+limited+randomness&rft.jtitle=Operations+research+letters&rft.au=Boros%2C+Endre&rft.au=Elbassioni%2C+Khaled&rft.au=Gurvich%2C+Vladimir&rft.au=Makino%2C+Kazuhisa&rft.date=2013-07-01&rft.issn=0167-6377&rft.volume=41&rft.issue=4&rft.spage=357&rft.epage=362&rft_id=info:doi/10.1016%2Fj.orl.2013.04.006&rft.externalDBID=n%2Fa&rft.externalDocID=10_1016_j_orl_2013_04_006 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0167-6377&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0167-6377&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0167-6377&client=summon |