BEERL: Both Ends Explanations for Reinforcement Learning
Deep Reinforcement Learning (RL) is a black-box method and is hard to understand because the agent employs a neural network (NN). To explain the behavior and decisions made by the agent, different eXplainable RL (XRL) methods are developed; for example, feature importance methods are applied to anal...
Gespeichert in:
| Veröffentlicht in: | Applied sciences Jg. 12; H. 21; S. 10947 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Basel
MDPI AG
01.11.2022
|
| Schlagworte: | |
| ISSN: | 2076-3417, 2076-3417 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Deep Reinforcement Learning (RL) is a black-box method and is hard to understand because the agent employs a neural network (NN). To explain the behavior and decisions made by the agent, different eXplainable RL (XRL) methods are developed; for example, feature importance methods are applied to analyze the contribution of the input side of the model, and reward decomposition methods are applied to explain the components of the output end of the RL model. In this study, we present a novel method to connect explanations from both input and output ends of a black-box model, which results in fine-grained explanations. Our method exposes the reward prioritization to the user, which in turn generates two different levels of explanation and allows RL agent reconfigurations when unwanted behaviors are observed. The method further summarizes the detailed explanations into a focus value that takes into account all reward components and quantifies the fulfillment of the explanation of desired properties. We evaluated our method by applying it to a remote electrical telecom-antenna-tilt use case and two openAI gym environments: lunar lander and cartpole. The results demonstrated fine-grained explanations by detailing input features’ contributions to certain rewards and revealed biases of the reward components, which are then addressed by adjusting the reward’s weights. |
|---|---|
| AbstractList | Deep Reinforcement Learning (RL) is a black-box method and is hard to understand because the agent employs a neural network (NN). To explain the behavior and decisions made by the agent, different eXplainable RL (XRL) methods are developed; for example, feature importance methods are applied to analyze the contribution of the input side of the model, and reward decomposition methods are applied to explain the components of the output end of the RL model. In this study, we present a novel method to connect explanations from both input and output ends of a black-box model, which results in fine-grained explanations. Our method exposes the reward prioritization to the user, which in turn generates two different levels of explanation and allows RL agent reconfigurations when unwanted behaviors are observed. The method further summarizes the detailed explanations into a focus value that takes into account all reward components and quantifies the fulfillment of the explanation of desired properties. We evaluated our method by applying it to a remote electrical telecom-antenna-tilt use case and two openAI gym environments: lunar lander and cartpole. The results demonstrated fine-grained explanations by detailing input features’ contributions to certain rewards and revealed biases of the reward components, which are then addressed by adjusting the reward’s weights. |
| Author | Inam, Rafia Fersman, Elena Terra, Ahmad |
| Author_xml | – sequence: 1 givenname: Ahmad orcidid: 0000-0002-6650-2789 surname: Terra fullname: Terra, Ahmad – sequence: 2 givenname: Rafia orcidid: 0000-0001-7448-3381 surname: Inam fullname: Inam, Rafia – sequence: 3 givenname: Elena orcidid: 0000-0002-0182-8390 surname: Fersman fullname: Fersman, Elena |
| BackLink | https://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-329022$$DView record from Swedish Publication Index (Kungliga Tekniska Högskolan) |
| BookMark | eNptkVtrVDEURoNUsLZ98wcc8NWjuV986-VUCwOFor6GXNuM0-SYZFD_fY8zCq00L18Iay_2zn4NDnLJAYA3CL4nRMEPZp4RxghBRcULcIih4COhSBw8ur8CJ62t4XIUIhLBQyDPpulm9XE4K_1umLJvw_Rr3phseiq5DbHU4SakvKQL9yH3YRVMzSnfHoOX0WxaOPmbR-Dr5fTl_PO4uv50dX66Gh2FrI9USieZ4RS7qLBl3HMrOXMRBRKVt5EEToXE3lOOWbDSSBFl9EogEiz35Ahc7b2-mLWea7o39bcuJundQ6m32tSe3CZoRqVVDAsTbKQKewOhFQwTJzlGjonFNe5d7WeYt_aJ7SJ9O93Zvvc7TbCCGC_82z0_1_JjG1rX67KteRlXYyEQZwxBulB4T7laWqshapf67v96NWmjEdR_FqQfL2gpevdf0b9mnsUfAKpikQ4 |
| CitedBy_id | crossref_primary_10_1007_s10994_023_06479_7 crossref_primary_10_1007_s00521_024_10437_2 |
| Cites_doi | 10.18653/v1/N16-3020 10.1016/j.media.2015.06.008 10.1145/3387166 10.1109/ICCV.2017.74 10.1609/aaai.v32i1.11833 10.1016/j.inffus.2019.12.012 10.24963/ijcai.2019/184 10.1145/3623377 10.1007/978-3-031-04083-2 10.1109/WCNC49053.2021.9417363 10.5220/0010256208740881 10.1038/nature16961 10.1109/GLOBECOM42002.2020.9322496 10.1007/978-3-030-10928-8_25 10.1109/VTC2020-Fall49728.2020.9348456 10.1007/978-3-319-04717-1 10.1109/INFOCOM48880.2022.9796783 |
| ContentType | Journal Article |
| Copyright | 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| Copyright_xml | – notice: 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
| DBID | AAYXX CITATION ABUWG AFKRA AZQEC BENPR CCPQU DWQXO PHGZM PHGZT PIMPY PKEHL PQEST PQQKQ PQUKI ADTPV AFDQA AOWAS D8T D8V ZZAVC DOA |
| DOI | 10.3390/app122110947 |
| DatabaseName | CrossRef ProQuest Central (Alumni) ProQuest Central UK/Ireland ProQuest Central Essentials - QC ProQuest Central ProQuest One ProQuest Central ProQuest Central Premium ProQuest One Academic Publicly Available Content Database (ProQuest) ProQuest One Academic Middle East (New) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Academic (retired) ProQuest One Academic UKI Edition SwePub SWEPUB Kungliga Tekniska Högskolan full text SwePub Articles SWEPUB Freely available online SWEPUB Kungliga Tekniska Högskolan SwePub Articles full text DOAJ Directory of Open Access Journals |
| DatabaseTitle | CrossRef Publicly Available Content Database ProQuest Central ProQuest One Academic Middle East (New) ProQuest One Academic UKI Edition ProQuest Central Essentials ProQuest Central Korea ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) ProQuest One Community College ProQuest Central (New) ProQuest One Academic ProQuest One Academic (New) |
| DatabaseTitleList | CrossRef Publicly Available Content Database |
| Database_xml | – sequence: 1 dbid: DOA name: DOAJ Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website – sequence: 2 dbid: PIMPY name: Publicly Available Content Database url: http://search.proquest.com/publiccontent sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Sciences (General) |
| EISSN | 2076-3417 |
| ExternalDocumentID | oai_doaj_org_article_548b9527aebf492da00b7523c8621c57 oai_DiVA_org_kth_329022 10_3390_app122110947 |
| GroupedDBID | .4S 2XV 5VS 7XC 8CJ 8FE 8FG 8FH AADQD AAFWJ AAYXX ADBBV ADMLS AFFHD AFKRA AFPKN AFZYC ALMA_UNASSIGNED_HOLDINGS APEBS ARCSS BCNDV BENPR CCPQU CITATION CZ9 D1I D1J D1K GROUPED_DOAJ IAO IGS ITC K6- K6V KC. KQ8 L6V LK5 LK8 M7R MODMG M~E OK1 P62 PHGZM PHGZT PIMPY PROAC TUS ABUWG AZQEC DWQXO PKEHL PQEST PQQKQ PQUKI ADTPV AFDQA AOWAS D8T D8V IPNFZ RIG ZZAVC |
| ID | FETCH-LOGICAL-c405t-488c85a642cf92b56d6b865cf1e3f9dbf3e64782dd4625eb8a87f8fd9713eb6d3 |
| IEDL.DBID | PIMPY |
| ISICitedReferencesCount | 3 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000880918200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2076-3417 |
| IngestDate | Fri Oct 03 12:43:47 EDT 2025 Tue Nov 04 16:36:02 EST 2025 Mon Jun 30 11:23:33 EDT 2025 Sat Nov 29 07:21:45 EST 2025 Tue Nov 18 21:18:38 EST 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 21 |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c405t-488c85a642cf92b56d6b865cf1e3f9dbf3e64782dd4625eb8a87f8fd9713eb6d3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0002-0182-8390 0000-0001-7448-3381 0000-0002-6650-2789 |
| OpenAccessLink | https://www.proquest.com/publiccontent/docview/2771655104?pq-origsite=%requestingapplication% |
| PQID | 2771655104 |
| PQPubID | 2032433 |
| ParticipantIDs | doaj_primary_oai_doaj_org_article_548b9527aebf492da00b7523c8621c57 swepub_primary_oai_DiVA_org_kth_329022 proquest_journals_2771655104 crossref_citationtrail_10_3390_app122110947 crossref_primary_10_3390_app122110947 |
| PublicationCentury | 2000 |
| PublicationDate | 2022-11-01 |
| PublicationDateYYYYMMDD | 2022-11-01 |
| PublicationDate_xml | – month: 11 year: 2022 text: 2022-11-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | Basel |
| PublicationPlace_xml | – name: Basel |
| PublicationTitle | Applied sciences |
| PublicationYear | 2022 |
| Publisher | MDPI AG |
| Publisher_xml | – name: MDPI AG |
| References | ref_14 ref_35 ref_12 ref_34 ref_11 ref_33 ref_10 ref_32 ref_31 ref_30 ref_18 ref_17 ref_16 ref_38 ref_15 ref_37 Balcan (ref_13) 2016; Volume 48 ref_25 Silver (ref_2) 2016; 529 ref_24 ref_22 ref_21 ref_20 ref_1 Bennetot (ref_36) 2020; 58 ref_3 Larochelle (ref_19) 2020; Volume 33 Mohseni (ref_23) 2021; 11 ref_29 ref_28 ref_26 ref_8 ref_5 ref_4 Madumal (ref_27) 2020; 34 ref_7 Gaonkar (ref_9) 2015; 24 ref_6 |
| References_xml | – ident: ref_7 – ident: ref_28 – ident: ref_24 – ident: ref_34 – ident: ref_5 doi: 10.18653/v1/N16-3020 – volume: 24 start-page: 190 year: 2015 ident: ref_9 article-title: Interpreting support vector machine models for multivariate group wise analysis in neuroimaging publication-title: Med. Image Anal. doi: 10.1016/j.media.2015.06.008 – volume: 11 start-page: 24 year: 2021 ident: ref_23 article-title: A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems publication-title: ACM Trans. Interact. Intell. Syst. doi: 10.1145/3387166 – ident: ref_37 – ident: ref_14 – ident: ref_1 – ident: ref_11 doi: 10.1109/ICCV.2017.74 – ident: ref_35 – ident: ref_21 – ident: ref_25 doi: 10.1609/aaai.v32i1.11833 – volume: 58 start-page: 82 year: 2020 ident: ref_36 article-title: Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI publication-title: Inf. Fusion doi: 10.1016/j.inffus.2019.12.012 – ident: ref_26 doi: 10.24963/ijcai.2019/184 – volume: Volume 33 start-page: 11298 year: 2020 ident: ref_19 article-title: RD2: Reward Decomposition with Representation Decomposition publication-title: Proceedings of the Advances in Neural Information Processing Systems – ident: ref_18 doi: 10.1145/3623377 – ident: ref_3 doi: 10.1007/978-3-031-04083-2 – ident: ref_6 – ident: ref_8 – ident: ref_4 – ident: ref_30 doi: 10.1109/WCNC49053.2021.9417363 – ident: ref_17 doi: 10.5220/0010256208740881 – ident: ref_33 – volume: 529 start-page: 484 year: 2016 ident: ref_2 article-title: Mastering the game of Go with deep neural networks and tree search publication-title: Nature doi: 10.1038/nature16961 – ident: ref_12 – volume: Volume 48 start-page: 1899 year: 2016 ident: ref_13 article-title: Graying the black box: Understanding DQNs publication-title: Proceedings of the 33rd International Conference on Machine Learning – volume: 34 start-page: 2493 year: 2020 ident: ref_27 article-title: Explainable Reinforcement Learning through a Causal Lens publication-title: Proc. AAAI Conf. Artif. Intell. – ident: ref_16 doi: 10.1109/GLOBECOM42002.2020.9322496 – ident: ref_15 – ident: ref_29 doi: 10.1007/978-3-030-10928-8_25 – ident: ref_31 doi: 10.1109/VTC2020-Fall49728.2020.9348456 – ident: ref_38 – ident: ref_10 doi: 10.1007/978-3-319-04717-1 – ident: ref_22 – ident: ref_20 – ident: ref_32 doi: 10.1109/INFOCOM48880.2022.9796783 |
| SSID | ssj0000913810 |
| Score | 2.275952 |
| Snippet | Deep Reinforcement Learning (RL) is a black-box method and is hard to understand because the agent employs a neural network (NN). To explain the behavior and... |
| SourceID | doaj swepub proquest crossref |
| SourceType | Open Website Open Access Repository Aggregation Database Enrichment Source Index Database |
| StartPage | 10947 |
| SubjectTerms | Artificial intelligence Bias Decomposition deep reinforcement learning explainability explainable reinforcement learning Methods Neural networks Performance evaluation reward decomposition reward prioritization Variables |
| SummonAdditionalLinks | – databaseName: DOAJ Directory of Open Access Journals dbid: DOA link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1NS8NAEB2keNCD2KpYrbIHFUWCaT53vbXa4qmIqPQW9lOLEqWJ_n5nN2lJD-LFa1iyy5ts5r1k5w3AScioYZjXvNAY5UVGa49Tn3mKYjrXodEJp67ZRDqZ0OmU3TdafdkzYZU9cAXcFTJqweIg5VqYiAWK-75IUT1JpOJ9Gbs6cmQ9DTHl3sGsb62rqpPuIep6-z-4H1i1w2wnlUYOclb9q_yy6Rnq8sx4G7ZqgkgG1cLasKbzDmw2bAM70K43ZEHOa9foix2gwxES02syROjJKFcFscfrePWxryDITcmDdjap0n0RJLWz6ssuPI1Hjzd3Xt0WwZPIrkoPt5ykMUfhIA0LRJyoRNAklsZiy5QwobYFpIFSEYobLSinqaFGMdSjWiQq3INW_pHrfSCcRVzh3WQqMJulVGC25tr4RqsojFTShcsFUJmsPcNt64r3DLWDhTVrwtqF0-Xoz8or45dxQ4v5cox1uHYXMO5ZHffsr7h3obeIWFZvuyILUpR_yAH9qAtnVRRXZrmdPQ_cLG_laxYGDAnMwX8s5hA2AlsW4WoUe9Aq51_6CNbldzkr5sfu0fwBU_Dmaw priority: 102 providerName: Directory of Open Access Journals |
| Title | BEERL: Both Ends Explanations for Reinforcement Learning |
| URI | https://www.proquest.com/docview/2771655104 https://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-329022 https://doaj.org/article/548b9527aebf492da00b7523c8621c57 |
| Volume | 12 |
| WOSCitedRecordID | wos000880918200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: DOAJ Directory of Open Access Journals customDbUrl: eissn: 2076-3417 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000913810 issn: 2076-3417 databaseCode: DOA dateStart: 20110101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2076-3417 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000913810 issn: 2076-3417 databaseCode: M~E dateStart: 20110101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre – providerCode: PRVPQU databaseName: ProQuest Central customDbUrl: eissn: 2076-3417 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000913810 issn: 2076-3417 databaseCode: BENPR dateStart: 20110101 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest – providerCode: PRVPQU databaseName: Publicly Available Content Database customDbUrl: eissn: 2076-3417 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000913810 issn: 2076-3417 databaseCode: PIMPY dateStart: 20110101 isFulltext: true titleUrlDefault: http://search.proquest.com/publiccontent providerName: ProQuest |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LTxsxEB5B0gMceLWooTTyoa1aVats9mlzQYQGtYdGEWorelr5SREogWzg93fG64RwKKdeV5a92vF4vm_W8w3Au1RwJzCuRalzJsqctZHksYgMx3BuU2cLyX2ziXI04hcXYhzKo-twrXJxJvqDulF7pnvbeAj3zFRTxryXlIjzMdjH2fHtXUQ9pOhfa2iosQ5tEt7iLWiPv30f_17mXEgDk_fj5v57imyf_hL3E-JAgvqrrEQmL-D_FHWuKon66HO2_X_fewe2AgplJ8222YU1O9mDzRVtwj3YDV5fs49BmvrTS-CDIaLfIzZA-7LhxNSM7vDJJqNYMwTA7Nx6LVbt044syLdevoKfZ8Mfp1-j0Hsh0gjh5hH6tea5RHainUhUXphC8SLXjgwojHKppSrVxJgMGZRVXPLScWcEkl6rCpPuQ2syndjXwKTIpMHZdKkwZJZcISSQ1sXOmizNTNGBz4vvXukgTE79MW4qJChkpWrVSh14vxx92why_GPcgEy4HEMy2v7BdHZZBa-skK4pkSeltMplIjEyjlWJ1Fwjz-vrHCc5XBi1Cr5dV4827MCHZlM8WeXL1a8Tv8r1_E-VJgJR0sHz87yBjYSqKnyJ4yG05rN7-xZe6If5VT3rQnswHI3Puz5d0A17-i8sgASx |
| linkProvider | ProQuest |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1LbxMxEB6VFIlyAFqoCBTwgSIQWrGx92EjIdTQVI3aRhEqqJwWP0sFSko2gPhT_EbGXm9ID3DrgevK8j787cw3Y883AI-Z4E6gX0uYcybJnLWJ5KlIDEd3bpmzheSh2UQ5GvGTEzFegV9tLYw_VtnaxGCozVT7HPkLWiKzR_eeZq_Pvya-a5TfXW1baDSwOLA_f2DIVr8a7uL6blO6Nzh-s5_ErgKJRnIyTxCxmucSebd2gqq8MIXiRa6dfzRhlGPW119SYzKMDazikpeOOyMwnLOqMAznvQKrGYKdd2B1PDwaf1hkdbzKJu-lzQl7xkTq96F71EdZwndwWfJ9oUXARV67rFUa_Nvezf_ty9yCG5FJk50G-uuwYicbcH1JX3ED1qPlqsnTKK_97Dbw_gAZ_EvSR4ySwcTUxJ9DlE1WtCZI4slbG_RkdUidkihBe3oH3l3K-2xCZzKd2LtApMikwdl0qdDtl1whrZHWpc6ajGWm6MLzdmUrHcXVfY-PLxUGWR4H1TIOurC9GH3eiIr8ZVzfg2QxxkuBhwvT2WkVLUuFIacSOS2lVS4T1Mg0VWVOmcZYtadznGSrhU0V7VNd_cFMF540sLtwl92z9zvhLp_nnypGBTK9e_-e5xFc2z8-OqwOh6OD-7BGfZVIKNncgs589s0-gKv6-_ysnj2M_wyBj5cNxt-Lt1T3 |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1NbxMxEB2VFCE4AC0gAgV8oAiEVt3Y-2EjIdSQRESFKEJQldPiz1KBkpINIP4av46x1xvSA9x64Lqy7N3188wbe_wG4CET3An0awlzziSZszaRPBWJ4ejOLXO2kDwUmygnE350JKYb8Ku9C-PTKlubGAy1mWu_R75HS2T26N7TbM_FtIjpYPTi9GviK0j5k9a2nEYDkQP78weGb_Xz8QDnepfS0fDdy1dJrDCQaCQqywTRq3kukYNrJ6jKC1MoXuTa-dcURjlm_V1MakyGcYJVXPLScWcEhnZWFYZhvxdgEyl5lnVgczp-M_2w2uHxipu8lzbZ9oyJ1J9J96iPuISv5rLmB0O5gLMcd123NPi60bX_-S9dh6uRYZP9ZklswYadbcOVNd3FbdiKFq0mj6Ps9pMbwPtDZPbPSB-xS4YzUxOfnyib3dKaILknb23QmdVhS5VEadrjm_D-XL7nFnRm85m9DUSKTBrsTZcK6UDJFdIdaV3qrMlYZoouPG1nudJRdN3X_vhSYfDlMVGtY6ILu6vWp43YyF_a9T1gVm28RHh4MF8cV9HiVBiKKpHTUlrlMkGNTFNV5pRpjGF7OsdOdloIVdFu1dUf_HThUQPBM6MMTg73wyifl58qRgUywDv_7ucBXEIEVq_Hk4O7cJn6yyPhJucOdJaLb_YeXNTflyf14n5cPgQ-njcWfwPTCl24 |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=BEERL%3A+Both+Ends+Explanations+for+Reinforcement+Learning&rft.jtitle=Applied+sciences&rft.au=Terra%2C+Ahmad&rft.au=Inam%2C+Rafia&rft.au=Fersman%2C+Elena&rft.date=2022-11-01&rft.issn=2076-3417&rft.eissn=2076-3417&rft.volume=12&rft.issue=21&rft.spage=10947&rft_id=info:doi/10.3390%2Fapp122110947&rft.externalDBID=n%2Fa&rft.externalDocID=10_3390_app122110947 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2076-3417&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2076-3417&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2076-3417&client=summon |