From Predictive Algorithms to Automatic Generation of Anomalies
Machine learning algorithms can find predictive signals that researchers fail to notice; yet they are notoriously hard-to-interpret. How can we extract theoretical insights from these black boxes? History provides a clue. Facing a similar problem – how to extract theoretical insights from their intu...
Uložené v:
| Vydané v: | NBER Working Paper Series |
|---|---|
| Hlavní autori: | , |
| Médium: | Paper |
| Jazyk: | English |
| Vydavateľské údaje: |
Cambridge
National Bureau of Economic Research, Inc
01.05.2024
|
| Predmet: | |
| ISSN: | 0898-2937 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Machine learning algorithms can find predictive signals that researchers fail to notice; yet they are notoriously hard-to-interpret. How can we extract theoretical insights from these black boxes? History provides a clue. Facing a similar problem – how to extract theoretical insights from their intuitions – researchers often turned to “anomalies:” constructed examples that highlight flaws in an existing theory and spur the development of new ones. Canonical examples include the Allais paradox and the Kahneman-Tversky choice experiments for expected utility theory. We suggest anomalies can extract theoretical insights from black box predictive algorithms. We develop procedures to automatically generate anomalies for an existing theory when given a predictive algorithm. We cast anomaly generation as an adversarial game between a theory and a falsifier, the solutions to which are anomalies: instances where the black box algorithm predicts - were we to collect data - we would likely observe violations of the theory. As an illustration, we generate anomalies for expected utility theory using a large, publicly available dataset on real lottery choices. Based on an estimated neural network that predicts lottery choices, our procedures recover known anomalies and discover new ones for expected utility theory. In incentivized experiments, subjects violate expected utility theory on these algorithmically generated anomalies; moreover, the violation rates are similar to observed rates for the Allais paradox and Common ratio effect. |
|---|---|
| AbstractList | Machine learning algorithms can find predictive signals that researchers fail to notice; yet they are notoriously hard-to-interpret. How can we extract theoretical insights from these black boxes? History provides a clue. Facing a similar problem – how to extract theoretical insights from their intuitions – researchers often turned to “anomalies:” constructed examples that highlight flaws in an existing theory and spur the development of new ones. Canonical examples include the Allais paradox and the Kahneman-Tversky choice experiments for expected utility theory. We suggest anomalies can extract theoretical insights from black box predictive algorithms. We develop procedures to automatically generate anomalies for an existing theory when given a predictive algorithm. We cast anomaly generation as an adversarial game between a theory and a falsifier, the solutions to which are anomalies: instances where the black box algorithm predicts - were we to collect data - we would likely observe violations of the theory. As an illustration, we generate anomalies for expected utility theory using a large, publicly available dataset on real lottery choices. Based on an estimated neural network that predicts lottery choices, our procedures recover known anomalies and discover new ones for expected utility theory. In incentivized experiments, subjects violate expected utility theory on these algorithmically generated anomalies; moreover, the violation rates are similar to observed rates for the Allais paradox and Common ratio effect. |
| Author | Rambachan, Ashesh Mullainathan, Sendhil |
| Author_xml | – sequence: 1 givenname: Sendhil surname: Mullainathan fullname: Mullainathan, Sendhil – sequence: 2 givenname: Ashesh surname: Rambachan fullname: Rambachan, Ashesh |
| BookMark | eNotjU1LAzEYhHOo0A_rbwh4Xk3y7maTkyzFtkJBD72XbD40ZTfRJKt_3wVlBuZhDjNrtAgxWITuKHkAEPzxB1jN2AKtiJCiYhLaJVrnfCWECUHoCj3tUxzxW7LG6-K_Le6G95h8-RgzLhF3U4mjKl7jgw02zRQDjg53Ya4Hb_MtunFqyHb7nxt03j-fd8fq9Hp42XWnKtRtWynaUKOt5jXnchbRHCRXTW-U6alTpp4NUoM00lnVCAEKeuGo0pz2XMMG3f_Nfqb4NdlcLtc4pTA_XoA0lMqG1S38AmMkSVU |
| ContentType | Paper |
| Copyright | Copyright National Bureau of Economic Research, Inc. 2024 |
| Copyright_xml | – notice: Copyright National Bureau of Economic Research, Inc. 2024 |
| DOI | 10.3386/w32422 |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Economics |
| Genre | Working Paper/Pre-Print |
| GroupedDBID | -~X 7WY 8FL AAWTO ABDNZ AFKRA ALMA_UNASSIGNED_HOLDINGS BENPR F5P K60 K6~ M0C NBR YNT |
| ID | FETCH-LOGICAL-n477-a151dcec646696960c6396a5bdadb1fad4ad439c39d9fea5883a3b8f1ac61b6c3 |
| ISSN | 0898-2937 |
| IngestDate | Sat Oct 25 08:22:13 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-n477-a151dcec646696960c6396a5bdadb1fad4ad439c39d9fea5883a3b8f1ac61b6c3 |
| Notes | SourceType-Working Papers-1 ObjectType-Working Paper/Pre-Print-1 content type line 50 |
| PQID | 3051195247 |
| PQPubID | 2036197 |
| ParticipantIDs | proquest_journals_3051195247 |
| PublicationCentury | 2000 |
| PublicationDate | 20240501 |
| PublicationDateYYYYMMDD | 2024-05-01 |
| PublicationDate_xml | – month: 05 year: 2024 text: 20240501 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | Cambridge |
| PublicationPlace_xml | – name: Cambridge |
| PublicationTitle | NBER Working Paper Series |
| PublicationYear | 2024 |
| Publisher | National Bureau of Economic Research, Inc |
| Publisher_xml | – name: National Bureau of Economic Research, Inc |
| SSID | ssj0028801 |
| Score | 1.7211838 |
| Snippet | Machine learning algorithms can find predictive signals that researchers fail to notice; yet they are notoriously hard-to-interpret. How can we extract... |
| SourceID | proquest |
| SourceType | Aggregation Database |
| SubjectTerms | Algorithms Expected utility |
| Title | From Predictive Algorithms to Automatic Generation of Anomalies |
| URI | https://www.proquest.com/docview/3051195247 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Li9swEBbdTaF765Nudxt8KL0U0ziSHzqVZEnoIeuG4NL0FPRys5C1U9tp9-d3Rn6FLpT2UDDCyGAhf_JoZvTNDCFvuNawbTLppsJjLtOpcrmvMTgHNlhDufSkRXoRxnG0XvNlU5mutOUEwiyL7u74_r9CDX0ANobO_gPc3UuhA-4BdGgBdmj_Cvg5BowsCzyAsbSgye5bXtxU21uby2FyqPI6S2udcLpVGCcZdO9aRmGjrcbT2ap1p4OuuTcFypYj2uE1GLAYfoX-d-tHNZne9qSNlbiVQjXPJuXWlNtjL8OY9Zy-4yTdu3dT5MofLOuxCZzuKIL3yJujiGOER53U5XdRDaYxeg1-okI37jej9gA-_rSZf14sNslsnbzdf3exTBgepzc1U07IYBz6HMXY9eiqs7BBGNkSie3AdTEpHOp9PdC9HdeqEcljMrAf8Ql5YLKn5FE7t_IZ-YCoOT1qTo-aU-VOh5rTo-bkqdOh9pwk81ly9dFtalu4GQtDV4CipZVRAQsCzE80UqApBsKXWmjppUIzuChXlGueGuFHERVURqknVAB_j6IvyGmWZ-YlcTwufaY1HnhSRsGAFZ7mYIVLqrSiVJ6Ty3bKm2aZlhuQ8pjrb8zCV39-fEHO-uVwSU6r4mBek4fqR3VTFkNyEn75OiSD6SxeroYWi1-znT1R |
| linkProvider | ProQuest |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=From+Predictive+Algorithms+to+Automatic+Generation+of+Anomalies&rft.jtitle=NBER+Working+Paper+Series&rft.au=Mullainathan%2C+Sendhil&rft.au=Rambachan%2C+Ashesh&rft.date=2024-05-01&rft.pub=National+Bureau+of+Economic+Research%2C+Inc&rft.issn=0898-2937&rft_id=info:doi/10.3386%2Fw32422&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0898-2937&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0898-2937&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0898-2937&client=summon |