Evaluating the effects of Generative AI on student learning outcomes: Insights from a meta-analysis
The emergence of Generative AI technologies, represented by ChatGPT, has triggered extensive discussions among scholars in the education sector. While relevant research continues to grow, there is a lack of comprehensive understanding that systematically measures the effects of Generative AI on stud...
Uloženo v:
| Vydáno v: | Educational Technology ＆ Society Ročník 28; číslo 3; s. 226 - 240 |
|---|---|
| Hlavní autoři: | , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
International Forum of Educational Technology ＆ Society
01.07.2025
International Forum of Educational Technology & Society, National Taiwan Normal University, Taiwan International Forum of Educational Technology & Society |
| Témata: | |
| ISSN: | 1176-3647, 1436-4522, 1436-4522 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | The emergence of Generative AI technologies, represented by ChatGPT, has triggered extensive discussions among scholars in the education sector. While relevant research continues to grow, there is a lack of comprehensive understanding that systematically measures the effects of Generative AI on student learning outcomes. This study employs meta-analysis to integrate findings from previous experimental and quasi-experimental research to evaluate the impact of Generative AI on student learning outcomes. The analysis of 44 effect sizes from 21 independent studies indicates that Generative AI tools, compared to traditional AI tools or no intervention, moderately enhance student learning outcomes (g = 0.572). These tools significantly improve the cognitive (g = 0.604), behavioral (g = 0.698), and affective (g = 0.478) dimensions of learning outcomes. In addition, the study identifies and examines 6 potential moderating variables: educational level, sample size, subject area, teaching model, intervention duration, and assessment instrument. The results of the moderating effects test reveal that sample size and assessment instrument significantly influence the effectiveness of Generative AI. For sample size, the effect of Generative AI on small samples (g = 1.216) is greater than that on medium (g = 0.476) and large samples (g = 0.547). For assessment instrument, the effect of Generative AI on self-developed tests (g = 0.984) is greater than that on standardized tests (g = 0.557). The meta-analysis result indicated that the use of Generative AI should be supplemented with detailed guidance and flexible strategies. Specific recommendations for future research and practical implementations of Generative AI in education are discussed. |
|---|---|
| AbstractList | The emergence of Generative AI technologies, represented by ChatGPT, has triggered extensive discussions among scholars in the education sector. While relevant research continues to grow, there is a lack of comprehensive understanding that systematically measures the effects of Generative AI on student learning outcomes. This study employs meta-analysis to integrate findings from previous experimental and quasi-experimental research to evaluate the impact of Generative AI on student learning outcomes. The analysis of 44 effect sizes from 21 independent studies indicates that Generative AI tools, compared to traditional AI tools or no intervention, moderately enhance student learning outcomes (g = 0.572). These tools significantly improve the cognitive (g = 0.604), behavioral (g = 0.698), and affective (g = 0.478) dimensions of learning outcomes. In addition, the study identifies and examines 6 potential moderating variables: educational level, sample size, subject area, teaching model, intervention duration, and assessment instrument. The results of the moderating effects test reveal that sample size and assessment instrument significantly influence the effectiveness of Generative AI. For sample size, the effect of Generative AI on small samples (g = 1.216) is greater than that on medium (g = 0.476) and large samples (g = 0.547). For assessment instrument, the effect of Generative AI on self-developed tests (g = 0.984) is greater than that on standardized tests (g = 0.557). The meta-analysis result indicated that the use of Generative AI should be supplemented with detailed guidance and flexible strategies. Specific recommendations for future research and practical implementations of Generative AI in education are discussed. The emergence of Generative AI technologies, represented by ChatGPT, has triggered extensive discussions among scholars in the education sector. While relevant research continues to grow, there is a lack of comprehensive understanding that systematically measures the effects of Generative AI on student learning outcomes. This study employs meta-analysis to integrate findings from previous experimental and quasi-experimental research to evaluate the impact of Generative AI on student learning outcomes. The analysis of 44 effect sizes from 21 independent studies indicates that Generative AI tools, compared to traditional AI tools or no intervention, moderately enhance student learning outcomes (g = 0.572). These tools significantly improve the cognitive (g = 0.604), behavioral (g = 0.698), and affective (g = 0.478) dimensions of learning outcomes. In addition, the study identifies and examines 6 potential moderating variables: educational level, sample size, subject area, teaching model, intervention duration, and assessment instrument. The results of the moderating effects test reveal that sample size and assessment instrument significantly influence the effectiveness of Generative AI. For sample size, the effect of Generative AI on small samples (g = 1.216) is greater than that on medium (g = 0.476) and large samples (g = 0.547). For assessment instrument, the effect of Generative AI on self-developed tests (g = 0.984) is greater than that on standardized tests (g = 0.557). The meta-analysis result indicated that the use of Generative AI should be supplemented with detailed guidance and flexible strategies. Specific recommendations for future research and practical implementations of Generative AI in education are discussed. Keywords: Education, Generative artificial intelligence, Generative AI, Learning outcomes, Meta-analysis |
| Audience | Academic |
| Author | Zhe Xing De-Xin Hu Dan-Dan Pang |
| Author_xml | – sequence: 1 givenname: De-Xin surname: Hu fullname: Hu, De-Xin – sequence: 2 givenname: Dan-Dan surname: Pang fullname: Pang, Dan-Dan – sequence: 3 givenname: Zhe surname: Xing fullname: Xing, Zhe |
| BookMark | eNptUVtrFDEUHqSCbfUnKAFf9GHW3GaS8W0pa10oWnB9DmeTk22WnUSSbKH_3rQrQqEETg7nuySc76I7iyli131gdCEom9iX1ebXglM-UGW4_iQ-Lza3lL_qzpkUYy8Hzs9az9TYi1GqN91FKXtKOZUjPe_s6h4OR6gh7ki9Q4Leo62FJE-uMWJuyD2S5ZqkSEo9OoyVHBByfBSkY7VpxvKVrGMJu7um8znNBMiMFXqIcHgoobztXns4FHz3777sfn9bba6-9zc_r9dXy5sexEhrrwBRI4CbGLhBb52brFfTBALcdtSgKOOCofSgJkn91krrpwEpOu4pUCsuu_XJ1yXYmz85zJAfTIJgngYp7wzkGuwBjZIouWSWOTZIpVUzG6z0YtTi8UlsXh9PXjto9BB9qhnsHIo1Sz3QcWRa08ZavMBqx-EcbIvJhzZ_Jnh_EuxLTfn_H6XWXE3D1HB1wiHkUIPZp2NuWyzmtgXMGeeiJWd-PKU9cEkpE4a2KsVfQiWd4w |
| ContentType | Journal Article |
| Copyright | COPYRIGHT 2025 International Forum of Educational Technology & Society |
| Copyright_xml | – notice: COPYRIGHT 2025 International Forum of Educational Technology & Society |
| DBID | 188 DOA |
| DOI | 10.30191/ETS.202507_28(3).TP02 |
| DatabaseName | Airiti Library DOAJ Directory of Open Access Journals |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: DOA name: DOAJ Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Education |
| EISSN | 1436-4522 |
| EndPage | 240 |
| ExternalDocumentID | oai_doaj_org_article_74e4241c1d154787bc45c4f3683f799e A850661880 48827959 P20221223002_N202505240013_00014 |
| GroupedDBID | -W8 0-V 0S8 188 29G 2UF 2WC 5GY 5VS 8FE 8FG 8FQ 8G5 AAFWJ AAGJD AAHSB AAKPC ABBHK ABCFB ABDBF ABIVO ABOPQ ABUWG ACGFO ACHQT ACUHS ADBBV ADDVE ADMLS ADNHR AEUPB AFDYH AFKRA AFPKN AINHJ ALMA_UNASSIGNED_HOLDINGS ALSLI ARALO ARAPS AZQEC BCNDV BENPR BGLVJ BPHCQ CCPQU CDVRH CJNVE CNMHZ DWQXO E3Z EAD EAP EAS EBS EDJ EJD EMK ESX FRA GNUQQ GROUPED_DOAJ GUQSH HCIFZ IAO IER IPSME ISE ISN ITC JAAYA JBMMH JENOY JHFFW JKQEH JLEZI JLXEF JPL JST K6V K7- M0P M2O M3F M~E OK1 OVT P2P P62 PHGZM PHGZT PIMPY PQEDU PQGLB PQQKQ PROAC PV9 QF4 QM7 QN7 RNS RZL SA0 TR2 UZ2 XH6 XSB AFFHD BAIFH BBTPI |
| ID | FETCH-LOGICAL-a360t-7aee8eaad91ad58bdd9cf799a3adb68a701231e4fa7940fbc4cf95e0ed2f0a0c3 |
| IEDL.DBID | DOA |
| ISICitedReferencesCount | 1 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001529254900014&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1176-3647 1436-4522 |
| IngestDate | Fri Oct 03 12:51:08 EDT 2025 Sat Nov 29 13:46:58 EST 2025 Sat Nov 29 10:28:54 EST 2025 Sun Jul 13 12:10:56 EDT 2025 Tue Aug 19 00:40:54 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 3 |
| Keywords | Generative AI Learning outcomes Education Generative artificial intelligence Meta-analysis |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a360t-7aee8eaad91ad58bdd9cf799a3adb68a701231e4fa7940fbc4cf95e0ed2f0a0c3 |
| OpenAccessLink | https://doaj.org/article/74e4241c1d154787bc45c4f3683f799e |
| PageCount | 15 |
| ParticipantIDs | doaj_primary_oai_doaj_org_article_74e4241c1d154787bc45c4f3683f799e gale_infotracmisc_A850661880 gale_infotracacademiconefile_A850661880 jstor_primary_48827959 airiti_journals_P20221223002_N202505240013_00014 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-07-01 |
| PublicationDateYYYYMMDD | 2025-07-01 |
| PublicationDate_xml | – month: 07 year: 2025 text: 2025-07-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationTitle | Educational Technology ＆ Society |
| PublicationYear | 2025 |
| Publisher | International Forum of Educational Technology ＆ Society International Forum of Educational Technology & Society, National Taiwan Normal University, Taiwan International Forum of Educational Technology & Society |
| Publisher_xml | – name: International Forum of Educational Technology ＆ Society – name: International Forum of Educational Technology & Society, National Taiwan Normal University, Taiwan – name: International Forum of Educational Technology & Society |
| SSID | ssj0020460 |
| Score | 2.4204507 |
| Snippet | The emergence of Generative AI technologies, represented by ChatGPT, has triggered extensive discussions among scholars in the education sector. While relevant... |
| SourceID | doaj gale jstor airiti |
| SourceType | Open Website Aggregation Database Publisher |
| StartPage | 226 |
| SubjectTerms | Analysis Education Educational evaluation generative ai generative artificial intelligence Generative artificial intelligence in education: Theories, technologies, and applications Learning learning outcomes meta-analysis Methods Study and teaching |
| Title | Evaluating the effects of Generative AI on student learning outcomes: Insights from a meta-analysis |
| URI | https://www.airitilibrary.com/Article/Detail/P20221223002-N202505240013-00014 https://www.jstor.org/stable/48827959 https://doaj.org/article/74e4241c1d154787bc45c4f3683f799e |
| Volume | 28 |
| WOSCitedRecordID | wos001529254900014&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: DOAJ Directory of Open Access Journals customDbUrl: eissn: 1436-4522 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0020460 issn: 1176-3647 databaseCode: DOA dateStart: 19980101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 1436-4522 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0020460 issn: 1176-3647 databaseCode: M~E dateStart: 19980101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre – providerCode: PRVPQU databaseName: Advanced Technologies & Aerospace Database customDbUrl: eissn: 1436-4522 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0020460 issn: 1176-3647 databaseCode: P5Z dateStart: 20040101 isFulltext: true titleUrlDefault: https://search.proquest.com/hightechjournals providerName: ProQuest – providerCode: PRVPQU databaseName: Canadian Business & Current Affairs Database customDbUrl: eissn: 1436-4522 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0020460 issn: 1176-3647 databaseCode: 8FQ dateStart: 20040101 isFulltext: true titleUrlDefault: https://search.proquest.com/cbcacomplete providerName: ProQuest – providerCode: PRVPQU databaseName: Computer Science Database customDbUrl: eissn: 1436-4522 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0020460 issn: 1176-3647 databaseCode: K7- dateStart: 20040101 isFulltext: true titleUrlDefault: http://search.proquest.com/compscijour providerName: ProQuest – providerCode: PRVPQU databaseName: Education Database customDbUrl: eissn: 1436-4522 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0020460 issn: 1176-3647 databaseCode: M0P dateStart: 20040101 isFulltext: true titleUrlDefault: https://search.proquest.com/education providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Central customDbUrl: eissn: 1436-4522 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0020460 issn: 1176-3647 databaseCode: BENPR dateStart: 20040101 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest – providerCode: PRVPQU databaseName: Publicly Available Content Database customDbUrl: eissn: 1436-4522 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0020460 issn: 1176-3647 databaseCode: PIMPY dateStart: 20040101 isFulltext: true titleUrlDefault: http://search.proquest.com/publiccontent providerName: ProQuest – providerCode: PRVPQU databaseName: Research Library customDbUrl: eissn: 1436-4522 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0020460 issn: 1176-3647 databaseCode: M2O dateStart: 20040101 isFulltext: true titleUrlDefault: https://search.proquest.com/pqrl providerName: ProQuest |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1Lb9QwELZQ1QMXxKtioUU-IAEHU8d5OOltQVvRA8tKLFJv1sQeVz2QrTYpv5-Z2FTtiQuXHBIfrG_e8fgbId71MSIFDlDR1qjIEmvVVyUqtKhj5yNH9XnYhF2v28vLbnNv1Bf3hCV64ATcqa2woijji1Aw9ZTtfVX7KpZNW0bbdcjel7Kev8VULrX4uC-NVWkUM6Snu8GkzF1xutr-oLqQQr91pv1Qfvy03fBPlUO4ZhqhzN1_56HvdynOMef8qXiSk0W5TJt8Jh7h8JznLOeejBfCrzJZ93AlKZOTuTtD7qJMfNLszOTyQu4GOSYWS5nnRFzJ3e1E6objmbwYRq7RR8mXTSTIXziBgkxX8lL8PF9tv3xVeWyCgrLRk7KA2CJA6AoIdduHQJgTVFBC6JsWLKdRBVYRyBZ1JEB97GrUGEzUoH15JA6G3YCvhASDrfY9hmjKCoxpMYSGlug6AuWB_ULohJrLmj-6DUFL4ZAqG23cesa55j7VopzPuauF-Mz4uptEpuGY3np-QUJ3WejuX0JfiPcsHcdGOO3BQ75LQLtmOiu3ZB6-hqnmFuL4wUoyHv_g89Es37vtkD8zPH799f_Y5hvxmAFI3b3H4mDa3-KJOPS_p-tx_3bWWHp-M9__ACeW664 |
| linkProvider | Directory of Open Access Journals |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Evaluating+the+effects+of+Generative+AI+on+student+learning+outcomes%3A+Insights+from+a+meta-analysis&rft.jtitle=Educational+technology+%26+society&rft.au=Hu%2C+De-Xin&rft.au=Pang%2C+Dan-Dan&rft.au=Xing%2C+Zhe&rft.date=2025-07-01&rft.pub=International+Forum+of+Educational+Technology+%26+Society&rft.issn=1436-4522&rft.volume=28&rft.issue=3&rft.spage=226&rft_id=info:doi/10.30191%2FETS.202507_28%283%29.TP02&rft.externalDocID=A850661880 |
| thumbnail_m | http://cvtisr.summon.serialssolutions.com/2.0.0/image/custom?url=https%3A%2F%2Fwww.airitilibrary.com%2Fjnltitledo%2FP20221223002-c.jpg |