Co-Designing a Real-Time Classroom Orchestration Tool to Support Teacher–AI Complementarity
Involving stakeholders throughout the creation of new educational technologies can help ensure their usefulness and usability in real-world contexts. However, given the complexity of learning analytics (LA) systems, it can be challenging to meaningfully involve non-technical stakeholders throughout...
Gespeichert in:
| Veröffentlicht in: | Journal of Learning Analytics Jg. 6; H. 2; S. 27 - 52 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Society for Learning Analytics Research
01.01.2019
|
| Schlagworte: | |
| ISSN: | 1929-7750, 1929-7750 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Involving stakeholders throughout the creation of new educational technologies can help ensure their usefulness and usability in real-world contexts. However, given the complexity of learning analytics (LA) systems, it can be challenging to meaningfully involve non-technical stakeholders throughout their design and development. This article reports on the iterative co-design, development, and classroom evaluation of Konscia, a wearable, real-time awareness tool for teachers working in AI-enhanced K-12 classrooms. In the process, we argue that the co-design of LA systems requires new kinds of prototyping methods. We introduce one of our own prototyping methods, REs, to address unique challenges of co-prototyping LA tools. This work presents the first end-to-end demonstration of how non-technical stakeholders can participate throughout the whole design process for a complex LA system—from early generative phases to the selection and tuning of analytics to evaluation in real-world contexts. We conclude by providing methodological recommendations for future LA co-design efforts. |
|---|---|
| AbstractList | Involving stakeholders throughout the creation of new educational technologies can help ensure their usefulness and usability in real-world contexts. However, given the complexity of learning analytics (LA) systems, it can be challenging to meaningfully involve non-technical stakeholders throughout their design and development. This article presents a detailed case study of the iterative co-design of Lumilo, a wearable, real-time learning analytics tool for teachers working in AI-enhanced K-12 classrooms. In the process, we argue that the co-design of LA systems requires "new kinds of prototyping methods." We introduce one of our own prototyping methods, REs, to address unique challenges of co-prototyping data-driven algorithmic systems such as LA tools. This work presents the first end-to-end demonstration in the literature of how non-technical stakeholders can participate throughout the whole design process for a complex LA system -- from early generative phases to the selection and tuning of analytics to evaluation in real-world contexts. We conclude with a summary of methodological recommendations for future LA co-design efforts. Involving stakeholders throughout the creation of new educational technologies can help ensure their usefulness and usability in real-world contexts. However, given the complexity of learning analytics (LA) systems, it can be challenging to meaningfully involve non-technical stakeholders throughout their design and development. This article reports on the iterative co-design, development, and classroom evaluation of Konscia, a wearable, real-time awareness tool for teachers working in AI-enhanced K-12 classrooms. In the process, we argue that the co-design of LA systems requires new kinds of prototyping methods. We introduce one of our own prototyping methods, REs, to address unique challenges of co-prototyping LA tools. This work presents the first end-to-end demonstration of how non-technical stakeholders can participate throughout the whole design process for a complex LA system—from early generative phases to the selection and tuning of analytics to evaluation in real-world contexts. We conclude by providing methodological recommendations for future LA co-design efforts. |
| Audience | Elementary Secondary Education Junior High Schools Middle Schools Secondary Education |
| Author | Aleven, Vincent McLaren, Bruce M. Holstein, Kenneth |
| Author_xml | – sequence: 1 givenname: Kenneth surname: Holstein fullname: Holstein, Kenneth – sequence: 2 givenname: Bruce M. surname: McLaren fullname: McLaren, Bruce M. – sequence: 3 givenname: Vincent surname: Aleven fullname: Aleven, Vincent |
| BackLink | http://eric.ed.gov/ERICWebPortal/detail?accno=EJ1224130$$DView record in ERIC |
| BookMark | eNp1kN1KwzAYhoNMcM6deibkBlq_JP3L4ahTJ4OB1kMJaZrNjLYpaTzYmffgHXoldpuCCB59H7w_8D7naNTaViN0SSAkWQLZ9baWIQXCw4SG7ASNCac8SNMYRr_-MzTt-y0AMEoY4zBGL7kNbnRvNq1pN1jiRy3roDCNxnkt-95Z2-CVU6-69056Y1tcWFtjb_HTW9dZ53Gh5SC7z_eP2QLntulq3ejWS2f87gKdrmXd6-n3naDn23mR3wfL1d0iny0DxYD4gCQadFpKkvBEsoxXKlWcApMEIhJnkNJMsSSWexePK8UrqsuyUlFFIk3iik3Q1bFXO6NE50wj3U7MHwilEWEw6NFRV84Oo_RaKOMPc4ZVphYExIGiGCiKPUWRUMGGWPgn9lP9T-ALXQd3AA |
| CitedBy_id | crossref_primary_10_1007_s10648_025_10011_9 crossref_primary_10_1016_j_caeai_2025_100461 crossref_primary_10_1145_3579620 crossref_primary_10_1057_s41599_023_01633_x crossref_primary_10_1016_j_compedu_2022_104615 crossref_primary_10_1007_s40593_021_00278_8 crossref_primary_10_1016_j_acalib_2024_102849 crossref_primary_10_1080_10510974_2025_2544831 crossref_primary_10_1007_s10648_024_09876_z crossref_primary_10_1007_s10758_025_09887_x crossref_primary_10_1007_s11528_025_01060_6 crossref_primary_10_1007_s00146_024_02047_2 crossref_primary_10_1007_s40593_023_00380_z crossref_primary_10_1002_fer3_46 crossref_primary_10_1007_s11528_025_01102_z crossref_primary_10_1145_3637358 crossref_primary_10_1080_10494820_2025_2488143 crossref_primary_10_1109_ACCESS_2022_3177752 crossref_primary_10_1111_jcal_13088 crossref_primary_10_1155_2022_1692080 crossref_primary_10_1177_07356331251337991 crossref_primary_10_1186_s40561_025_00410_4 crossref_primary_10_1145_3579628 crossref_primary_10_1111_jcal_70025 crossref_primary_10_1111_cogs_13365 crossref_primary_10_1111_bjet_13372 crossref_primary_10_1186_s40561_024_00360_3 crossref_primary_10_1007_s40593_024_00410_4 crossref_primary_10_1007_s40593_024_00455_5 crossref_primary_10_1007_s40670_025_02459_9 crossref_primary_10_1007_s10639_023_11909_z crossref_primary_10_1111_bjet_12876 crossref_primary_10_1111_bjet_13326 crossref_primary_10_30605_onoma_v11i3_6554 crossref_primary_10_1080_10494820_2025_2472288 crossref_primary_10_1016_j_caeai_2024_100355 crossref_primary_10_3389_feduc_2022_736194 crossref_primary_10_1109_TLT_2023_3248155 crossref_primary_10_1111_ejed_12527 crossref_primary_10_15700_saje_v44n4a2540 crossref_primary_10_1080_0144929X_2024_2345295 crossref_primary_10_1186_s40561_024_00335_4 crossref_primary_10_61007_QdC_2023_2_119 crossref_primary_10_1016_j_compedu_2020_103933 crossref_primary_10_1186_s40594_022_00377_5 crossref_primary_10_1111_bjet_13363 crossref_primary_10_1007_s11412_023_09414_z crossref_primary_10_1108_IJLLS_01_2025_0029 crossref_primary_10_1007_s40573_025_00177_8 crossref_primary_10_1016_j_system_2022_102830 crossref_primary_10_1007_s11528_022_00715_y crossref_primary_10_1080_15391523_2022_2148785 crossref_primary_10_1080_2331186X_2024_2393530 crossref_primary_10_1007_s10639_023_11834_1 crossref_primary_10_1111_bjet_13435 crossref_primary_10_1145_3579601 crossref_primary_10_1007_s10758_022_09598_7 crossref_primary_10_1002_aaai_12058 crossref_primary_10_3390_educsci15060724 crossref_primary_10_29333_iejme_12588 crossref_primary_10_3389_feduc_2025_1406699 crossref_primary_10_1007_s10639_023_12109_5 crossref_primary_10_1007_s10758_024_09759_w crossref_primary_10_1007_s40593_020_00226_y crossref_primary_10_1145_3449193 crossref_primary_10_12688_f1000research_160011_1 crossref_primary_10_1002_berj_4200 crossref_primary_10_1007_s11528_023_00903_4 crossref_primary_10_1007_s40593_020_00198_z crossref_primary_10_1007_s10956_024_10173_1 crossref_primary_10_1007_s40593_024_00456_4 crossref_primary_10_1016_j_compedu_2023_104864 crossref_primary_10_1007_s40692_024_00332_z crossref_primary_10_1080_0144929X_2024_2394886 crossref_primary_10_1007_s10639_025_13421_y crossref_primary_10_1111_jcal_12711 crossref_primary_10_1111_jcal_12997 crossref_primary_10_1007_s11409_023_09346_x crossref_primary_10_1080_15391523_2022_2142873 crossref_primary_10_1007_s10639_023_12327_x crossref_primary_10_1080_10494820_2019_1710541 crossref_primary_10_1109_ACCESS_2025_3532883 crossref_primary_10_1007_s40593_021_00259_x crossref_primary_10_3390_informatics10030074 crossref_primary_10_1007_s10639_023_12059_y crossref_primary_10_1016_j_chb_2022_107468 crossref_primary_10_1007_s10956_022_09984_x crossref_primary_10_1016_j_edurev_2024_100616 |
| ContentType | Journal Article |
| DBID | AAYXX CITATION ERI GA5 |
| DOI | 10.18608/jla.2019.62.3 |
| DatabaseName | CrossRef ERIC ERIC - Full Text Only (Discovery) |
| DatabaseTitle | CrossRef ERIC |
| DatabaseTitleList | ERIC CrossRef |
| DeliveryMethod | fulltext_linktorsrc |
| EISSN | 1929-7750 |
| ERIC | EJ1224130 |
| ExternalDocumentID | EJ1224130 10_18608_jla_2019_62_3 |
| GroupedDBID | AAYXX ABOPQ ALMA_UNASSIGNED_HOLDINGS CITATION FRS M~E OK1 ERI GA5 |
| ID | FETCH-LOGICAL-c301t-16e0e7ba1696a389dc7c9203a1041580728c365ae0e795dc9d2ebbdc4d14e15d3 |
| ISICitedReferencesCount | 175 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000541107300003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1929-7750 |
| IngestDate | Tue Dec 02 16:48:47 EST 2025 Tue Nov 18 22:19:07 EST 2025 Sat Nov 29 02:59:07 EST 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 2 |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c301t-16e0e7ba1696a389dc7c9203a1041580728c365ae0e795dc9d2ebbdc4d14e15d3 |
| OpenAccessLink | http://eric.ed.gov/ERICWebPortal/detail?accno=EJ1224130 |
| PageCount | 26 |
| ParticipantIDs | eric_primary_EJ1224130 crossref_citationtrail_10_18608_jla_2019_62_3 crossref_primary_10_18608_jla_2019_62_3 |
| PublicationCentury | 2000 |
| PublicationDate | 2019-01-01 |
| PublicationDateYYYYMMDD | 2019-01-01 |
| PublicationDate_xml | – month: 01 year: 2019 text: 2019-01-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationTitle | Journal of Learning Analytics |
| PublicationYear | 2019 |
| Publisher | Society for Learning Analytics Research |
| Publisher_xml | – name: Society for Learning Analytics Research |
| SSID | ssj0003213390 |
| Score | 2.5342693 |
| Snippet | Involving stakeholders throughout the creation of new educational technologies can help ensure their usefulness and usability in real-world contexts. However,... |
| SourceID | eric crossref |
| SourceType | Open Access Repository Enrichment Source Index Database |
| StartPage | 27 |
| SubjectTerms | Artificial Intelligence Design Elementary Secondary Education Learning Analytics Middle School Teachers Stakeholders Teacher Participation Technology Uses in Education |
| Title | Co-Designing a Real-Time Classroom Orchestration Tool to Support Teacher–AI Complementarity |
| URI | http://eric.ed.gov/ERICWebPortal/detail?accno=EJ1224130 |
| Volume | 6 |
| WOSCitedRecordID | wos000541107300003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 1929-7750 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0003213390 issn: 1929-7750 databaseCode: M~E dateStart: 20140101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Na9wwEBXbbSG9lJQ2NP1Ch0IPRVtbtiXruKQpSdimPWwh0IOxZAVSjDds3JBTf0F_dGdkWfY2WWgPvRgjC9nMPI9H4zczhLyJjMpSm5as4ipiqY0rpk1ascToUgojdGxc15KFPD3Nz87Ul8nkV58Lc13LpslvbtTlf1U1jIGyMXX2H9QdFoUBOAelwxHUDse_UvzBin1wtAyXfogs-ZphokfX_xId5Xef165Nllf-EstwgguKDT7BGe-LPLP5sbMWnl-OTe62eLKLEF7BCiftiD9_tKqv-naaPgNoCAQuynVn8xy-hrDsvMaqUo6Ae9GYnpjjIxOYDLURmRjTTm8_SGAWjowvuGrg7XeFaGf2jjFvscUImHxsfeWdH4VcuEyH7zXWmYrVTPBZMnz-Ainx8MT_ZbxH7nOZKbSRn34OEbuEw37eBe3CQ_kqoHiD9xvL75CdsNyGxzPtmXDeg1nukkdeYXTeQeYxmdjmCfk2hgstaYALDXChG3ChCBfarqiHCx3gQv-Ay1Py9ePh8uCI-YYbzICdb1ksbGSlLmOhRAmebGWkUTxKyhgLOeSR5LlJRFbiLJVVRlXcal3Bux3DW55VyR6ZNqvGPiMUJCXP00hrYzT-Otc5yELmaZLYc9hFy33CeqEUxlejx6YodYG7UhRoAQItUKCF4EWyT96G-ZddHZatM_dQxmFW0MPzbRdekIcDfF-Sabv-YV-RB-a6vbhav3Yg-A1BB4HG |
| linkProvider | ISSN International Centre |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Co-Designing+a+Real-Time+Classroom+Orchestration+Tool+to+Support+Teacher-AI+Complementarity&rft.jtitle=Journal+of+Learning+Analytics&rft.au=Holstein%2C+Kenneth&rft.au=McLaren%2C+Bruce+M&rft.au=Aleven%2C+Vincent&rft.date=2019-01-01&rft.pub=Society+for+Learning+Analytics+Research&rft.issn=1929-7750&rft.eissn=1929-7750&rft.volume=6&rft.issue=2&rft.spage=27&rft_id=info:doi/10.18608%2Fjla.2019.62.3&rft.externalDocID=EJ1224130 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1929-7750&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1929-7750&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1929-7750&client=summon |