A Comparison of Constraint Programming and Mixed-Integer Programming for Automated Test-Form Generation
The final step of the typical process of developing educational and psychological tests is to place the selected test items in a formatted form. The step involves the grouping and ordering of the items to meet a variety of formatting constraints. As this activity tends to be time-intensive, the use...
Uloženo v:
| Vydáno v: | Journal of educational measurement Ročník 55; číslo 4; s. 435 - 456 |
|---|---|
| Hlavní autoři: | , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Madison
National Council on Measurement in Education
01.12.2018
Wiley-Blackwell Wiley Subscription Services, Inc |
| Témata: | |
| ISSN: | 0022-0655, 1745-3984 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | The final step of the typical process of developing educational and psychological tests is to place the selected test items in a formatted form. The step involves the grouping and ordering of the items to meet a variety of formatting constraints. As this activity tends to be time-intensive, the use of mixed-integer programming (MIP) has been proposed to automate it. The goal of this article is to show how constraint programming (CP) can be used as an alternative to automate test-form generation problems with a large variety of formatting constraints, and how it compares with MIP-based form generation as for its models, solutions, and running times. Two empirical examples are presented: (i) automated generation of a computerized fixedform; and (ii) automated generation of shadow tests for multistage testing. Both examples show that CP works well with feasible solutions and running times likely to be better than that for MIP-based applications. |
|---|---|
| AbstractList | The final step of the typical process of developing educational and psychological tests is to place the selected test items in a formatted form. The step involves the grouping and ordering of the items to meet a variety of formatting constraints. As this activity tends to be time-intensive, the use of mixed-integer programming (MIP) has been proposed to automate it. The goal of this article is to show how constraint programming (CP) can be used as an alternative to automate test-form generation problems with a large variety of formatting constraints, and how it compares with MIP-based form generation as for its models, solutions, and running times. Two empirical examples are presented: (i) automated generation of a computerized fixedform; and (ii) automated generation of shadow tests for multistage testing. Both examples show that CP works well with feasible solutions and running times likely to be better than that for MIP-based applications. The final step of the typical process of developing educational and psychological tests is to place the selected test items in a formatted form. The step involves the grouping and ordering of the items to meet a variety of formatting constraints. As this activity tends to be time‐intensive, the use of mixed‐integer programming (MIP) has been proposed to automate it. The goal of this article is to show how constraint programming (CP) can be used as an alternative to automate test‐form generation problems with a large variety of formatting constraints, and how it compares with MIP‐based form generation as for its models, solutions, and running times. Two empirical examples are presented: (i) automated generation of a computerized fixed‐form; and (ii) automated generation of shadow tests for multistage testing. Both examples show that CP works well with feasible solutions and running times likely to be better than that for MIP‐based applications. |
| Author | Li, Jie van der Linden, Wim J. |
| Author_xml | – sequence: 1 givenname: Jie surname: Li fullname: Li, Jie – sequence: 2 givenname: Wim J. surname: van der Linden fullname: van der Linden, Wim J. |
| BackLink | http://eric.ed.gov/ERICWebPortal/detail?accno=EJ1198337$$DView record in ERIC |
| BookMark | eNp9kM1LAzEQxYMoWKsX78KCN2E1s0l2N8dSq1YUPeg5xN1JSekmNUlR_3u3VgVFzGUI7_fm4-2RbecdEnII9BT6dzbHtjuFAupqiwyg4iJnsubbZEBpUeS0FGKX7MU4pxREJWBAZqNs7LulDjZ6l3nT_1xMQVuXsvvgZ0F3nXWzTLs2u7Wv2OZTl3CG4YdqfMhGq-Q7nbDNHjCm_MKHLrtEh0En690-2TF6EfHgsw7J48XkYXyV39xdTsejm7xhXFQ5IsOSaSkoK3kjCpClNLLlNTyBlEWpm7p9Kg2vJW1Ba9bygmtdF5IbgxoMG5LjTd9l8M-rfhE196vg-pGqAE45MC5ZTx1tKAy2UctgOx3e1OQaQNaMVb1ON3oTfIwBjWps-rhjHc1CAVXruNU6bvURd285-WX5avsnDBv4xS7w7R9SXU_Ob788nzvPY_Lh28MF8JpXwN4Bt8qbMw |
| CitedBy_id | crossref_primary_10_1186_s12909_024_05905_1 crossref_primary_10_1016_j_ref_2024_100571 crossref_primary_10_1108_IJQRM_03_2023_0069 |
| Cites_doi | 10.1111/j.1745-3984.1999.tb00557.x 10.1007/978-0-387-85461-8 10.1177/014662169301700206 10.1177/014662169301700205 10.1111/j.1745-3984.2011.00140.x 10.1177/0146621616654597 10.1177/0146621613476157 10.1007/0-387-29054-0 10.1201/b19166 10.1007/0-306-47531-6_6 10.1177/0146621616664075 |
| ContentType | Journal Article |
| Copyright | 2018 National Council on Measurement in Education 2018 by the National Council on Measurement in Education |
| Copyright_xml | – notice: 2018 National Council on Measurement in Education – notice: 2018 by the National Council on Measurement in Education |
| DBID | AAYXX CITATION 7SW BJH BNH BNI BNJ BNO ERI PET REK WWN 7QJ AHOVV |
| DOI | 10.1111/jedm.12187 |
| DatabaseName | CrossRef ERIC ERIC (Ovid) ERIC ERIC ERIC (Legacy Platform) ERIC( SilverPlatter ) ERIC ERIC PlusText (Legacy Platform) Education Resources Information Center (ERIC) ERIC Applied Social Sciences Index & Abstracts (ASSIA) Education Research Index |
| DatabaseTitle | CrossRef ERIC Applied Social Sciences Index and Abstracts (ASSIA) |
| DatabaseTitleList | CrossRef ERIC Applied Social Sciences Index and Abstracts (ASSIA) |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Education |
| EISSN | 1745-3984 |
| ERIC | EJ1198337 |
| EndPage | 456 |
| ExternalDocumentID | EJ1198337 10_1111_jedm_12187 JEDM12187 45148471 |
| Genre | article |
| GroupedDBID | -DZ -W8 -~X .3N .GA .GO 05W 0R~ 10A 1OC 29K 33P 4.4 50Y 50Z 51W 51Y 52M 52O 52Q 52S 52T 52U 52W 5GY 5HH 5LA 5VS 702 7PT 8-0 8-1 8-3 8-4 8-5 85S 8UM 930 A04 AABNI AAESR AAHQN AAHSB AAMMB AAMNL AAONW AAOUF AASGY AAXRX AAYCA AAZKR ABBHK ABCQN ABCUV ABIVO ABJNI ABPPZ ABPVW ABSOO ABXSQ ACAHQ ACBKW ACCZN ACFBH ACGFS ACHQT ACNCT ACPOU ACXQS ADBBV ADEMA ADEOM ADEPB ADIZJ ADMGS ADMHG ADNFJ ADXAS AEFGJ AEGXH AEIGN AEIMD AEUPB AEUYR AEYWJ AFBPY AFEBI AFFPM AFGKR AFKFF AFWVQ AFZJQ AGXDD AHBTC AIDQK AIDYY AIURR ALAGY ALMA_UNASSIGNED_HOLDINGS ALUQN ALVPJ AMBMR AMYDB ASTYK AZBYB AZVAB BAFTC BFHJK BMXJE BNVMJ BQESF BROTX BRXPI BY8 CS3 D-C D-D D-I DCZOG DPXWK DR2 DRFUL DRSSH DU5 EBS EDJ EJD F00 G-S G.N G50 GENNL GODZA HGLYW HZI HZ~ IPSME J0M JAAYA JAV JBMMH JBZCM JENOY JHFFW JKQEH JLEZI JLXEF JPL JST K48 LATKE LC2 LC4 LEEKS LH4 LITHE LOXES LP6 LP7 LUTES LW6 LYRES MEWTI MK4 MRFUL MRSSH MSFUL MSSSH MXFUL MXSSH N04 N06 N9A NF~ O66 O9- OIG P2W P2Y P4C PQQKQ Q.N Q11 QB0 R.K ROL RWL RX1 SA0 SUPJJ TAE TN5 UB1 UBH UPT V8K W8V W99 WBKPD WGMDG WH7 WIH WII WOHZO WQZ WSUWO WXSBR XG1 XSW YQT YZZ ZZTAW ~IA ~WP .GJ .Y3 1OB 31~ 41~ AAAZS AAHHS AANHP AAYJJ AAYOK ABDPE ABEML ABLWH ABOPQ ACBWZ ACCFJ ACRPL ACSCC ACYXJ ADNMO ADULT AEEZP AEQDE AEUQT AFFNX AFPWT AFYRF AIFKG AIWBW AJBDE AJPNJ AQSKT ASPBG AVWKF AZFZN BDRZF CAG COF FEDTE HF~ HGD HVGLF JSODD NHB QZG SAMSI VJK VQA WHG XOL YCJ ZCG ZXP AAYXX ABAWQ ACHJO AGHNM AGQPQ CITATION O8X 7SW BJH BNH BNI BNJ BNO ERI PET REK WWN 7QJ AHOVV |
| ID | FETCH-LOGICAL-c3457-ee3e63a950364c521969f9d481b19926ac8db6f4890d1aa3d424aa8294ffea1f3 |
| IEDL.DBID | DRFUL |
| ISICitedReferencesCount | 3 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000452005600001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0022-0655 |
| IngestDate | Sun Nov 09 13:23:40 EST 2025 Tue Dec 02 16:24:55 EST 2025 Sat Nov 29 03:43:39 EST 2025 Tue Nov 18 20:55:45 EST 2025 Wed Jan 22 17:09:41 EST 2025 Thu Jul 03 21:55:25 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | false |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 4 |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c3457-ee3e63a950364c521969f9d481b19926ac8db6f4890d1aa3d424aa8294ffea1f3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| PQID | 2140413493 |
| PQPubID | 49010 |
| PageCount | 22 |
| ParticipantIDs | proquest_journals_2140413493 eric_primary_EJ1198337 crossref_citationtrail_10_1111_jedm_12187 crossref_primary_10_1111_jedm_12187 wiley_primary_10_1111_jedm_12187_JEDM12187 jstor_primary_45148471 |
| PublicationCentury | 2000 |
| PublicationDate | 2018-12-01 |
| PublicationDateYYYYMMDD | 2018-12-01 |
| PublicationDate_xml | – month: 12 year: 2018 text: 2018-12-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationPlace | Madison |
| PublicationPlace_xml | – name: Madison |
| PublicationSubtitle | Journal of Educational Measurement |
| PublicationTitle | Journal of educational measurement |
| PublicationYear | 2018 |
| Publisher | National Council on Measurement in Education Wiley-Blackwell Wiley Subscription Services, Inc |
| Publisher_xml | – name: National Council on Measurement in Education – name: Wiley-Blackwell – name: Wiley Subscription Services, Inc |
| References | 2013; 37 1993; 17 2000 2010 1999; 36 2016; 40 2006 2017 2016 2005 2011; 48 2014 2013 1999 e_1_2_8_17_1 e_1_2_8_18_1 e_1_2_8_13_1 Apt K. R. (e_1_2_8_2_1) 2013 e_1_2_8_15_1 Nemhauser G. L. (e_1_2_8_8_1) 1999 der Linden W. J. (e_1_2_8_16_1) 2014 der Linden W. J. (e_1_2_8_14_1) 2017 e_1_2_8_3_1 e_1_2_8_5_1 Rossi F. (e_1_2_8_9_1) 2006 e_1_2_8_4_1 e_1_2_8_7_1 e_1_2_8_6_1 e_1_2_8_10_1 e_1_2_8_11_1 e_1_2_8_12_1 |
| References_xml | – volume: 36 start-page: 253 year: 1999 end-page: 266 article-title: Multi‐objective test assembly problems publication-title: Journal of Educational Measurement – year: 2005 – year: 2006 – start-page: 117 year: 2000 end-page: 128 – volume: 37 start-page: 361 year: 2013 end-page: 374 article-title: Integrating test‐form formatting into automated test assembly publication-title: Applied Psychological Measurement – start-page: 101 year: 2014 end-page: 118 – start-page: 31 year: 2010 end-page: 55 – volume: 17 start-page: 151 year: 1993 end-page: 166 article-title: A model and heuristic for solving very large item selection problems publication-title: Applied Psychological Measurement – volume: 40 start-page: 641 year: 2016 end-page: 649 article-title: Comment on three‐element item selection procedures for multiple forms assembly: An item matching approach publication-title: Applied Psychological Measurement – volume: 40 start-page: 641 year: 2016 end-page: 649 article-title: Optimal reassembly of shadow tests in CAT publication-title: Applied Psychological Measurement – year: 2017 – year: 2016 – volume: 17 start-page: 167 year: 1993 end-page: 176 article-title: Application of an automated item selection method to real data publication-title: Applied Psychological Measurement – volume: 48 start-page: 206 year: 2011 end-page: 222 article-title: Automated test‐form generation publication-title: Journal of Educational Measurement – year: 2013 – year: 1999 – ident: e_1_2_8_18_1 doi: 10.1111/j.1745-3984.1999.tb00557.x – volume-title: Principles of constraint programming year: 2013 ident: e_1_2_8_2_1 – ident: e_1_2_8_13_1 doi: 10.1007/978-0-387-85461-8 – volume-title: Handbook of constraint programming: Foundations of artificial intelligence year: 2006 ident: e_1_2_8_9_1 – ident: e_1_2_8_6_1 – ident: e_1_2_8_3_1 – ident: e_1_2_8_10_1 doi: 10.1177/014662169301700206 – ident: e_1_2_8_11_1 doi: 10.1177/014662169301700205 – ident: e_1_2_8_15_1 doi: 10.1111/j.1745-3984.2011.00140.x – ident: e_1_2_8_4_1 doi: 10.1177/0146621616654597 – ident: e_1_2_8_5_1 doi: 10.1177/0146621613476157 – ident: e_1_2_8_12_1 doi: 10.1007/0-387-29054-0 – volume-title: Handbook of item response theory year: 2017 ident: e_1_2_8_14_1 doi: 10.1201/b19166 – start-page: 101 volume-title: Computerized multistage testing: Theory and applications year: 2014 ident: e_1_2_8_16_1 – volume-title: Integer and combinatorial optimization year: 1999 ident: e_1_2_8_8_1 – ident: e_1_2_8_7_1 doi: 10.1007/0-306-47531-6_6 – ident: e_1_2_8_17_1 doi: 10.1177/0146621616664075 |
| SSID | ssj0015751 |
| Score | 2.1649199 |
| Snippet | The final step of the typical process of developing educational and psychological tests is to place the selected test items in a formatted form. The step... |
| SourceID | proquest eric crossref wiley jstor |
| SourceType | Aggregation Database Index Database Enrichment Source Publisher |
| StartPage | 435 |
| SubjectTerms | Alternative approaches Automation Comparative Analysis Computer Assisted Testing Computerization Constraints Educational Development Integer programming Numbers Programming Psychological tests Rule ordering Test Format Test Items Time use |
| Title | A Comparison of Constraint Programming and Mixed-Integer Programming for Automated Test-Form Generation |
| URI | https://www.jstor.org/stable/45148471 https://onlinelibrary.wiley.com/doi/abs/10.1111%2Fjedm.12187 http://eric.ed.gov/ERICWebPortal/detail?accno=EJ1198337 https://www.proquest.com/docview/2140413493 |
| Volume | 55 |
| WOSCitedRecordID | wos000452005600001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVWIB databaseName: Wiley Online Library - Journals customDbUrl: eissn: 1745-3984 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0015751 issn: 0022-0655 databaseCode: DRFUL dateStart: 19970101 isFulltext: true titleUrlDefault: https://onlinelibrary.wiley.com providerName: Wiley-Blackwell |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV3dS8MwED_m5oMvfovzi4C-KBTWJm0T8GW4DREnIgp7K2mTiOg22Yf46J_g3-hfYi7tygYiiG-FXEmTu-v9klx-B3ASmJAaJe0yVQShxwItPB6mzAtkykPOtTGpI3G9jm9ueK8nbitwPrsLk_NDlBtu6Bnuf40OLtPxvJNr1UduBB4vQS2whmutvNa66zxcl6cIeKYwYwu3oTYs6EldJk_59kJAKnKe89TEBdA5D11d7Oms_e-r12G1wJykmRvJBlT0YBPLNRepHVvw3CQXZT1CMjQEy3i64hETcpsncPVtiCNyoEj36V2rr49P3Ep81KOFdguASXM6GVoUrBW5tyOygh2LiknObo29bcNDp31_cekVVRi8jLIw9rSmOqJShHhimdloLyJhhGIW72LqaiQzjlf5GBcN5UtJFQuYlDwQzBgtfUN3oDoYDvQuECkzxSMlTWSRgZ9JLjLGdNpQLNRx1IjrcDpTRZIVFOU42JekXKrY6Uvc9NXhuJR9zYk5fpTaQY2WEu0r3xecUmxwOi5bmEWOGKnrcDBTelK48zgJkIQIiRxpHc6cen_pM7lqt7ruae8vwvuwYsEYz1NlDqA6GU31ISxnb5On8eioMO1vbF79QQ |
| linkProvider | Wiley-Blackwell |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1bS8MwFD7oFPTFuzivAX1RKKxN2iaPQze8bENkgm8laxIRdZM5xUd_gr_RX2JOmhUHIohvhZySJifp-ZJ8-Q7AQWRiapS0y1QRxQGLtAh43GNBJHs85lwb03Mirq200-E3N-LSc3PwLkyhD1FuuOHMcP9rnOC4If19lmv1iOIIPJ2GGWbHEavAzMlV87pVHiPgocJYLtzG2tjrkzoqT_n2RETypOeCmziBOr9jVxd8mov__OwlWPCok9SLYbIMU7q_ggmbPbljFe7r5LjMSEgGhmAiT5c-YkQuCwrXow1yRPYVad-9afX5_oGbibd6OFFuITCpv4wGFgdrRbq2SdawaXExKfStsbY1uG42usengc_DEOSUxWmgNdUJlSLGM8vcxnuRCCMUs4gXyauJzDle5mNc1FQoJVUsYlLySDBjtAwNXYdKf9DXG0CkzBVPlDSJxQZhLrnIGdO9mmKxTpNaWoXDsS-y3IuUY2MfsnKxYrsvc91Xhf3S9qmQ5vjRah1dWlo0zsNQcEqxwDm5LGEWO2KsrsL22OuZn9DPWYQyRCjlSKtw5Pz7S53ZeeOk7Z42_2K8B3On3XYra511LrZg3kIzXhBntqEyGr7oHZjNX0d3z8NdP86_AI8aAUA |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV3RSiMxFL1oFfFldXXFqusG9EVhoDNJZpLHYjvsai1FFHwb0kkiorZSq_joJ-w37pdsbiYdLIiw7NtA7pBJbjL3JDk5F-AwsZxardwyVSY8YomRkeBDFiVqKLgQxtqhF3HtZf2-uL6Wg8DNwbswlT5EveGGM8P_r3GCm0dt389yox9QHEFki7DEuIPiDVjqXORXvfoYAQ8VZnLhLtbyoE_qqTz123MRKZCeK27iHOp8j1198MnX_vOz1-FLQJ2kXQ2Tr7BgRhuYsDmQOzbhrk1O6oyEZGwJJvL06SOmZFBRuB5ckCNqpMn57avRf95-42bijZnMlTsITNrP07HDwUaTS9ckZ5g7XEwqfWus7Rtc5d3Lk59RyMMQlZTxLDKGmpQqyfHMsnTxXqbSSs0c4kXyaqpKgZf5mJAtHStFNUuYUiKRzFqjYku3oDEaj8w2EKVKLVKtbOqwQVwqIUvGzLClGTdZ2sqacDTzRVEGkXJs7H1RL1Zc9xW--5pwUNs-VtIcH1ptoUtri-5pHEtBKRZ4J9clzGFHjNVN2Jt5vQgT-qlIUIYIpRxpE469fz-pszjtds79086_GP-AlUEnL3q_-me7sOqQmah4M3vQmE6ezXdYLl-mt0-T_TDM_wIZuQC7 |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Comparison+of+Constraint+Programming+and+Mixed-Integer+Programming+for+Automated+Test-Form+Generation&rft.jtitle=Journal+of+educational+measurement&rft.au=Li%2C+Jie&rft.au=van+der+Linden%2C+Wim+J.&rft.date=2018-12-01&rft.pub=National+Council+on+Measurement+in+Education&rft.issn=0022-0655&rft.eissn=1745-3984&rft.volume=55&rft.issue=4&rft.spage=435&rft.epage=456&rft_id=info:doi/10.1111%2Fjedm.12187&rft.externalDocID=45148471 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0022-0655&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0022-0655&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0022-0655&client=summon |