A Comparison of Constraint Programming and Mixed-Integer Programming for Automated Test-Form Generation

The final step of the typical process of developing educational and psychological tests is to place the selected test items in a formatted form. The step involves the grouping and ordering of the items to meet a variety of formatting constraints. As this activity tends to be time-intensive, the use...

Full description

Saved in:
Bibliographic Details
Published in:Journal of educational measurement Vol. 55; no. 4; pp. 435 - 456
Main Authors: Li, Jie, van der Linden, Wim J.
Format: Journal Article
Language:English
Published: Madison National Council on Measurement in Education 01.12.2018
Wiley-Blackwell
Wiley Subscription Services, Inc
Subjects:
Alternative approaches
Automation
Comparative Analysis
Computer Assisted Testing
Computerization
Constraints
Educational Development
Integer programming
Numbers
Programming
Psychological tests
Rule ordering
Test Format
Test Items
Time use
ISSN:0022-0655, 1745-3984
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract The final step of the typical process of developing educational and psychological tests is to place the selected test items in a formatted form. The step involves the grouping and ordering of the items to meet a variety of formatting constraints. As this activity tends to be time-intensive, the use of mixed-integer programming (MIP) has been proposed to automate it. The goal of this article is to show how constraint programming (CP) can be used as an alternative to automate test-form generation problems with a large variety of formatting constraints, and how it compares with MIP-based form generation as for its models, solutions, and running times. Two empirical examples are presented: (i) automated generation of a computerized fixedform; and (ii) automated generation of shadow tests for multistage testing. Both examples show that CP works well with feasible solutions and running times likely to be better than that for MIP-based applications.
AbstractList The final step of the typical process of developing educational and psychological tests is to place the selected test items in a formatted form. The step involves the grouping and ordering of the items to meet a variety of formatting constraints. As this activity tends to be time-intensive, the use of mixed-integer programming (MIP) has been proposed to automate it. The goal of this article is to show how constraint programming (CP) can be used as an alternative to automate test-form generation problems with a large variety of formatting constraints, and how it compares with MIP-based form generation as for its models, solutions, and running times. Two empirical examples are presented: (i) automated generation of a computerized fixed-form; and (ii) automated generation of shadow tests for multistage testing. Both examples show that CP works well with feasible solutions and running times likely to be better than that for MIP-based applications.
The final step of the typical process of developing educational and psychological tests is to place the selected test items in a formatted form. The step involves the grouping and ordering of the items to meet a variety of formatting constraints. As this activity tends to be time-intensive, the use of mixed-integer programming (MIP) has been proposed to automate it. The goal of this article is to show how constraint programming (CP) can be used as an alternative to automate test-form generation problems with a large variety of formatting constraints, and how it compares with MIP-based form generation as for its models, solutions, and running times. Two empirical examples are presented: (i) automated generation of a computerized fixedform; and (ii) automated generation of shadow tests for multistage testing. Both examples show that CP works well with feasible solutions and running times likely to be better than that for MIP-based applications.
Author Li, Jie
van der Linden, Wim J.
Author_xml – sequence: 1
  givenname: Jie
  surname: Li
  fullname: Li, Jie
– sequence: 2
  givenname: Wim J.
  surname: van der Linden
  fullname: van der Linden, Wim J.
BackLink http://eric.ed.gov/ERICWebPortal/detail?accno=EJ1198337$$DView record in ERIC
BookMark eNp9kM1LAzEQxYMoWKsX78KCN2E1s0l2N8dSq1YUPeg5xN1JSekmNUlR_3u3VgVFzGUI7_fm4-2RbecdEnII9BT6dzbHtjuFAupqiwyg4iJnsubbZEBpUeS0FGKX7MU4pxREJWBAZqNs7LulDjZ6l3nT_1xMQVuXsvvgZ0F3nXWzTLs2u7Wv2OZTl3CG4YdqfMhGq-Q7nbDNHjCm_MKHLrtEh0En690-2TF6EfHgsw7J48XkYXyV39xdTsejm7xhXFQ5IsOSaSkoK3kjCpClNLLlNTyBlEWpm7p9Kg2vJW1Ba9bygmtdF5IbgxoMG5LjTd9l8M-rfhE196vg-pGqAE45MC5ZTx1tKAy2UctgOx3e1OQaQNaMVb1ON3oTfIwBjWps-rhjHc1CAVXruNU6bvURd285-WX5avsnDBv4xS7w7R9SXU_Ob788nzvPY_Lh28MF8JpXwN4Bt8qbMw
CitedBy_id crossref_primary_10_1186_s12909_024_05905_1
crossref_primary_10_1016_j_ref_2024_100571
crossref_primary_10_1108_IJQRM_03_2023_0069
Cites_doi 10.1111/j.1745-3984.1999.tb00557.x
10.1007/978-0-387-85461-8
10.1177/014662169301700206
10.1177/014662169301700205
10.1111/j.1745-3984.2011.00140.x
10.1177/0146621616654597
10.1177/0146621613476157
10.1007/0-387-29054-0
10.1201/b19166
10.1007/0-306-47531-6_6
10.1177/0146621616664075
ContentType Journal Article
Copyright 2018 National Council on Measurement in Education
2018 by the National Council on Measurement in Education
Copyright_xml – notice: 2018 National Council on Measurement in Education
– notice: 2018 by the National Council on Measurement in Education
DBID AAYXX
CITATION
7SW
BJH
BNH
BNI
BNJ
BNO
ERI
PET
REK
WWN
7QJ
AHOVV
DOI 10.1111/jedm.12187
DatabaseName CrossRef
ERIC
ERIC (Ovid)
ERIC
ERIC
ERIC (Legacy Platform)
ERIC( SilverPlatter )
ERIC
ERIC PlusText (Legacy Platform)
Education Resources Information Center (ERIC)
ERIC
Applied Social Sciences Index & Abstracts (ASSIA)
Education Research Index
DatabaseTitle CrossRef
ERIC
Applied Social Sciences Index and Abstracts (ASSIA)
DatabaseTitleList ERIC
Applied Social Sciences Index and Abstracts (ASSIA)


CrossRef
DeliveryMethod fulltext_linktorsrc
Discipline Education
EISSN 1745-3984
ERIC EJ1198337
EndPage 456
ExternalDocumentID EJ1198337
10_1111_jedm_12187
JEDM12187
45148471
Genre article
GroupedDBID -DZ
-W8
-~X
.3N
.GA
.GO
05W
0R~
10A
1OC
29K
33P
4.4
50Y
50Z
51W
51Y
52M
52O
52Q
52S
52T
52U
52W
5GY
5HH
5LA
5VS
702
7PT
8-0
8-1
8-3
8-4
8-5
85S
8UM
930
A04
AABNI
AAESR
AAHQN
AAHSB
AAMMB
AAMNL
AAONW
AAOUF
AASGY
AAXRX
AAYCA
AAZKR
ABBHK
ABCQN
ABCUV
ABIVO
ABJNI
ABPPZ
ABPVW
ABSOO
ABXSQ
ACAHQ
ACBKW
ACCZN
ACFBH
ACGFS
ACHQT
ACNCT
ACPOU
ACXQS
ADBBV
ADEMA
ADEOM
ADEPB
ADIZJ
ADMGS
ADMHG
ADNFJ
ADXAS
AEFGJ
AEGXH
AEIGN
AEIMD
AEUPB
AEUYR
AEYWJ
AFBPY
AFEBI
AFFPM
AFGKR
AFKFF
AFWVQ
AFZJQ
AGXDD
AHBTC
AIDQK
AIDYY
AIURR
ALAGY
ALMA_UNASSIGNED_HOLDINGS
ALUQN
ALVPJ
AMBMR
AMYDB
ASTYK
AZBYB
AZVAB
BAFTC
BFHJK
BMXJE
BNVMJ
BQESF
BROTX
BRXPI
BY8
CS3
D-C
D-D
D-I
DCZOG
DPXWK
DR2
DRFUL
DRSSH
DU5
EBS
EDJ
EJD
F00
G-S
G.N
G50
GENNL
GODZA
HGLYW
HZI
HZ~
IPSME
J0M
JAAYA
JAV
JBMMH
JBZCM
JENOY
JHFFW
JKQEH
JLEZI
JLXEF
JPL
JST
K48
LATKE
LC2
LC4
LEEKS
LH4
LITHE
LOXES
LP6
LP7
LUTES
LW6
LYRES
MEWTI
MK4
MRFUL
MRSSH
MSFUL
MSSSH
MXFUL
MXSSH
N04
N06
N9A
NF~
O66
O9-
OIG
P2W
P2Y
P4C
PQQKQ
Q.N
Q11
QB0
R.K
ROL
RWL
RX1
SA0
SUPJJ
TAE
TN5
UB1
UBH
UPT
V8K
W8V
W99
WBKPD
WGMDG
WH7
WIH
WII
WOHZO
WQZ
WSUWO
WXSBR
XG1
XSW
YQT
YZZ
ZZTAW
~IA
~WP
.GJ
.Y3
1OB
31~
41~
AAAZS
AAHHS
AANHP
AAYJJ
AAYOK
ABDPE
ABEML
ABLWH
ABOPQ
ACBWZ
ACCFJ
ACRPL
ACSCC
ACYXJ
ADNMO
ADULT
AEEZP
AEQDE
AEUQT
AFFNX
AFPWT
AFYRF
AIFKG
AIWBW
AJBDE
AJPNJ
AQSKT
ASPBG
AVWKF
AZFZN
BDRZF
CAG
COF
FEDTE
HF~
HGD
HVGLF
JSODD
NHB
QZG
SAMSI
VJK
VQA
WHG
XOL
YCJ
ZCG
ZXP
AAYXX
ABAWQ
ACHJO
AGHNM
AGQPQ
CITATION
O8X
7SW
BJH
BNH
BNI
BNJ
BNO
ERI
PET
REK
WWN
7QJ
AHOVV
ID FETCH-LOGICAL-c3457-ee3e63a950364c521969f9d481b19926ac8db6f4890d1aa3d424aa8294ffea1f3
IEDL.DBID DRFUL
ISICitedReferencesCount 3
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000452005600001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 0022-0655
IngestDate Sun Nov 09 13:23:40 EST 2025
Tue Dec 02 16:24:55 EST 2025
Sat Nov 29 03:43:39 EST 2025
Tue Nov 18 20:55:45 EST 2025
Wed Jan 22 17:09:41 EST 2025
Thu Jul 03 21:55:25 EDT 2025
IsDoiOpenAccess false
IsOpenAccess false
IsPeerReviewed true
IsScholarly true
Issue 4
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c3457-ee3e63a950364c521969f9d481b19926ac8db6f4890d1aa3d424aa8294ffea1f3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
PQID 2140413493
PQPubID 49010
PageCount 22
ParticipantIDs proquest_journals_2140413493
eric_primary_EJ1198337
crossref_citationtrail_10_1111_jedm_12187
crossref_primary_10_1111_jedm_12187
wiley_primary_10_1111_jedm_12187_JEDM12187
jstor_primary_45148471
PublicationCentury 2000
PublicationDate 2018-12-01
PublicationDateYYYYMMDD 2018-12-01
PublicationDate_xml – month: 12
  year: 2018
  text: 2018-12-01
  day: 01
PublicationDecade 2010
PublicationPlace Madison
PublicationPlace_xml – name: Madison
PublicationSubtitle Journal of Educational Measurement
PublicationTitle Journal of educational measurement
PublicationYear 2018
Publisher National Council on Measurement in Education
Wiley-Blackwell
Wiley Subscription Services, Inc
Publisher_xml – name: National Council on Measurement in Education
– name: Wiley-Blackwell
– name: Wiley Subscription Services, Inc
References 2013; 37
1993; 17
2000
2010
1999; 36
2016; 40
2006
2017
2016
2005
2011; 48
2014
2013
1999
e_1_2_8_17_1
e_1_2_8_18_1
e_1_2_8_13_1
Apt K. R. (e_1_2_8_2_1) 2013
e_1_2_8_15_1
Nemhauser G. L. (e_1_2_8_8_1) 1999
der Linden W. J. (e_1_2_8_16_1) 2014
der Linden W. J. (e_1_2_8_14_1) 2017
e_1_2_8_3_1
e_1_2_8_5_1
Rossi F. (e_1_2_8_9_1) 2006
e_1_2_8_4_1
e_1_2_8_7_1
e_1_2_8_6_1
e_1_2_8_10_1
e_1_2_8_11_1
e_1_2_8_12_1
References_xml – volume: 36
  start-page: 253
  year: 1999
  end-page: 266
  article-title: Multi‐objective test assembly problems
  publication-title: Journal of Educational Measurement
– year: 2005
– year: 2006
– start-page: 117
  year: 2000
  end-page: 128
– volume: 37
  start-page: 361
  year: 2013
  end-page: 374
  article-title: Integrating test‐form formatting into automated test assembly
  publication-title: Applied Psychological Measurement
– start-page: 101
  year: 2014
  end-page: 118
– start-page: 31
  year: 2010
  end-page: 55
– volume: 17
  start-page: 151
  year: 1993
  end-page: 166
  article-title: A model and heuristic for solving very large item selection problems
  publication-title: Applied Psychological Measurement
– volume: 40
  start-page: 641
  year: 2016
  end-page: 649
  article-title: Comment on three‐element item selection procedures for multiple forms assembly: An item matching approach
  publication-title: Applied Psychological Measurement
– volume: 40
  start-page: 641
  year: 2016
  end-page: 649
  article-title: Optimal reassembly of shadow tests in CAT
  publication-title: Applied Psychological Measurement
– year: 2017
– year: 2016
– volume: 17
  start-page: 167
  year: 1993
  end-page: 176
  article-title: Application of an automated item selection method to real data
  publication-title: Applied Psychological Measurement
– volume: 48
  start-page: 206
  year: 2011
  end-page: 222
  article-title: Automated test‐form generation
  publication-title: Journal of Educational Measurement
– year: 2013
– year: 1999
– ident: e_1_2_8_18_1
  doi: 10.1111/j.1745-3984.1999.tb00557.x
– volume-title: Principles of constraint programming
  year: 2013
  ident: e_1_2_8_2_1
– ident: e_1_2_8_13_1
  doi: 10.1007/978-0-387-85461-8
– volume-title: Handbook of constraint programming: Foundations of artificial intelligence
  year: 2006
  ident: e_1_2_8_9_1
– ident: e_1_2_8_6_1
– ident: e_1_2_8_3_1
– ident: e_1_2_8_10_1
  doi: 10.1177/014662169301700206
– ident: e_1_2_8_11_1
  doi: 10.1177/014662169301700205
– ident: e_1_2_8_15_1
  doi: 10.1111/j.1745-3984.2011.00140.x
– ident: e_1_2_8_4_1
  doi: 10.1177/0146621616654597
– ident: e_1_2_8_5_1
  doi: 10.1177/0146621613476157
– ident: e_1_2_8_12_1
  doi: 10.1007/0-387-29054-0
– volume-title: Handbook of item response theory
  year: 2017
  ident: e_1_2_8_14_1
  doi: 10.1201/b19166
– start-page: 101
  volume-title: Computerized multistage testing: Theory and applications
  year: 2014
  ident: e_1_2_8_16_1
– volume-title: Integer and combinatorial optimization
  year: 1999
  ident: e_1_2_8_8_1
– ident: e_1_2_8_7_1
  doi: 10.1007/0-306-47531-6_6
– ident: e_1_2_8_17_1
  doi: 10.1177/0146621616664075
SSID ssj0015751
Score 2.1648157
Snippet The final step of the typical process of developing educational and psychological tests is to place the selected test items in a formatted form. The step...
SourceID proquest
eric
crossref
wiley
jstor
SourceType Aggregation Database
Index Database
Enrichment Source
Publisher
StartPage 435
SubjectTerms Alternative approaches
Automation
Comparative Analysis
Computer Assisted Testing
Computerization
Constraints
Educational Development
Integer programming
Numbers
Programming
Psychological tests
Rule ordering
Test Format
Test Items
Time use
Title A Comparison of Constraint Programming and Mixed-Integer Programming for Automated Test-Form Generation
URI https://www.jstor.org/stable/45148471
https://onlinelibrary.wiley.com/doi/abs/10.1111%2Fjedm.12187
http://eric.ed.gov/ERICWebPortal/detail?accno=EJ1198337
https://www.proquest.com/docview/2140413493
Volume 55
WOSCitedRecordID wos000452005600001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVWIB
  databaseName: Wiley Online Library Full Collection 2020
  customDbUrl:
  eissn: 1745-3984
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0015751
  issn: 0022-0655
  databaseCode: DRFUL
  dateStart: 19970101
  isFulltext: true
  titleUrlDefault: https://onlinelibrary.wiley.com
  providerName: Wiley-Blackwell
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LSwMxEB5q68GL72K1SkAvCgvtJvsIeCl9IKUtRVrobcluslK0rfQhHv0J_kZ_iZnsdmlBBPG2kFmyyWR2vmQm3wDcOEzEruDcorbSGxRjUtRVVkXaoRDSt8OkaknH6_X80Yj3c3C_vguT8ENkB25oGeZ_jQYuwsWmkSs5QW4E39uBAt6q0luvQuOxNexkUQSMKazZwrWrdVJ6UpPJk7295ZDSnOckNXELdG5CV-N7Wgf_--pD2E8xJ6kli-QIcmp6jOWa09SOE3iukXpWj5DMYoJlPE3xiCXpJwlcE-3iiJhK0h2_K_n18YlHiU9qvtWuATCprZYzjYKVJAM9Ii3Y0qiYJOzW2NspDFvNQf3BSqswWBFljmcpRZVLBXcwYhlpb89dHnPJNN7F1FVXRD5e5WM-r8iqEFQymwnh25zFsRLVmBYhP51N1RkQDNqx0FGR50TMVTKkticqTIVSVGhIvRLcrlURRClFOQ72Jci2Knr6AjN9JbjOZF8TYo4fpYqo0Uyi2a5WuU-xq6LRcdbCNHJET12C8lrpQWrOi8BGEiIkcqQluDPq_aXPoN1sdM3T-V-EL2BPgzE_SZUpQ345X6lL2I3eluPF_Cpd2t9vuvzW
linkProvider Wiley-Blackwell
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV3dS8MwED90Cvri93B-BvRFobA1aZs8jrnhxzZEJvhW0iaVoW4yN_HRP8G_0b_EXJoVByKIb4VcSZPL9X7JXX4HcBwwmYVSCI_62mxQrEnRUHtV5SdSKu4nedWSdtTt8rs7ce1yc_AuTM4PURy4oWXY_zUaOB5If7dyrZ6QHIFH87DAQhrxEiyc3bRu20UYAYMKU7pw42sDx09qU3mKt2c8kkt6znMTZ1Dnd-xqnU9r9Z-fvQYrDnWSer5M1mFODzawYLNL7tiEhzppFBUJyTAjWMjTlo8Yk-s8hevJODkiB4p0-m9afb5_4GHivR7NtBsITOqT8dDgYK1IzwzJCLYMLiY5vzX2tgW3rWavce65OgxeSlkQeVpTHVIpAoxZpsbfi1BkQjGDeDF5NZQpx8t8jIuqqklJFfOZlNwXLMu0rGW0DKXBcKC3gWDYjiWBTqMgZaFWCfUjWWU6UbJKExpV4GSqizh1JOU42Me42KyY6Yvt9FXgqJB9zqk5fpQqo0oLieZlrSY4xa7KVslFCzPYEX11BfamWo-dQb_EPtIQIZUjrcCp1e8vfcaXzbOOfdr5i_AhLJ33Ou24fdG92oVlA814njizB6XxaKL3YTF9HfdfRgdunX8BjQMA1Q
linkToPdf http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1fSyMxEB_OKuKLnncWe-oZ8F48WGg32T95LLaL9mopouDbkt3MSrnrH2o97vE-gp_RT2Immy4tHAfi20JmySaT2fklM_kNwLdAqCJUUnrcR7NBsSbFQ_Sa2s-U0rGflVVL-tFgEN_fy6HLzaG7MCU_RHXgRpZh_9dk4DjTxaqVox4TOUIcbcCmCGQgarDZuUnu-lUYgYIKS7pw42sDx09qU3mqt9c8kkt6LnMT11DnKna1zifZe-dnf4RdhzpZu1wm-_ABJ5-oYLNL7vgMP9vsoqpIyKYFo0KetnzEgg3LFK6xcXJMTTS7Hv1B_fL3mQ4TH3C-1m4gMGs_LaYGB6Nmt2ZIRjAxuJiV_NbU2wHcJd3bi0vP1WHwci6CyEPkGHIlA4pZ5sbfy1AWUguDeCl5NVR5TJf5RCybuqUU18IXSsW-FEWBqlXwOtQm0wkeAqOwncgCzKMgFyHqjPuRagrMtGryjEcNOF_qIs0dSTkN9ldabVbM9KV2-hpwVsnOSmqOf0rVSaWVRLfXasmYU1d1q-SqRRjsSL66AcdLrafOoB9Tn2iIiMqRN-C71e9_-kx73c61ffryFuFT2B52krR_NfhxBDsGmcVl3swx1BbzJzyBrfz3YvQ4_-qW-SsYDQBQ
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Comparison+of+Constraint+Programming+and+Mixed%E2%80%90Integer+Programming+for+Automated+Test%E2%80%90Form+Generation&rft.jtitle=Journal+of+educational+measurement&rft.au=Li%2C+Jie&rft.au=Wim+J+van+der+Linden&rft.date=2018-12-01&rft.pub=Wiley+Subscription+Services%2C+Inc&rft.issn=0022-0655&rft.eissn=1745-3984&rft.volume=55&rft.issue=4&rft.spage=435&rft.epage=456&rft_id=info:doi/10.1111%2Fjedm.12187&rft.externalDBID=NO_FULL_TEXT
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0022-0655&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0022-0655&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0022-0655&client=summon