Profiling Programming Language Learning

This paper documents a year-long experiment to “profile” the process of learning a programming language: gathering data to understand what makes a language hard to learn, and using that data to improve the learning process. We added interactive quizzes to The Rust Programming Language, the official...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings of ACM on programming languages Ročník 8; číslo OOPSLA1; s. 29 - 54
Hlavní autoři: Crichton, Will, Krishnamurthi, Shriram
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York, NY, USA ACM 29.04.2024
Témata:
ISSN:2475-1421, 2475-1421
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Abstract This paper documents a year-long experiment to “profile” the process of learning a programming language: gathering data to understand what makes a language hard to learn, and using that data to improve the learning process. We added interactive quizzes to The Rust Programming Language, the official textbook for learning Rust. Over 13 months, 62,526 readers answered questions 1,140,202 times. First, we analyze the trajectories of readers. We find that many readers drop-out of the book early when faced with difficult language concepts like Rust’s ownership types. Second, we use classical test theory and item response theory to analyze the characteristics of quiz questions. We find that better questions are more conceptual in nature, such as asking why a program does not compile vs. whether a program compiles. Third, we performed 12 interventions into the book to help readers with difficult questions. We find that on average, interventions improved quiz scores on the targeted questions by +20%. Fourth, we show that our technique can likely generalize to languages with smaller user bases by simulating our statistical inferences on small N. These results demonstrate that quizzes are a simple and useful technique for understanding language learning at all scales.
AbstractList This paper documents a year-long experiment to “profile” the process of learning a programming language: gathering data to understand what makes a language hard to learn, and using that data to improve the learning process. We added interactive quizzes to The Rust Programming Language, the official textbook for learning Rust. Over 13 months, 62,526 readers answered questions 1,140,202 times. First, we analyze the trajectories of readers. We find that many readers drop-out of the book early when faced with difficult language concepts like Rust’s ownership types. Second, we use classical test theory and item response theory to analyze the characteristics of quiz questions. We find that better questions are more conceptual in nature, such as asking why a program does not compile vs. whether a program compiles. Third, we performed 12 interventions into the book to help readers with difficult questions. We find that on average, interventions improved quiz scores on the targeted questions by +20
This paper documents a year-long experiment to “profile” the process of learning a programming language: gathering data to understand what makes a language hard to learn, and using that data to improve the learning process. We added interactive quizzes to The Rust Programming Language, the official textbook for learning Rust. Over 13 months, 62,526 readers answered questions 1,140,202 times. First, we analyze the trajectories of readers. We find that many readers drop-out of the book early when faced with difficult language concepts like Rust’s ownership types. Second, we use classical test theory and item response theory to analyze the characteristics of quiz questions. We find that better questions are more conceptual in nature, such as asking why a program does not compile vs. whether a program compiles. Third, we performed 12 interventions into the book to help readers with difficult questions. We find that on average, interventions improved quiz scores on the targeted questions by +20%. Fourth, we show that our technique can likely generalize to languages with smaller user bases by simulating our statistical inferences on small N. These results demonstrate that quizzes are a simple and useful technique for understanding language learning at all scales.
ArticleNumber 95
Author Crichton, Will
Krishnamurthi, Shriram
Author_xml – sequence: 1
  givenname: Will
  orcidid: 0000-0001-8639-6541
  surname: Crichton
  fullname: Crichton, Will
  email: wcrichto@brown.edu
  organization: Brown University, Providence, USA
– sequence: 2
  givenname: Shriram
  orcidid: 0000-0001-5184-1975
  surname: Krishnamurthi
  fullname: Krishnamurthi, Shriram
  email: shriram@gmail.com
  organization: Brown University, Providence, USA
BookMark eNpNj01Lw0AQhhepYK3Fu6feeoru7Fd2j1L8goA91HOYzc6GSJPIrh7896a0inOZh3ceBt5LNhvGgRi7Bn4LoPSdNMpZEGdsLlSpC1ACZv_4gi1zfuecg5PKSjdn620aY7fvhnY1UZuw7w9c4dB-YUurijANU3LFziPuMy1Pe8HeHh92m-eien162dxXBQrpPgvgnqwPpaImQADQIVi0GP00ofQgPDiDhETWaqmUjtZE4dAIZww6kAu2Pv5t0phzolh_pK7H9F0Drw8V61PFybw5mtj0f9Lv8Qe6RUxe
Cites_doi 10.1145/3291279.3339404
10.1145/3328778.3366950
10.1111/j.1745-3984.1988.tb00288.x 10.1111/j.1745-3984.1988.tb00288.x
10.1037/h0024117
10.1111/j.2517-6161.1995.tb02031.x 10.1111/j.2517-6161.1995.tb02031.x
10.1145/3287324.3287370
10.3102/00346543049002280
10.1207/S1532690XCI2103_01
10.1145/3478431.3499354
10.4230/OASIcs.PLATEAU.2018.5
10.5281/zenodo.10798571
10.19173/irrodl.v15i1.1651
10.1145/2509136.2509515
10.18653/v1/D19-1434
10.1145/3510003.3510164
10.1145/3133850.3133863
10.1016/j.compedu.2022.104562
10.1145/3622841
10.1079/searchRxiv.2022.00005
10.1111/j.2517-6161.1995.tb02031.x
10.1111/j.1745-3984.1988.tb00288.x
10.18653/v1
ContentType Journal Article
Copyright Owner/Author
Copyright_xml – notice: Owner/Author
DBID AAYXX
CITATION
DOI 10.1145/3649812
DatabaseName CrossRef
DatabaseTitle CrossRef
DatabaseTitleList CrossRef

DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 2475-1421
EndPage 54
ExternalDocumentID 10_1145_3649812
3649812
GrantInformation_xml – fundername: National Science Foundation
  grantid: 2319014
  funderid: https:\/\/doi.org\/10.13039\/100000001
GroupedDBID AAKMM
AAYFX
ACM
AEFXT
AEJOY
AIKLT
AKRVB
ALMA_UNASSIGNED_HOLDINGS
GUFHI
LHSKQ
M~E
OK1
ROL
AAYXX
CITATION
ID FETCH-LOGICAL-a239t-10be8bd74ecd1d115dd8a8afbbbbd7b12b196aeaee8853445f86f29a62966a913
ISICitedReferencesCount 0
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001209927600002&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 2475-1421
IngestDate Sat Nov 29 07:54:36 EST 2025
Mon Jul 07 16:40:28 EDT 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue OOPSLA1
Keywords rust education
digital textbooks
item response theory
Language English
License This work is licensed under a Creative Commons Attribution International 4.0 License.
LinkModel OpenURL
MergedId FETCHMERGED-LOGICAL-a239t-10be8bd74ecd1d115dd8a8afbbbbd7b12b196aeaee8853445f86f29a62966a913
ORCID 0000-0001-8639-6541
0000-0001-5184-1975
OpenAccessLink https://dl.acm.org/doi/10.1145/3649812
PageCount 26
ParticipantIDs crossref_primary_10_1145_3649812
acm_primary_3649812
PublicationCentury 2000
PublicationDate 2024-04-29
PublicationDateYYYYMMDD 2024-04-29
PublicationDate_xml – month: 04
  year: 2024
  text: 2024-04-29
  day: 29
PublicationDecade 2020
PublicationPlace New York, NY, USA
PublicationPlace_xml – name: New York, NY, USA
PublicationTitle Proceedings of ACM on programming languages
PublicationTitleAbbrev ACM PACMPL
PublicationYear 2024
Publisher ACM
Publisher_xml – name: ACM
References (bib24) 1994
(bib29) 2022
(bib19) 1988; 25
(bib14) 2022
(bib25) 2019
(bib35) 2019
(bib6) 2012; 2
(bib12) 2021
(bib18) 2019
(bib26) 2022
(bib1) 1979; 49
(bib11) 2022
(bib21) 2023
(bib16) 2022
(bib36) 2022
(bib8) 2024
(bib15) 2014; 15
(bib4) 1995; 57
(bib5) 2019; 20
(bib28) 1967; 58
(bib3) 2001
(bib31) 2020
(bib20) 2017
(bib23) 2022
(bib34) 2019
(bib13) 2023
(bib9) 2022
(bib22) 2013
(bib32) 2017
(bib33) 2003; 21
(bib7) 2023; 7
(bib27) 2011
(bib10) 2020
(bib2) 2023
(bib17) 2022; 187
(bib30) 2023
Xie Benjamin (e_1_2_1_34_1)
Overflow Stack (e_1_2_1_30_1) 2023
Ericson Barbara J. (e_1_2_1_9_1) 2022; 1
e_1_2_1_20_1
Braun Virginia (e_1_2_1_6_1)
e_1_2_1_21_1
e_1_2_1_28_1
e_1_2_1_26_1
Baker Frank B. (e_1_2_1_3_1)
Raykov Tenko (e_1_2_1_27_1)
e_1_2_1_29_1
Crichton Will (e_1_2_1_7_1) 2023
Barbara (e_1_2_1_10_1)
Foundation Caml Software (e_1_2_1_23_1) 2022
David (e_1_2_1_24_1) 1994
Zhu Shuofei (e_1_2_1_36_1) 2022
Wilson Preston Tunnell (e_1_2_1_32_1) 2017
Fulton Kelsey R. (e_1_2_1_12_1) 2021
Bingham Eli (e_1_2_1_5_1) 2019; 20
(e_1_2_1_14_1) 2022
e_1_2_1_31_1
e_1_2_1_8_1
e_1_2_1_4_1
e_1_2_1_13_1
e_1_2_1_1_1
Leo (e_1_2_1_22_1)
e_1_2_1_33_1
e_1_2_1_2_1
e_1_2_1_11_1
e_1_2_1_16_1
Porter Leo (e_1_2_1_25_1) 2019
e_1_2_1_17_1
e_1_2_1_15_1
e_1_2_1_18_1
e_1_2_1_19_1
Zeng Anna (e_1_2_1_35_1) 2019; 6
References_xml – year: 2022
  ident: bib16
  publication-title: The Rust Programming Language
– start-page: 111
  year: 2019
  end-page: 119
  ident: bib25
  publication-title: Proceedings of the 2019 ACM Conference on International Computing Education Research (Toronto ON, Canada) (ICER ’19)
  doi: 10.1145/3291279.3339404
– year: 2022
  ident: bib11
  publication-title: 2022 State of Haskell Survey Results
– start-page: 1012
  year: 2020
  end-page: 1018
  ident: bib10
  publication-title: Proceedings of the 51st ACM Technical Symposium on Computer Science Education (Portland, OR, USA) (SIGCSE ’20)
  doi: 10.1145/3328778.3366950
– volume: 25
  start-page: 15
  issue: 1
  year: 1988
  end-page: 29
  ident: bib19
  article-title: Item Discrimination: When More Is Worse
  publication-title: Journal of Educational Measurement
  doi: 10.1111/j.1745-3984.1988.tb00288.x 10.1111/j.1745-3984.1988.tb00288.x
– volume: 58
  start-page: 56
  issue: 1
  year: 1967
  end-page: 61
  ident: bib28
  article-title: Selective facilitative effects of interspersed questions on learning from written materials
  publication-title: Journal of Educational Psychology
  doi: 10.1037/h0024117
– year: 2020
  ident: bib31
  publication-title: Rust Survey 2020 Results
– year: 2001
  ident: bib3
  publication-title: The Basics of Item Response Theory
– volume: 2
  year: 2012
  ident: bib6
  publication-title: APA Handbook of Research Methods in Psychology
– volume: 57
  start-page: 289
  issue: 1
  year: 1995
  end-page: 300
  ident: bib4
  article-title: Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
  publication-title: Journal of the Royal Statistical Society: Series B (Methodological)
  doi: 10.1111/j.2517-6161.1995.tb02031.x 10.1111/j.2517-6161.1995.tb02031.x
– year: 2022
  ident: bib14
  publication-title: The State of Developer Ecosystem 2022
– start-page: 699
  year: 2019
  end-page: 705
  ident: bib34
  publication-title: Proceedings of the 50th ACM Technical Symposium on Computer Science Education (Minneapolis, MN, USA) (SIGCSE ’19)
  doi: 10.1145/3287324.3287370
– volume: 49
  start-page: 280
  issue: 2
  year: 1979
  end-page: 318
  ident: bib1
  article-title: Does Answering Higher-Level Questions While Reading Facilitate Productive Learning?
  publication-title: Review of Educational Research
  doi: 10.3102/00346543049002280
– volume: 21
  start-page: 209
  issue: 3
  year: 2003
  end-page: 249
  ident: bib33
  article-title: Why Do Only Some Events Cause Learning During Human Tutoring?
  publication-title: Cognition and Instruction
  doi: 10.1207/S1532690XCI2103_01
– start-page: 418
  year: 2022
  end-page: 424
  ident: bib9
  publication-title: Proceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 1 (Providence, RI, USA) (SIGCSE 2022)
  doi: 10.1145/3478431.3499354
– volume: 20
  start-page: 1
  issue: 28
  year: 2019
  end-page: 6
  ident: bib5
  article-title: Pyro: Deep Universal Probabilistic Programming
  publication-title: Journal of Machine Learning Research
– year: 2023
  ident: bib2
  publication-title: Theorem Proving in Lean 4
– year: 1994
  ident: bib24
  publication-title: International Encyclopedia of Education
– start-page: 5:1
  year: 2019
  end-page: 5:6
  ident: bib35
  publication-title: 9th Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU 2018) (OpenAccess Series in Informatics (OASIcs), Vol. 67)
  doi: 10.4230/OASIcs.PLATEAU.2018.5
– year: 2022
  ident: bib26
  publication-title: State of Clojure 2022 Results
– year: 2022
  ident: bib23
  publication-title: OCaml Users Survey 2022
– year: 2011
  ident: bib27
  publication-title: Introduction to psychometric theory
– year: 2017
  ident: bib20
  publication-title: Non-lexical lifetimes
– year: 2024
  ident: bib8
  publication-title: Artifact for “Profiling Programming Language Learning”
  doi: 10.5281/zenodo.10798571
– volume: 15
  start-page: 133
  issue: 1
  year: 2014
  end-page: 160
  ident: bib15
  article-title: Initial Trends in Enrolment and Completion of Massive Open Online Courses
  publication-title: International Review of Research in Open and Distributed Learning
  doi: 10.19173/irrodl.v15i1.1651
– start-page: 1
  year: 2013
  end-page: 18
  ident: bib22
  publication-title: Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages &t Applications (Indianapolis, Indiana, USA) (OOPSLA ’13)
  doi: 10.1145/2509136.2509515
– start-page: 4249
  year: 2019
  end-page: 4259
  ident: bib18
  publication-title: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
  doi: 10.18653/v1/D19-1434
– start-page: 1269
  year: 2022
  end-page: 1281
  ident: bib36
  publication-title: Proceedings of the 44th International Conference on Software Engineering (Pittsburgh, Pennsylvania) (ICSE ’22)
  doi: 10.1145/3510003.3510164
– start-page: 1
  year: 2017
  end-page: 17
  ident: bib32
  publication-title: Proceedings of the 2017 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Vancouver, BC, Canada) (Onward! 2017)
  doi: 10.1145/3133850.3133863
– year: 2023
  ident: bib21
  publication-title: Go Developer Survey 2023 Q1 Results
– year: 2023
  ident: bib30
  publication-title: Stack Overflow Developer Survey 2023
– volume: 187
  year: 2022
  ident: bib17
  article-title: Validating a computational thinking concepts test for primary education using item response theory: An analysis of students' responses
  publication-title: Computers & Education
  doi: 10.1016/j.compedu.2022.104562
– volume: 7
  start-page: 29
  year: 2023
  ident: bib7
  article-title: A Grounded Conceptual Model for Ownership Types in Rust
  publication-title: Proc. ACM Program. Lang
  doi: 10.1145/3622841
– year: 2023
  ident: bib13
  publication-title: Comprehensive Rust
– year: 2022
  ident: bib29
  publication-title: Scala Developer Survey 2022 Results
– start-page: 597
  year: 2021
  end-page: 616
  ident: bib12
  publication-title: Seventeenth Symposium on Usable Privacy and Security (SOUPS 2021)
– ident: e_1_2_1_11_1
  doi: 10.1079/searchRxiv.2022.00005
– ident: e_1_2_1_15_1
  doi: 10.19173/irrodl.v15i1.1651
– volume-title: Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages &t Applications (OOPSLA ’13)
  ident: e_1_2_1_22_1
– ident: e_1_2_1_21_1
– volume-title: Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE ’19)
  ident: e_1_2_1_34_1
– volume: 6
  volume-title: 9th Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU 2018
  year: 2019
  ident: e_1_2_1_35_1
– volume-title: The State of Developer Ecosystem
  year: 2022
  ident: e_1_2_1_14_1
– volume-title: Seventeenth Symposium on Usable Privacy and Security (SOUPS 2021
  year: 2021
  ident: e_1_2_1_12_1
– ident: e_1_2_1_16_1
– ident: e_1_2_1_4_1
  doi: 10.1111/j.2517-6161.1995.tb02031.x
– ident: e_1_2_1_26_1
– ident: e_1_2_1_20_1
– ident: e_1_2_1_29_1
– volume: 20
  start-page: 1
  year: 2019
  ident: e_1_2_1_5_1
  article-title: Pyro: Deep Universal Probabilistic Programming
  publication-title: Journal of Machine Learning Research
– ident: e_1_2_1_31_1
– ident: e_1_2_1_8_1
  doi: 10.5281/zenodo.10798571
– volume-title: Proceedings of the 2017 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward!
  year: 2017
  ident: e_1_2_1_32_1
– ident: e_1_2_1_33_1
  doi: 10.1207/S1532690XCI2103_01
– volume: 1
  volume-title: Proceedings of the 53rd ACM Technical Symposium on Computer Science Education -
  year: 2022
  ident: e_1_2_1_9_1
– volume-title: APA Handbook of Research Methods in Psychology. 2
  ident: e_1_2_1_6_1
– volume-title: Proceedings of the 2019 ACM Conference on International Computing Education Research (ICER ’19)
  year: 2019
  ident: e_1_2_1_25_1
– volume-title: OCaml Users Survey
  year: 2022
  ident: e_1_2_1_23_1
– volume-title: Stack Overflow Developer Survey
  year: 2023
  ident: e_1_2_1_30_1
– volume-title: Proc. ACM Program. Lang., 7, OOPSLA2
  year: 2023
  ident: e_1_2_1_7_1
– volume-title: The Basics of Item Response Theory (2 ed.)
  ident: e_1_2_1_3_1
– ident: e_1_2_1_19_1
  doi: 10.1111/j.1745-3984.1988.tb00288.x
– ident: e_1_2_1_2_1
– ident: e_1_2_1_18_1
  doi: 10.18653/v1
– volume-title: Perkins and Gavriel Salomon
  year: 1994
  ident: e_1_2_1_24_1
– ident: e_1_2_1_17_1
  doi: 10.1016/j.compedu.2022.104562
– volume-title: Proceedings of the 44th International Conference on Software Engineering (ICSE ’22)
  year: 2022
  ident: e_1_2_1_36_1
– ident: e_1_2_1_28_1
  doi: 10.1037/h0024117
– ident: e_1_2_1_13_1
– volume-title: Introduction to psychometric theory
  ident: e_1_2_1_27_1
– volume-title: Proceedings of the 51st ACM Technical Symposium on Computer Science Education (SIGCSE ’20)
  ident: e_1_2_1_10_1
– ident: e_1_2_1_1_1
  doi: 10.3102/00346543049002280
SSID ssj0001934839
Score 2.268313
Snippet This paper documents a year-long experiment to “profile” the process of learning a programming language: gathering data to understand what makes a language...
SourceID crossref
acm
SourceType Index Database
Publisher
StartPage 29
SubjectTerms Applied computing
E-learning
General programming languages
Software and its engineering
SubjectTermsDisplay Applied computing -- E-learning
Software and its engineering -- General programming languages
Title Profiling Programming Language Learning
URI https://dl.acm.org/doi/10.1145/3649812
Volume 8
WOSCitedRecordID wos001209927600002&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVHPJ
  databaseName: ROAD: Directory of Open Access Scholarly Resources
  customDbUrl:
  eissn: 2475-1421
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0001934839
  issn: 2475-1421
  databaseCode: M~E
  dateStart: 20170101
  isFulltext: true
  titleUrlDefault: https://road.issn.org
  providerName: ISSN International Centre
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1LS8QwEA66evDiY1V804PgQYo0nTbJcVkUD7oWXMGbpG2qHrYu3a548rc7SdMa0YMe7CGUvKD5wuTLdB6EHAuIZY63Hb9gLPchFaGf4sHmZ1SJGBk-UJAm2QQbjfj9vUhswruZSSfAypK_vYnpv0KNdQi2dp39A9zdpFiB7wg6lgg7lr8CPjFZuLUGIGlsrybGkcnqJduAqo8uK026U8wYdgyG1_ofwtQZ3qo1OwY-RPn5ZE3vtc6mE9w6Y30pJ_OqNumCT2-fqudKTlztAjVGKVQ4-8FarBuJRIFFfgCNS3MrPrmzS25ukturQeAKROEcrU246O9CG3R8izAGwa1J9dcI2LZlkSxRFgltt3f97ijSRAjI8hqHaD3Xme2v-UY2cfiGQxzG62TVMn5v0CC1QRZU2SdrbTYNzwrXTXLSAec5wHktcF4L3Ba5uzgfDy99m8bClzQUNR50qeJpzkBleZAjA89zLrksUnxylgY0RSkolVSKI3cCiAoeF1TImOJVVIog3Ca98qVUO8TDy6CCoEDOpQM1FrEAxQAyapQcKRS7pI-f_DBtApU82IXYJV67BF1T448etV32fhy4T1Y-98QB6dXVXB2S5ey1fp5VRwaGD2kwNc4
linkProvider ISSN International Centre
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Profiling+Programming+Language+Learning&rft.jtitle=Proceedings+of+ACM+on+programming+languages&rft.au=Crichton%2C+Will&rft.au=Krishnamurthi%2C+Shriram&rft.date=2024-04-29&rft.pub=ACM&rft.eissn=2475-1421&rft.volume=8&rft.issue=OOPSLA1&rft.spage=29&rft.epage=54&rft_id=info:doi/10.1145%2F3649812&rft.externalDocID=3649812
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2475-1421&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2475-1421&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2475-1421&client=summon