An evaluation of percentile measures of citation impact, and a proposal for making them better

Percentiles are statistics pointing to the standing of a paper’s citation impact relative to other papers in a given citation distribution. Percentile Ranks ( PR s) often play an important role in evaluating the impact of researchers, institutions, and similar lines of study. Because PR s are so imp...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	Scientometrics Jg. 124; H. 2; S. 1457 - 1478
Hauptverfasser:	Bornmann, Lutz, Williams, Richard
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	Cham Springer International Publishing 01.08.2020 Springer Nature B.V
Schlagworte:	Bibliometrics Citation analysis Citations Computer Science Evaluation Graphs Information Storage and Retrieval Library Science Normalization Public relations Researchers Statistics Percentile Plotting position Bibliometrics Citation analysis Percentile rank Percentile point Field-normalization
ISSN:	0138-9130, 1588-2861
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Abstract	Percentiles are statistics pointing to the standing of a paper’s citation impact relative to other papers in a given citation distribution. Percentile Ranks ( PR s) often play an important role in evaluating the impact of researchers, institutions, and similar lines of study. Because PR s are so important for the assessment of scholarly impact, and because citations differ greatly across time and fields, various percentile approaches have been proposed to time- and field-normalize citations. Unfortunately, current popular methods often face significant problems in time- and field-normalization, including when papers are assigned to multiple fields or have been published by more than one unit (e.g., researchers or countries). They also face problems for estimating citation counts for pre-defined PR s (e.g., the 90th PR ). We offer a series of guidelines and procedures that, we argue, address these problems and others and provide a superior means to make the use of percentile methods more accurate and informative. In particular, we introduce two approaches, CP - IN and CP - EX , that should be preferred in bibliometric studies because they consider the complete citation distribution and can be accurately interpreted. Both approaches are based on cumulative frequencies in percentages ( CP s). The paper further shows how bar graphs and beamplots can present PR s in a more meaningful and accurate manner.
AbstractList	Percentiles are statistics pointing to the standing of a paper’s citation impact relative to other papers in a given citation distribution. Percentile Ranks ( PR s) often play an important role in evaluating the impact of researchers, institutions, and similar lines of study. Because PR s are so important for the assessment of scholarly impact, and because citations differ greatly across time and fields, various percentile approaches have been proposed to time- and field-normalize citations. Unfortunately, current popular methods often face significant problems in time- and field-normalization, including when papers are assigned to multiple fields or have been published by more than one unit (e.g., researchers or countries). They also face problems for estimating citation counts for pre-defined PR s (e.g., the 90th PR ). We offer a series of guidelines and procedures that, we argue, address these problems and others and provide a superior means to make the use of percentile methods more accurate and informative. In particular, we introduce two approaches, CP - IN and CP - EX , that should be preferred in bibliometric studies because they consider the complete citation distribution and can be accurately interpreted. Both approaches are based on cumulative frequencies in percentages ( CP s). The paper further shows how bar graphs and beamplots can present PR s in a more meaningful and accurate manner. Percentiles are statistics pointing to the standing of a paper’s citation impact relative to other papers in a given citation distribution. Percentile Ranks (PRs) often play an important role in evaluating the impact of researchers, institutions, and similar lines of study. Because PRs are so important for the assessment of scholarly impact, and because citations differ greatly across time and fields, various percentile approaches have been proposed to time- and field-normalize citations. Unfortunately, current popular methods often face significant problems in time- and field-normalization, including when papers are assigned to multiple fields or have been published by more than one unit (e.g., researchers or countries). They also face problems for estimating citation counts for pre-defined PRs (e.g., the 90th PR). We offer a series of guidelines and procedures that, we argue, address these problems and others and provide a superior means to make the use of percentile methods more accurate and informative. In particular, we introduce two approaches, CP-IN and CP-EX, that should be preferred in bibliometric studies because they consider the complete citation distribution and can be accurately interpreted. Both approaches are based on cumulative frequencies in percentages (CPs). The paper further shows how bar graphs and beamplots can present PRs in a more meaningful and accurate manner.
Author	Williams, Richard Bornmann, Lutz
Author_xml	– sequence: 1 givenname: Lutz orcidid: 0000-0003-0810-7091 surname: Bornmann fullname: Bornmann, Lutz email: bornmann@gv.mpg.de organization: Division for Science and Innovation Studies, Administrative Headquarters of the Max Planck Society – sequence: 2 givenname: Richard surname: Williams fullname: Williams, Richard organization: Department of Sociology, University of Notre Dame
BookMark	eNp9kE1LxDAQhoMouK7-AU8Br1Zn0maTPS6LXyB40ashTWfXrm1ak6zgv7drFcGDp4Hhfd4ZniO27ztPjJ0iXCCAuoyIOBcZCMgglygytccmKLXOhJ7hPpsA5jqbYw6H7CjGDQxQDnrCnhee07tttjbVnefdivcUHPlUN8RbsnEbKO7Wrk5jpG5769I5t77ilveh67toG77qAm_ta-3XPL1Qy0tKicIxO1jZJtLJ95yyp-urx-Vtdv9wc7dc3GeuQEzZHLAsKiVmc6FAuhIUuQqsVahRVxWWDlAIKsVMCCdJlzJXRSFtUVYEUqp8ys7G3uGfty3FZDbdNvjhpBGFEKC0kjCkxJhyoYsx0Mr0oW5t-DAIZufRjB7N4NF8eTS7av0H-lGRgq2b_9F8RONwx68p_H71D_UJZHqIzg
CitedBy_id	crossref_primary_10_1177_01655515231165632 crossref_primary_10_1162_qss_a_00089 crossref_primary_10_1007_s11192_024_05028_w crossref_primary_10_1007_s11192_021_03944_9 crossref_primary_10_1007_s11192_021_04141_4 crossref_primary_10_1177_01655515231204800 crossref_primary_10_1016_j_joi_2024_101531 crossref_primary_10_1016_j_joi_2020_101098 crossref_primary_10_3390_healthcare12020259 crossref_primary_10_1016_j_joi_2023_101391 crossref_primary_10_1007_s11192_020_03660_w crossref_primary_10_1016_j_ipm_2023_103323 crossref_primary_10_1162_qss_a_00253 crossref_primary_10_3390_publications9040051 crossref_primary_10_1007_s11192_021_03995_y crossref_primary_10_1155_2023_7281075 crossref_primary_10_1007_s11192_021_04199_0 crossref_primary_10_1038_s41597_023_02198_9 crossref_primary_10_1371_journal_pone_0257307 crossref_primary_10_3389_frma_2020_628703 crossref_primary_10_1007_s11192_024_04997_2 crossref_primary_10_1007_s11192_022_04322_9 crossref_primary_10_1140_epjds_s13688_023_00421_6 crossref_primary_10_1371_journal_pone_0283893 crossref_primary_10_1038_s41598_024_65989_z crossref_primary_10_1007_s10462_025_11315_6 crossref_primary_10_1007_s11192_025_05338_7
Cites_doi	10.1016/j.joi.2010.08.001 10.1002/asi.22641 10.1007/s11192-007-1934-2 10.1016/j.joi.2016.02.007 10.1002/asi.23152 10.1007/978-3-319-10377-8_12 10.1016/j.joi.2015.08.001 10.1002/asi.22996 10.1007/s11192-013-1161-y 10.1007/BF02017249 10.1007/s11192-019-03018-x 10.1016/j.joi.2018.07.005 10.2307/2685780 10.1016/j.joi.2012.10.001 10.1016/j.joi.2015.01.006 10.1002/asi.22708 10.1002/asi.21609 10.1016/j.joi.2013.09.003 10.1108/S1876-0562(2005)05 10.1007/s11192-018-2658-1 10.1093/spp/14.2.99 10.1371/journal.pbio.1002542 10.1016/j.joi.2011.11.006 10.1061/taceat.0002563 10.1007/s11192-019-03099-8 10.1002/asi.22792 10.1109/TEM.1983.6448622 10.1093/reseval/rvu002 10.1038/520429a 10.1002/asi.22775
ContentType	Journal Article
Copyright	The Author(s) 2020 The Author(s) 2020. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml	– notice: The Author(s) 2020 – notice: The Author(s) 2020. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID	C6C AAYXX CITATION E3H F2A JQ2
DOI	10.1007/s11192-020-03512-7
DatabaseName	Springer Nature OA Free Journals CrossRef Library & Information Sciences Abstracts (LISA) Library & Information Science Abstracts (LISA) ProQuest Computer Science Collection
DatabaseTitle	CrossRef Library and Information Science Abstracts (LISA) ProQuest Computer Science Collection
DatabaseTitleList	CrossRef Library and Information Science Abstracts (LISA)
DeliveryMethod	fulltext_linktorsrc
Discipline	Sciences (General) Computer Science Statistics
EISSN	1588-2861
EndPage	1478
ExternalDocumentID	10_1007_s11192_020_03512_7
GroupedDBID	--K -4Z -59 -5G -BR -EM -Y2 -~C -~X .86 .DC .VR 06D 0R~ 0VY 123 1B1 1N0 1SB 2.D 203 28- 2J2 2JN 2JY 2KG 2KM 2LR 2P1 2VQ 2~H 30V 4.4 406 408 409 40D 40E 53G 5QI 5VS 67Z 6NX 77K 8TC 8UJ 95- 95. 95~ 96X AAAVM AABHQ AACDK AAHNG AAIAL AAJBT AANZL AAOBN AARHV AARTL AASML AATNV AATVU AAUYE AAWCG AAWTL AAYIU AAYQN AAYTO AAYZH ABAKF ABBBX ABBXA ABDZT ABECU ABFTV ABHLI ABHQN ABJNI ABJOX ABKCH ABKTR ABMNI ABMQK ABNWP ABQBU ABQSL ABSXP ABTEG ABTHY ABTKH ABTMW ABULA ABWNU ABXPI ACAOD ACBXY ACDTI ACGFS ACHQT ACHSB ACHXU ACIWK ACKNC ACMDZ ACMLO ACOKC ACOMO ACPIV ACPRK ACSNA ACYUM ACZOJ ADHHG ADHIR ADIMF ADINQ ADKNI ADKPE ADMLS ADRFC ADTPH ADURQ ADYFF ADZKW AEBTG AEFIE AEFQL AEGAL AEGNC AEJHL AEJRE AEKMD AEMSY AENEX AEOHA AEPYU AESKC AETLH AEVLU AEXYK AFBBN AFEXP AFGCZ AFLOW AFQWF AFRAH AFWTZ AFZKB AGAYW AGDGC AGGDS AGJBK AGMZJ AGQEE AGQMX AGRTI AGWIL AGWZB AGYKE AHAVH AHBYD AHSBF AHYZX AIAKS AIGIU AIIXL AILAN AITGF AJBLW AJRNO AJZVZ ALMA_UNASSIGNED_HOLDINGS ALWAN AMKLP AMXSW AMYLF AMYQR AOCGG ARMRJ ASPBG AVWKF AXYYD AYJHY AZFZN B-. BA0 BBWZM BDATZ BGNMA BSONS C6C CAG COF CS3 CSCUP DDRTE DL5 DNIVK DPUIP DU5 EBLON EBS EIOEI EJD ESBYG F5P FEDTE FERAY FFXSO FIGPU FINBP FNLPD FRRFC FSGXE FWDCC GGCAI GGRSB GJIRD GNWQR GQ6 GQ7 GQ8 GXS H13 HF~ HG5 HG6 HMJXF HQYDN HRMNR HVGLF HZ~ IHE IJ- IKXTQ ITM IWAJR IXC IZIGR IZQ I~X I~Y I~Z J-C J0Z JBSCW JCJTX JZLTJ KDC KOV KOW LLZTM M4Y MA- MET MKB N2Q N9A NB0 NDZJH NPVJJ NQ- NQJWS NU0 O-J O9- O93 O9G O9I O9J OAM OVD P19 P9O PF0 PT4 PT5 QOK QOS R-Y R4E R89 R9I RHV RIG RKA RNI ROL RPX RPZ RSV RZC RZE RZK S16 S1Z S26 S27 S28 S3B SAP SCJ SCLPG SCO SDH SDM SHX SISQX SJYHP SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW STPWE SZN T13 T16 TEORI TN5 TSG TSK TSV TUC U2A UG4 UOJIU UTJUX UZXMN VC2 VFIZW W23 W48 WH7 WHG WK6 WK8 YLTOR Z45 Z5O Z7R Z7U Z7V Z7X Z7Z Z81 Z83 Z88 Z8M Z8R Z8T Z8W Z92 ZMTXR ~A9 ~EX 77I 7X2 AAPKM AAYXX ABBRH ABDBE ABFSG ABJCF ABRTQ ACSTC ADHKG AEUYN AEZWR AFDZB AFFHD AFHIU AFKRA AFOHR AGQPQ AHPBZ AHWEU AIXLP ARAPS ATCPS ATHPR AYFIA BBNVY BENPR BGLVJ BHPHI BKSAR CCPQU CITATION HCIFZ K7- KB. M0K M7P M7S PATMY PCBAR PDBOC PHGZM PHGZT PQGLB PTHSS PYCSY E3H F2A JQ2
ID	FETCH-LOGICAL-c411t-901b4d72692705cb07ecd0aa71818dd1bc0122eb2622c5e8b537445a4bde05573
IEDL.DBID	RSV
ISICitedReferencesCount	31
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000534949200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0138-9130
IngestDate	Sat Nov 15 07:21:39 EST 2025 Sat Nov 29 05:58:37 EST 2025 Tue Nov 18 21:53:29 EST 2025 Fri Feb 21 02:35:16 EST 2025
IsDoiOpenAccess	true
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	2
Keywords	Percentile Plotting position Bibliometrics Citation analysis Percentile rank Percentile point Field-normalization
Language	English
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c411t-901b4d72692705cb07ecd0aa71818dd1bc0122eb2622c5e8b537445a4bde05573
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0003-0810-7091
OpenAccessLink	https://link.springer.com/10.1007/s11192-020-03512-7
PQID	2422078750
PQPubID	2043823
PageCount	22
ParticipantIDs	proquest_journals_2422078750 crossref_primary_10_1007_s11192_020_03512_7 crossref_citationtrail_10_1007_s11192_020_03512_7 springer_journals_10_1007_s11192_020_03512_7
PublicationCentury	2000
PublicationDate	2020-08-01
PublicationDateYYYYMMDD	2020-08-01
PublicationDate_xml	– month: 08 year: 2020 text: 2020-08-01 day: 01
PublicationDecade	2020
PublicationPlace	Cham
PublicationPlace_xml	– name: Cham – name: Dordrecht
PublicationSubtitle	An International Journal for all Quantitative Aspects of the Science of Science, Communication in Science and Science Policy
PublicationTitle	Scientometrics
PublicationTitleAbbrev	Scientometrics
PublicationYear	2020
Publisher	Springer International Publishing Springer Nature B.V
Publisher_xml	– name: Springer International Publishing – name: Springer Nature B.V
References	HicksDWoutersPWaltmanLde RijckeSRafolsIBibliometrics: The Leiden manifesto for research metricsNature2015520754842943110.1038/520429a BornmannLMarxWMethods for the generation of normalized citation impact scores in bibliometrics: Which method best reflects the judgements of experts?Journal of Informetrics20159240841810.1016/j.joi.2015.01.006 LeydesdorffLBornmannLIntegrated impact indicators (I3) compared with impact factors (IFs): An alternative research design with policy implicationsJournal of the American Society of Information Science and Technology201162112133214610.1002/asi.21609 BornmannLLeydesdorffLWangJWhich percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P100)Journal of Informetrics20137493394410.1016/j.joi.2013.09.003 BornmannLHow to analyze percentile citation impact data meaningfully in bibliometrics: The statistical analysis of distributions, percentile rank classes, and top-cited papersJournal of the American Society for Information Science and Technology201364358759510.1002/asi.22792 LavrakasPJEncyclopedia of survey research methods2008Thousand Oaks, CASage LeydesdorffLBornmannLPercentile ranks and the integrated impact indicator (I3)Journal of the American Society for Information Science and Technology20126391901190210.1002/asi.22641 IoannidisJPABoyackKWoutersPFCitation metrics: A primer on how (not) to normalizePLoS Biology2016149e100254210.1371/journal.pbio.1002542 SchubertABraunTRelative indicators and relational charts for comparative assessment of publication output and citation impactScientometrics198695–628129110.1007/BF02017249 BornmannLMarxWHow to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citationsScientometrics201498148750910.1007/s11192-013-1161-y WilliamsRBornmannLDingYRousseauRWolframDThe substantive and practical significance of citation impact differences between institutions: Guidelines for the analysis of percentiles using effect sizes and confidence intervalsMeasuring scholarly impact: Methods and practice2014HeidelbergSpringer259281 AdamsJMcVeighMPendleburyDSzomszorMProfiles, not metrics2019Philadelphia, PAClarivate Analytics BornmannLMutzRFrom P100 to P100′: A new citation-rank approachJournal of the Association For Information Science and Technology20146591939194310.1002/asi.23152 BornmannLde Moya AnegónFLeydesdorffLThe new excellence indicator in the world report of the SCImago institutions rankings 2011Journal of Informetrics20126233333510.1016/j.joi.2011.11.006 WaltmanLvan EckNJField-normalized citation impact indicators and the choice of an appropriate counting methodJournal of Informetrics20159487289410.1016/j.joi.2015.08.001 BornmannLMarxWDistributions instead of single numbers: Percentiles and beam plots for the assessment of single researchersJournal of the American Society of Information Science and Technology2014651206208 NarinFBibliometric techniques in the evaluation of research programsScience and Public Policy19871429910610.1093/spp/14.2.99 HazenAStorage to be provided in impounding reservoirs for municipal water supplyTransactions of American Society of Civil Engineers19147715391640 BornmannLHow are excellent (highly cited) papers defined in bibliometrics? A quantitative analysis of the literatureResearch Evaluation201423216617310.1093/reseval/rvu002 StataCorp.Stata statistical software: Release 152017College Station, TXStata Corporation Cox, N. J. (2005). Calculating percentile ranks or plotting positions. Retrieved May 30, 2019, from http://www.stata.com/support/faqs/stat/pcrank.html. TahamtanIBornmannLCreativity in science and the link to cited references: Is the creative potential of papers reflected in their cited references?Journal of Informetrics201812390693010.1016/j.joi.2018.07.005 EveredDHamettSNarinFEveredDHamettSThe impact of different modes of research fundingThe evaluation of scientific research1989ChichesterWiley120140 WaltmanLSchreiberMOn the calculation of percentile-based bibliometric indicatorsJournal of the American Society for Information Science and Technology201364237237910.1002/asi.22775 WaltmanLvan EckNJvan LeeuwenTNVisserMSvan RaanAFJTowards a new crown indicator: Some theoretical considerationsJournal of Informetrics201151374710.1016/j.joi.2010.08.001 WaltmanLA review of the literature on citation impact indicatorsJournal of Informetrics201610236539110.1016/j.joi.2016.02.007 LeydesdorffLBornmannLAdamsJThe integrated impact indicator (I3) revisited: A non-parametric alternative to the journal impact factorScientometrics201911931669169410.1007/s11192-019-03099-8 BornmannLMarewskiJNHeuristics as conceptual lens for understanding and studying the usage of bibliometrics in research evaluationScientometrics2019120241945910.1007/s11192-019-03018-x WaltmanLCalero-MedinaCKostenJNoyonsECMTijssenRJWvan EckNJWoutersPThe Leiden Ranking 2011/2012: Data collection, indicators, and interpretationJournal of the American Society for Information Science and Technology201263122419243210.1002/asi.22708 BornmannLLeydesdorffLMutzRThe use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limitsJournal of Informetrics20137115816510.1016/j.joi.2012.10.001 Barrett, P. (2003). Percentiles and textbook definitions—Confused or what? Retrieved November 11, 2019, from https://www.pbarrett.net/techpapers/percentiles.pdf. DoaneDPTracyRLUsing beam and fulcrum displays to explore dataAmerican Statistician2000544289290181556710.2307/2685780 GauffriauMLarsenPOMayeIRoulin-PerriardAvon InsMComparisons of results of publication counting using different methodsScientometrics200877114717610.1007/s11192-007-1934-2 McAllisterPRNarinFCorriganJGProgrammatic evaluation and comparison based on standardized citation scoresIEEE Transactions on Engineering Management198330420521110.1109/TEM.1983.6448622 BornmannLHaunschildRPlots for visualizing paper impact and journal impact of single researchers in a single graphScientometrics2018115138539410.1007/s11192-018-2658-1 EggheLPower laws in the information production process: Lotkaian informetrics2005KidlingtonElsevier Academic Press10.1108/S1876-0562(2005)05 L Bornmann (3512_CR7) 2013; 7 L Leydesdorff (3512_CR25) 2019; 119 L Waltman (3512_CR33) 2013; 64 F Narin (3512_CR27) 1987; 14 L Waltman (3512_CR31) 2016; 10 L Bornmann (3512_CR11) 2014; 98 D Evered (3512_CR17) 1989 I Tahamtan (3512_CR30) 2018; 12 M Gauffriau (3512_CR18) 2008; 77 L Bornmann (3512_CR10) 2014; 65 L Bornmann (3512_CR6) 2018; 115 L Waltman (3512_CR35) 2011; 5 L Bornmann (3512_CR4) 2014; 23 R Williams (3512_CR36) 2014 (3512_CR22) 2008 L Egghe (3512_CR16) 2005 JPA Ioannidis (3512_CR21) 2016; 14 PR McAllister (3512_CR26) 1983; 30 StataCorp. (3512_CR29) 2017 L Waltman (3512_CR32) 2012; 63 DP Doane (3512_CR15) 2000; 54 L Bornmann (3512_CR13) 2014; 65 L Leydesdorff (3512_CR23) 2011; 62 A Schubert (3512_CR28) 1986; 9 L Bornmann (3512_CR3) 2013; 64 L Bornmann (3512_CR8) 2013; 7 L Bornmann (3512_CR9) 2019; 120 3512_CR14 L Leydesdorff (3512_CR24) 2012; 63 A Hazen (3512_CR19) 1914; 77 3512_CR2 D Hicks (3512_CR20) 2015; 520 L Bornmann (3512_CR5) 2012; 6 L Waltman (3512_CR34) 2015; 9 J Adams (3512_CR1) 2019 L Bornmann (3512_CR12) 2015; 9
References_xml	– reference: WilliamsRBornmannLDingYRousseauRWolframDThe substantive and practical significance of citation impact differences between institutions: Guidelines for the analysis of percentiles using effect sizes and confidence intervalsMeasuring scholarly impact: Methods and practice2014HeidelbergSpringer259281 – reference: StataCorp.Stata statistical software: Release 152017College Station, TXStata Corporation – reference: HicksDWoutersPWaltmanLde RijckeSRafolsIBibliometrics: The Leiden manifesto for research metricsNature2015520754842943110.1038/520429a – reference: BornmannLMutzRFrom P100 to P100′: A new citation-rank approachJournal of the Association For Information Science and Technology20146591939194310.1002/asi.23152 – reference: LeydesdorffLBornmannLPercentile ranks and the integrated impact indicator (I3)Journal of the American Society for Information Science and Technology20126391901190210.1002/asi.22641 – reference: AdamsJMcVeighMPendleburyDSzomszorMProfiles, not metrics2019Philadelphia, PAClarivate Analytics – reference: EveredDHamettSNarinFEveredDHamettSThe impact of different modes of research fundingThe evaluation of scientific research1989ChichesterWiley120140 – reference: WaltmanLCalero-MedinaCKostenJNoyonsECMTijssenRJWvan EckNJWoutersPThe Leiden Ranking 2011/2012: Data collection, indicators, and interpretationJournal of the American Society for Information Science and Technology201263122419243210.1002/asi.22708 – reference: BornmannLMarxWHow to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citationsScientometrics201498148750910.1007/s11192-013-1161-y – reference: LeydesdorffLBornmannLIntegrated impact indicators (I3) compared with impact factors (IFs): An alternative research design with policy implicationsJournal of the American Society of Information Science and Technology201162112133214610.1002/asi.21609 – reference: BornmannLMarxWDistributions instead of single numbers: Percentiles and beam plots for the assessment of single researchersJournal of the American Society of Information Science and Technology2014651206208 – reference: EggheLPower laws in the information production process: Lotkaian informetrics2005KidlingtonElsevier Academic Press10.1108/S1876-0562(2005)05 – reference: BornmannLMarewskiJNHeuristics as conceptual lens for understanding and studying the usage of bibliometrics in research evaluationScientometrics2019120241945910.1007/s11192-019-03018-x – reference: NarinFBibliometric techniques in the evaluation of research programsScience and Public Policy19871429910610.1093/spp/14.2.99 – reference: BornmannLMarxWMethods for the generation of normalized citation impact scores in bibliometrics: Which method best reflects the judgements of experts?Journal of Informetrics20159240841810.1016/j.joi.2015.01.006 – reference: WaltmanLvan EckNJField-normalized citation impact indicators and the choice of an appropriate counting methodJournal of Informetrics20159487289410.1016/j.joi.2015.08.001 – reference: TahamtanIBornmannLCreativity in science and the link to cited references: Is the creative potential of papers reflected in their cited references?Journal of Informetrics201812390693010.1016/j.joi.2018.07.005 – reference: McAllisterPRNarinFCorriganJGProgrammatic evaluation and comparison based on standardized citation scoresIEEE Transactions on Engineering Management198330420521110.1109/TEM.1983.6448622 – reference: WaltmanLSchreiberMOn the calculation of percentile-based bibliometric indicatorsJournal of the American Society for Information Science and Technology201364237237910.1002/asi.22775 – reference: BornmannLHow are excellent (highly cited) papers defined in bibliometrics? A quantitative analysis of the literatureResearch Evaluation201423216617310.1093/reseval/rvu002 – reference: LeydesdorffLBornmannLAdamsJThe integrated impact indicator (I3) revisited: A non-parametric alternative to the journal impact factorScientometrics201911931669169410.1007/s11192-019-03099-8 – reference: IoannidisJPABoyackKWoutersPFCitation metrics: A primer on how (not) to normalizePLoS Biology2016149e100254210.1371/journal.pbio.1002542 – reference: BornmannLHaunschildRPlots for visualizing paper impact and journal impact of single researchers in a single graphScientometrics2018115138539410.1007/s11192-018-2658-1 – reference: HazenAStorage to be provided in impounding reservoirs for municipal water supplyTransactions of American Society of Civil Engineers19147715391640 – reference: BornmannLde Moya AnegónFLeydesdorffLThe new excellence indicator in the world report of the SCImago institutions rankings 2011Journal of Informetrics20126233333510.1016/j.joi.2011.11.006 – reference: WaltmanLA review of the literature on citation impact indicatorsJournal of Informetrics201610236539110.1016/j.joi.2016.02.007 – reference: DoaneDPTracyRLUsing beam and fulcrum displays to explore dataAmerican Statistician2000544289290181556710.2307/2685780 – reference: Barrett, P. (2003). Percentiles and textbook definitions—Confused or what? Retrieved November 11, 2019, from https://www.pbarrett.net/techpapers/percentiles.pdf. – reference: WaltmanLvan EckNJvan LeeuwenTNVisserMSvan RaanAFJTowards a new crown indicator: Some theoretical considerationsJournal of Informetrics201151374710.1016/j.joi.2010.08.001 – reference: BornmannLLeydesdorffLMutzRThe use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limitsJournal of Informetrics20137115816510.1016/j.joi.2012.10.001 – reference: BornmannLLeydesdorffLWangJWhich percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P100)Journal of Informetrics20137493394410.1016/j.joi.2013.09.003 – reference: LavrakasPJEncyclopedia of survey research methods2008Thousand Oaks, CASage – reference: GauffriauMLarsenPOMayeIRoulin-PerriardAvon InsMComparisons of results of publication counting using different methodsScientometrics200877114717610.1007/s11192-007-1934-2 – reference: SchubertABraunTRelative indicators and relational charts for comparative assessment of publication output and citation impactScientometrics198695–628129110.1007/BF02017249 – reference: Cox, N. J. (2005). Calculating percentile ranks or plotting positions. Retrieved May 30, 2019, from http://www.stata.com/support/faqs/stat/pcrank.html. – reference: BornmannLHow to analyze percentile citation impact data meaningfully in bibliometrics: The statistical analysis of distributions, percentile rank classes, and top-cited papersJournal of the American Society for Information Science and Technology201364358759510.1002/asi.22792 – volume: 5 start-page: 37 issue: 1 year: 2011 ident: 3512_CR35 publication-title: Journal of Informetrics doi: 10.1016/j.joi.2010.08.001 – volume: 63 start-page: 1901 issue: 9 year: 2012 ident: 3512_CR24 publication-title: Journal of the American Society for Information Science and Technology doi: 10.1002/asi.22641 – volume: 77 start-page: 147 issue: 1 year: 2008 ident: 3512_CR18 publication-title: Scientometrics doi: 10.1007/s11192-007-1934-2 – volume: 10 start-page: 365 issue: 2 year: 2016 ident: 3512_CR31 publication-title: Journal of Informetrics doi: 10.1016/j.joi.2016.02.007 – volume: 65 start-page: 1939 issue: 9 year: 2014 ident: 3512_CR13 publication-title: Journal of the Association For Information Science and Technology doi: 10.1002/asi.23152 – start-page: 259 volume-title: Measuring scholarly impact: Methods and practice year: 2014 ident: 3512_CR36 doi: 10.1007/978-3-319-10377-8_12 – volume: 9 start-page: 872 issue: 4 year: 2015 ident: 3512_CR34 publication-title: Journal of Informetrics doi: 10.1016/j.joi.2015.08.001 – volume: 65 start-page: 206 issue: 1 year: 2014 ident: 3512_CR10 publication-title: Journal of the American Society of Information Science and Technology doi: 10.1002/asi.22996 – ident: 3512_CR14 – volume: 98 start-page: 487 issue: 1 year: 2014 ident: 3512_CR11 publication-title: Scientometrics doi: 10.1007/s11192-013-1161-y – volume-title: Encyclopedia of survey research methods year: 2008 ident: 3512_CR22 – volume: 9 start-page: 281 issue: 5–6 year: 1986 ident: 3512_CR28 publication-title: Scientometrics doi: 10.1007/BF02017249 – volume: 120 start-page: 419 issue: 2 year: 2019 ident: 3512_CR9 publication-title: Scientometrics doi: 10.1007/s11192-019-03018-x – volume: 12 start-page: 906 issue: 3 year: 2018 ident: 3512_CR30 publication-title: Journal of Informetrics doi: 10.1016/j.joi.2018.07.005 – volume: 54 start-page: 289 issue: 4 year: 2000 ident: 3512_CR15 publication-title: American Statistician doi: 10.2307/2685780 – volume: 7 start-page: 158 issue: 1 year: 2013 ident: 3512_CR7 publication-title: Journal of Informetrics doi: 10.1016/j.joi.2012.10.001 – volume: 9 start-page: 408 issue: 2 year: 2015 ident: 3512_CR12 publication-title: Journal of Informetrics doi: 10.1016/j.joi.2015.01.006 – volume: 63 start-page: 2419 issue: 12 year: 2012 ident: 3512_CR32 publication-title: Journal of the American Society for Information Science and Technology doi: 10.1002/asi.22708 – start-page: 120 volume-title: The evaluation of scientific research year: 1989 ident: 3512_CR17 – volume: 62 start-page: 2133 issue: 11 year: 2011 ident: 3512_CR23 publication-title: Journal of the American Society of Information Science and Technology doi: 10.1002/asi.21609 – volume: 7 start-page: 933 issue: 4 year: 2013 ident: 3512_CR8 publication-title: Journal of Informetrics doi: 10.1016/j.joi.2013.09.003 – volume-title: Power laws in the information production process: Lotkaian informetrics year: 2005 ident: 3512_CR16 doi: 10.1108/S1876-0562(2005)05 – ident: 3512_CR2 – volume: 115 start-page: 385 issue: 1 year: 2018 ident: 3512_CR6 publication-title: Scientometrics doi: 10.1007/s11192-018-2658-1 – volume: 14 start-page: 99 issue: 2 year: 1987 ident: 3512_CR27 publication-title: Science and Public Policy doi: 10.1093/spp/14.2.99 – volume: 14 start-page: e1002542 issue: 9 year: 2016 ident: 3512_CR21 publication-title: PLoS Biology doi: 10.1371/journal.pbio.1002542 – volume: 6 start-page: 333 issue: 2 year: 2012 ident: 3512_CR5 publication-title: Journal of Informetrics doi: 10.1016/j.joi.2011.11.006 – volume-title: Stata statistical software: Release 15 year: 2017 ident: 3512_CR29 – volume: 77 start-page: 1539 year: 1914 ident: 3512_CR19 publication-title: Transactions of American Society of Civil Engineers doi: 10.1061/taceat.0002563 – volume: 119 start-page: 1669 issue: 3 year: 2019 ident: 3512_CR25 publication-title: Scientometrics doi: 10.1007/s11192-019-03099-8 – volume: 64 start-page: 587 issue: 3 year: 2013 ident: 3512_CR3 publication-title: Journal of the American Society for Information Science and Technology doi: 10.1002/asi.22792 – volume: 30 start-page: 205 issue: 4 year: 1983 ident: 3512_CR26 publication-title: IEEE Transactions on Engineering Management doi: 10.1109/TEM.1983.6448622 – volume: 23 start-page: 166 issue: 2 year: 2014 ident: 3512_CR4 publication-title: Research Evaluation doi: 10.1093/reseval/rvu002 – volume-title: Profiles, not metrics year: 2019 ident: 3512_CR1 – volume: 520 start-page: 429 issue: 7548 year: 2015 ident: 3512_CR20 publication-title: Nature doi: 10.1038/520429a – volume: 64 start-page: 372 issue: 2 year: 2013 ident: 3512_CR33 publication-title: Journal of the American Society for Information Science and Technology doi: 10.1002/asi.22775
SSID	ssj0007308
Score	2.4792438
Snippet	Percentiles are statistics pointing to the standing of a paper’s citation impact relative to other papers in a given citation distribution. Percentile Ranks (... Percentiles are statistics pointing to the standing of a paper’s citation impact relative to other papers in a given citation distribution. Percentile Ranks...
SourceID	proquest crossref springer
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	1457
SubjectTerms	Bibliometrics Citation analysis Citations Computer Science Evaluation Graphs Information Storage and Retrieval Library Science Normalization Public relations Researchers Statistics
Title	An evaluation of percentile measures of citation impact, and a proposal for making them better
URI	https://link.springer.com/article/10.1007/s11192-020-03512-7 https://www.proquest.com/docview/2422078750
Volume	124
WOSCitedRecordID	wos000534949200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVAVX databaseName: SpringerLINK Contemporary 1997-Present customDbUrl: eissn: 1588-2861 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0007308 issn: 0138-9130 databaseCode: RSV dateStart: 19970101 isFulltext: true titleUrlDefault: https://link.springer.com/search?facet-content-type=%22Journal%22 providerName: Springer Nature
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1LS8NAEB60evBibVWsVtmDB8UGsunmdSyieJAiPkpPhn0VBJuWpvr73dlsGhQV9BaSzSbZnZn9NjPzDcBpJNJQpxOGBU1Sz2ii9nisI09QqULp83BiowlHt_FwmIzH6Z1LCiuqaPfKJWktdZ3sRilGSprtDnq_DC5chw2z3CWojvcPo5X9NTJr7S-1qtz3XarM9318Xo5qjPnFLWpXm-vm_95zB7YduiSDUhxasKbzNjSryg3EKXIbWu6oIGeOd_p8F54HOanJv8lsQuYY9ZIvjeEg0_JfYoGnpeP1JmWKZY_wXBFO5lhwoTCPNziYTG2VK4KcsETYlKE9eLq-ery88Vz5BU8ySpcYuCGYioMoDWI_lMKPtVQ-52Y1o4lSVEh0y5mdeRQEMtSJCPsxYyFnQmlk9urvQyOf5foASICF1X3EElQzJWRiQBgyfZnOeCRZ0gFazUJWfQOWyHjNalZlHNXMjGpmRzWLO3CxumdeMnP82rpbTW7mtLTIDDwJDEQyoKkDvWoy68s_93b4t-ZHsBVYecC4wS40los3fQyb8n35UixOrPR-ACnA5cg
linkProvider	Springer Nature
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3fS8MwED50Cvri3FScTs2DD4oWmi799TjEMXEO0Tn2ZGnSDATXjbX695vL0hVFBX0rbZq2yd3lS-_uO4BTj4euDMcMC5qEltJEacW-9CxOReIKO3bHOppw2PP7_WA0Cu9NUlhWRLsXLkltqctkN0oxUlJtd9D7pXDhKqwxtWJhIN_D43Bpf5XMavtLtSq3bJMq830fn5ejEmN-cYvq1aZT_d97bsOWQZekvRCHGqzItA7VonIDMYpch5o5ysiZ4Z0-34HndkpK8m8yHZMZRr2kuTIcZLL4l5jhaWF4vckixfKSxGlCYjLDgguZerzCwWSiq1wR5IQlXKcM7cJT53pw1bVM-QVLMEpzDNzgLPEdL3R82xXc9qVI7DhWqxkNkoRygW45tTP3HEe4MuBuy2fMjRlPJDJ7tfagkk5TuQ_EwcLqNmIJKlnCRaBAGDJ9qc5iT7CgAbSYhaj4BiyR8RqVrMo4qpEa1UiPauQ34GJ5z2zBzPFr62YxuZHR0ixS8MRREEmBpgZcFpNZXv65t4O_NT-Bje7grhf1bvq3h7DpaNnAGMImVPL5mzyCdfGev2TzYy3JH_cb6Kw
linkToPdf	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LS8QwEB50FfHiuj5wddUcPChabLrp6yjqorgsgg88WZpHQdBu2VZ_v5m03aqoIN5Km6ZtMpN8aWa-D2DP46GrwoShoEloaU9UVuwrz-JUSFfYsZuYaML7oT8aBQ8P4fWHLH4T7V5vSZY5DcjSlBbHmUyOm8Q3SjFqUi99cCdMY8RZmGMoGoTr9Zv76Vis7deMxdS4dd-u0ma-r-Pz1NTgzS9bpGbmGbT__87LsFShTnJSmkkHZlS6Au1a0YFUDr4CneooJ_sVH_XBKjyepKQhBSfjhGQYDZMWekAhL-U_xhxPi4rvm5Spl0ckTiWJSYZCDLl-vMbH5MWoXxHkiiXcpBKtwd3g_Pb0wqpkGSzBKC0woIMz6Tte6Pi2K7jtKyHtONazHA2kpFzgdp1esXuOI1wVcLfvM-bGjEuFjF_9dWil41RtAHFQcN1GjEEVk1wEGpwhA5iuLPYEC7pA6x6J6m9A6YznqGFbxlaNdKtGplUjvwuH03uykrHj19K9uqOjynvzSMMWR0MnDaa6cFR3bHP559o2_1Z8FxauzwbR8HJ0tQWLjjENDC3sQauYvKptmBdvxVM-2TFG_Q6czfGQ
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=An+evaluation+of+percentile+measures+of+citation+impact%2C+and+a+proposal+for+making+them+better&rft.jtitle=Scientometrics&rft.au=Bornmann+Lutz&rft.au=Williams%2C+Richard&rft.date=2020-08-01&rft.pub=Springer+Nature+B.V&rft.issn=0138-9130&rft.eissn=1588-2861&rft.volume=124&rft.issue=2&rft.spage=1457&rft.epage=1478&rft_id=info:doi/10.1007%2Fs11192-020-03512-7&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0138-9130&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0138-9130&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0138-9130&client=summon