Finding “similar” universities using ChatGPT for institutional benchmarking: A large‐scale comparison of European universities

The study objective was to evaluate the efficacy of ChatGPT in identifying “similar” institutions for benchmarking the research performance of a university. Benchmarking is deemed a promising approach to compare “similar with similar” as a better alternative to rankings (comparing “different” univer...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of the Association for Information Science and Technology Jg. 76; H. 9; S. 1174 - 1187
Hauptverfasser: Lepori, Benedetto, Bornmann, Lutz, Gay, Mario
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Hoboken, USA John Wiley & Sons, Inc 01.09.2025
ISSN:2330-1635, 2330-1643
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The study objective was to evaluate the efficacy of ChatGPT in identifying “similar” institutions for benchmarking the research performance of a university. Benchmarking is deemed a promising approach to compare “similar with similar” as a better alternative to rankings (comparing “different” universities). Current approaches either focus on a limited number of “quantitative” dimensions or are too complex for most users. We conducted large‐scale testing by tasking ChatGPT with identifying the most similar European universities in terms of research performance, utilizing the European Tertiary Education Register data. We tested whether the peers suggested by ChatGPT were similar to the focal university on size, research intensity, and subject composition. Additionally, we evaluated whether providing more specific instructions improved the results. The findings offer a nuanced perspective on the potential and risks of using ChatGPT to identify peer institutions for benchmarking. On one hand, solely using ChatGPT would replicate the visibility biases associated with university rankings, thereby undermining the rationale for benchmarking. On the other hand, relying on semantic associations might capture dimensions of university similarity that are relevant and difficult to capture through quantitative methods. We finally reflected on the broader implications for scholars in higher education and science studies research.
ISSN:2330-1635
2330-1643
DOI:10.1002/asi.25010