Biased by Design? Evaluating Bias and Behavioral Diversity in LLM Annotation of Real-World and Synthetic Hotel Reviews

As large language models (LLMs) gain traction among researchers and practitioners, particularly in digital marketing for tasks such as customer feedback analysis and automated communication, concerns remain about the reliability and consistency of their outputs. This study investigates annotation bi...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:AI (Basel) Ročník 6; číslo 8; s. 178
Hlavní autoři: Voutsa, Maria C., Tsapatsoulis, Nicolas, Djouvas, Constantinos
Médium: Journal Article
Jazyk:angličtina
Vydáno: Basel MDPI AG 01.08.2025
Témata:
AI bias
Algorithms
annotation bias
Annotations
Artificial intelligence
aspect classification
Automation
Batch processing
Behavior
Bias
Chatbots
Datasets
Decision making
Hate speech
Human error
Labeling
Labels
Large language models
Marketing
Sentences
Sentiment analysis
Synthetic data
topic modeling
User generated content
ISSN:2673-2688, 2673-2688
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Abstract As large language models (LLMs) gain traction among researchers and practitioners, particularly in digital marketing for tasks such as customer feedback analysis and automated communication, concerns remain about the reliability and consistency of their outputs. This study investigates annotation bias in LLMs by comparing human and AI-generated annotation labels across sentiment, topic, and aspect dimensions in hotel booking reviews. Using the HRAST dataset, which includes 23,114 real user-generated review sentences and a synthetically generated corpus of 2000 LLM-authored sentences, we evaluate inter-annotator agreement between a human expert and three LLMs (ChatGPT-3.5, ChatGPT-4, and ChatGPT-4-mini) as a proxy for assessing annotation bias. Our findings show high agreement among LLMs, especially on synthetic data, but only moderate to fair alignment with human annotations, particularly in sentiment and aspect-based sentiment analysis. LLMs display a pronounced neutrality bias, often defaulting to neutral sentiment in ambiguous cases. Moreover, annotation behavior varies notably with task design, as manual, one-to-one prompting produces higher agreement with human labels than automated batch processing. The study identifies three distinct AI biases—repetition bias, behavioral bias, and neutrality bias—that shape annotation outcomes. These findings highlight how dataset complexity and annotation mode influence LLM behavior, offering important theoretical, methodological, and practical implications for AI-assisted annotation and synthetic content generation.
AbstractList As large language models (LLMs) gain traction among researchers and practitioners, particularly in digital marketing for tasks such as customer feedback analysis and automated communication, concerns remain about the reliability and consistency of their outputs. This study investigates annotation bias in LLMs by comparing human and AI-generated annotation labels across sentiment, topic, and aspect dimensions in hotel booking reviews. Using the HRAST dataset, which includes 23,114 real user-generated review sentences and a synthetically generated corpus of 2000 LLM-authored sentences, we evaluate inter-annotator agreement between a human expert and three LLMs (ChatGPT-3.5, ChatGPT-4, and ChatGPT-4-mini) as a proxy for assessing annotation bias. Our findings show high agreement among LLMs, especially on synthetic data, but only moderate to fair alignment with human annotations, particularly in sentiment and aspect-based sentiment analysis. LLMs display a pronounced neutrality bias, often defaulting to neutral sentiment in ambiguous cases. Moreover, annotation behavior varies notably with task design, as manual, one-to-one prompting produces higher agreement with human labels than automated batch processing. The study identifies three distinct AI biases—repetition bias, behavioral bias, and neutrality bias—that shape annotation outcomes. These findings highlight how dataset complexity and annotation mode influence LLM behavior, offering important theoretical, methodological, and practical implications for AI-assisted annotation and synthetic content generation.
Author Djouvas, Constantinos
Tsapatsoulis, Nicolas
Voutsa, Maria C.
Author_xml – sequence: 1
  givenname: Maria C.
  orcidid: 0000-0001-7889-3804
  surname: Voutsa
  fullname: Voutsa, Maria C.
– sequence: 2
  givenname: Nicolas
  orcidid: 0000-0002-6739-8602
  surname: Tsapatsoulis
  fullname: Tsapatsoulis, Nicolas
– sequence: 3
  givenname: Constantinos
  orcidid: 0000-0003-1215-7294
  surname: Djouvas
  fullname: Djouvas, Constantinos
BookMark eNplUcluFDEQtVCQCCEH_sASJw6d2G63lxPKBok0CIlFHK1qLxOPGjvYnkbz9_RkACE4VaneUk9Vz9FRyskj9JKSs77X5ByiIIpQqZ6gYyZk3zGh1NFf_TN0WuuGEMIGynhPjtF8GaF6h8cdvvY1rtMbfDPDtIUW0xrvQQzJ4Ut_D3PMBSZ8HWdfamw7HBNerd7ji5RyW_g54RzwRw9T9zWXyT0KP-1Su_ctWnybm58WeI7-R32BngaYqj_9VU_Ql7c3n69uu9WHd3dXF6vOckFbB6N0igNTTAhr-eCop1aNg6eKCi6DU1QHqcOopRgccC51P3ISqAblBin6E3R38HUZNuahxG9QdiZDNI-DXNYGypJu8oZJQe0wMO-Z5pYSJaXTQTErue1pkIvXq4PXQ8nft742s8nbkpb4pl-OqYVibL_x_MCyJddafDA2Hq7TCsTJUGL2vzJ_frUoXv-j-J3zf-5Pa-OUGg
CitedBy_id crossref_primary_10_3390_ai6090236
Cites_doi 10.1145/3613904.3641960
10.2196/preprints.48399
10.1109/TCSS.2019.2914080
10.18653/v1/2023.eacl-main.130
10.1145/3627508.3638322
10.1017/CBO9781139062367
10.1007/978-3-031-25752-0_26
10.3390/ai6050092
10.1057/s41270-025-00379-6
10.1093/OED/4832698884
10.1109/ACCESS.2024.3402809
10.3390/ai6030056
10.18653/v1/2023.emnlp-main.647
10.3390/tourhosp6010036
10.18653/v1/2024.hcinlp-1.8
10.1007/s11042-012-0995-2
10.3390/ai6050106
10.1371/journal.pdig.0000082
10.1609/aaai.v35i16.17671
10.1136/bmj.320.7237.768
10.1177/1609406919899220
10.3115/1613715.1613751
10.1145/1743384.1743478
10.1007/s10462-024-10903-2
10.1007/978-3-031-93429-2_25
10.1080/0960085X.2021.1927212
10.2196/43251
10.1109/IC3INA48034.2019.8949568
10.1109/ICMLA58977.2023.00089
10.1007/978-3-030-75178-4
10.1609/aaai.v34i01.5385
10.2307/2529310
10.1145/3308560.3317590
10.1609/icwsm.v19i1.35837
10.1145/3341161.3342931
10.1079/9781800626607.0011
10.18653/v1/2022.findings-acl.58
10.1080/13683500.2023.2300030
10.18653/v1/D19-1107
10.1016/j.sapharm.2012.04.004
10.1108/JD-05-2019-0102
10.1136/eb-2014-101946
10.1371/journal.pdig.0000278
10.1177/0022242920972933
10.1109/SMAP59435.2023.10255219
10.18653/v1/2024.acl-long.760
10.1145/3716554.3716620
10.1016/j.jclepro.2022.134120
10.1136/jech.2003.008466
10.1016/j.ijhm.2024.104030
10.3233/FAIA240191
10.1016/j.tourman.2010.12.011
ContentType Journal Article
Copyright 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml – notice: 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID AAYXX
CITATION
8FE
8FG
ABUWG
AFKRA
ARAPS
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
HCIFZ
P5Z
P62
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
DOA
DOI 10.3390/ai6080178
DatabaseName CrossRef
ProQuest SciTech Collection
ProQuest Technology Collection
ProQuest Central (Alumni)
ProQuest Central UK/Ireland
Advanced Technologies & Computer Science Collection
ProQuest Central Essentials - QC
ProQuest Central
ProQuest Technology Collection
ProQuest One
ProQuest Central Korea
SciTech Collection (ProQuest)
Advanced Technologies & Aerospace Database
ProQuest Advanced Technologies & Aerospace Collection
ProQuest Central Premium
ProQuest One Academic (New)
ProQuest Publicly Available Content Database
ProQuest One Academic Middle East (New)
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Applied & Life Sciences
ProQuest One Academic (retired)
ProQuest One Academic UKI Edition
ProQuest Central China
DOAJ Directory of Open Access Journals
DatabaseTitle CrossRef
Publicly Available Content Database
Advanced Technologies & Aerospace Collection
Technology Collection
ProQuest One Academic Middle East (New)
ProQuest Advanced Technologies & Aerospace Collection
ProQuest Central Essentials
ProQuest One Academic Eastern Edition
ProQuest Central (Alumni Edition)
SciTech Premium Collection
ProQuest One Community College
ProQuest Technology Collection
ProQuest SciTech Collection
ProQuest Central China
ProQuest Central
Advanced Technologies & Aerospace Database
ProQuest One Applied & Life Sciences
ProQuest One Academic UKI Edition
ProQuest Central Korea
ProQuest Central (New)
ProQuest One Academic
ProQuest One Academic (New)
DatabaseTitleList Publicly Available Content Database
CrossRef
Database_xml – sequence: 1
  dbid: DOA
  name: DOAJ Directory of Open Access Journals
  url: https://www.doaj.org/
  sourceTypes: Open Website
– sequence: 2
  dbid: PIMPY
  name: Publicly Available Content Database
  url: http://search.proquest.com/publiccontent
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 2673-2688
ExternalDocumentID oai_doaj_org_article_2761c552ee294c10877d9f82c74c31f7
10_3390_ai6080178
GroupedDBID AADQD
AAYXX
ABDBF
ACUHS
AFFHD
AFKRA
AFZYC
ALMA_UNASSIGNED_HOLDINGS
ARAPS
BENPR
BGLVJ
CCPQU
CITATION
GROUPED_DOAJ
HCIFZ
IAO
ICD
IGS
ISR
ITC
MODMG
M~E
OK1
PHGZM
PHGZT
PIMPY
PQGLB
8FE
8FG
ABUWG
AZQEC
DWQXO
P62
PKEHL
PQEST
PQQKQ
PQUKI
PRINS
ID FETCH-LOGICAL-c461t-ab7d84a28266cc45d1e1c8b5e181647fd819f79fb9765da44793b40f19a8d5763
IEDL.DBID DOA
ISICitedReferencesCount 2
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001557107900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 2673-2688
IngestDate Fri Oct 03 12:44:07 EDT 2025
Sat Nov 01 15:11:00 EDT 2025
Sat Nov 29 07:13:57 EST 2025
Tue Nov 18 21:46:39 EST 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 8
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c461t-ab7d84a28266cc45d1e1c8b5e181647fd819f79fb9765da44793b40f19a8d5763
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0003-1215-7294
0000-0001-7889-3804
0000-0002-6739-8602
OpenAccessLink https://doaj.org/article/2761c552ee294c10877d9f82c74c31f7
PQID 3243968226
PQPubID 5046920
ParticipantIDs doaj_primary_oai_doaj_org_article_2761c552ee294c10877d9f82c74c31f7
proquest_journals_3243968226
crossref_citationtrail_10_3390_ai6080178
crossref_primary_10_3390_ai6080178
PublicationCentury 2000
PublicationDate 2025-08-01
PublicationDateYYYYMMDD 2025-08-01
PublicationDate_xml – month: 08
  year: 2025
  text: 2025-08-01
  day: 01
PublicationDecade 2020
PublicationPlace Basel
PublicationPlace_xml – name: Basel
PublicationTitle AI (Basel)
PublicationYear 2025
Publisher MDPI AG
Publisher_xml – name: MDPI AG
References Smith (ref_27) 2014; 17
ref_58
ref_57
ref_12
ref_56
ref_11
ref_55
ref_10
ref_54
Nazer (ref_29) 2023; 2
Chen (ref_7) 2023; 25
ref_19
ref_18
ref_17
ref_16
ref_15
ref_59
Llorca (ref_28) 2004; 58
Giannoulakis (ref_37) 2019; 6
Ntalianis (ref_36) 2014; 69
Landis (ref_53) 1977; 33
Nasution (ref_21) 2024; 12
ref_24
ref_67
ref_66
ref_65
ref_20
ref_64
ref_63
ref_62
Reason (ref_34) 2000; 320
Bacalhau (ref_23) 2025; 13
Kar (ref_22) 2022; 376
ref_26
Kozinets (ref_14) 2020; 85
Jia (ref_49) 2025; 126
Sparks (ref_50) 2011; 32
Varsha (ref_25) 2023; 3
ref_35
Macdonald (ref_60) 2019; 76
ref_33
ref_32
ref_31
ref_30
Donohue (ref_61) 1972; 1
ref_39
ref_38
Wang (ref_2) 2024; 28
Gonzales (ref_13) 2023; 2
Joffe (ref_51) 2020; 19
Akter (ref_5) 2021; 60
ref_46
ref_45
Kordzadeh (ref_6) 2021; 31
ref_44
ref_43
ref_42
ref_41
ref_40
ref_1
ref_3
Vilares (ref_47) 2024; 57
ref_48
ref_9
Gisev (ref_52) 2013; 9
ref_8
ref_4
References_xml – ident: ref_43
  doi: 10.1145/3613904.3641960
– ident: ref_44
  doi: 10.2196/preprints.48399
– volume: 6
  start-page: 592
  year: 2019
  ident: ref_37
  article-title: Filtering Instagram Hashtags Through Crowdtagging and the HITS Algorithm
  publication-title: IEEE Trans. Comput. Soc. Syst.
  doi: 10.1109/TCSS.2019.2914080
– ident: ref_9
  doi: 10.18653/v1/2023.eacl-main.130
– ident: ref_41
  doi: 10.1145/3627508.3638322
– ident: ref_35
  doi: 10.1017/CBO9781139062367
– ident: ref_1
  doi: 10.1007/978-3-031-25752-0_26
– ident: ref_30
  doi: 10.3390/ai6050092
– volume: 13
  start-page: 308
  year: 2025
  ident: ref_23
  article-title: A bibliometric analysis of AI bias in marketing: Field evolution and future research agenda
  publication-title: J. Mark. Anal.
  doi: 10.1057/s41270-025-00379-6
– ident: ref_26
  doi: 10.1093/OED/4832698884
– ident: ref_39
– volume: 12
  start-page: 71876
  year: 2024
  ident: ref_21
  article-title: Chatgpt label: Comparing the quality of human-generated and llm-generated annotations in low-resource language nlp tasks
  publication-title: IEEE Access
  doi: 10.1109/ACCESS.2024.3402809
– ident: ref_66
  doi: 10.3390/ai6030056
– ident: ref_58
– ident: ref_15
  doi: 10.18653/v1/2023.emnlp-main.647
– ident: ref_3
  doi: 10.3390/tourhosp6010036
– ident: ref_32
  doi: 10.18653/v1/2024.hcinlp-1.8
– volume: 69
  start-page: 397
  year: 2014
  ident: ref_36
  article-title: Automatic annotation of image databases based on implicit crowdsourcing, visual concept modeling and evolution
  publication-title: Multimed. Tools Appl.
  doi: 10.1007/s11042-012-0995-2
– ident: ref_31
  doi: 10.3390/ai6050106
– volume: 2
  start-page: e0000082
  year: 2023
  ident: ref_13
  article-title: Synthetic data in health care: A narrative review
  publication-title: PLoS Digit. Health
  doi: 10.1371/journal.pdig.0000082
– volume: 3
  start-page: 100165
  year: 2023
  ident: ref_25
  article-title: How can we manage biases in artificial intelligence systems—A systematic literature review
  publication-title: Int. J. Inf. Manag. Data Insights
– ident: ref_56
– ident: ref_10
– ident: ref_17
– ident: ref_48
  doi: 10.1609/aaai.v35i16.17671
– volume: 60
  start-page: 102387
  year: 2021
  ident: ref_5
  article-title: Algorithmic bias in data-driven innovation in the age of AI
  publication-title: Int. J. Inf. Manag.
– ident: ref_20
– ident: ref_59
– volume: 320
  start-page: 768
  year: 2000
  ident: ref_34
  article-title: Human error: Models and management
  publication-title: BMJ
  doi: 10.1136/bmj.320.7237.768
– volume: 19
  start-page: 1609406919899220
  year: 2020
  ident: ref_51
  article-title: Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines
  publication-title: Int. J. Qual. Methods
  doi: 10.1177/1609406919899220
– ident: ref_63
  doi: 10.3115/1613715.1613751
– ident: ref_64
  doi: 10.1145/1743384.1743478
– volume: 57
  start-page: 265
  year: 2024
  ident: ref_47
  article-title: Contrasting linguistic patterns in human and LLM-generated news text
  publication-title: Artif. Intell. Rev.
  doi: 10.1007/s10462-024-10903-2
– ident: ref_40
  doi: 10.1007/978-3-031-93429-2_25
– volume: 31
  start-page: 388
  year: 2021
  ident: ref_6
  article-title: Algorithmic bias: Review, synthesis, and future research directions
  publication-title: Eur. J. Inf. Syst.
  doi: 10.1080/0960085X.2021.1927212
– volume: 25
  start-page: e43251
  year: 2023
  ident: ref_7
  article-title: Human-Centered Design to Address Biases in Artificial Intelligence
  publication-title: J. Med. Internet Res.
  doi: 10.2196/43251
– ident: ref_65
  doi: 10.1109/IC3INA48034.2019.8949568
– ident: ref_42
  doi: 10.1109/ICMLA58977.2023.00089
– ident: ref_16
  doi: 10.1007/978-3-030-75178-4
– ident: ref_62
  doi: 10.1609/aaai.v34i01.5385
– ident: ref_67
– volume: 33
  start-page: 159
  year: 1977
  ident: ref_53
  article-title: The Measurement of Observer Agreement for Categorical Data
  publication-title: Biometrics
  doi: 10.2307/2529310
– ident: ref_24
  doi: 10.1145/3308560.3317590
– ident: ref_12
  doi: 10.1609/icwsm.v19i1.35837
– ident: ref_33
  doi: 10.1145/3341161.3342931
– ident: ref_4
  doi: 10.1079/9781800626607.0011
– ident: ref_55
  doi: 10.18653/v1/2022.findings-acl.58
– volume: 28
  start-page: 527
  year: 2024
  ident: ref_2
  article-title: Personalizing guest experience with generative AI in the hotel industry: There’s more to it than meets a Kiwi’s eye
  publication-title: Curr. Issues Tour.
  doi: 10.1080/13683500.2023.2300030
– ident: ref_38
  doi: 10.18653/v1/D19-1107
– ident: ref_54
– ident: ref_46
– volume: 9
  start-page: 330
  year: 2013
  ident: ref_52
  article-title: Interrater agreement and interrater reliability: Key concepts, approaches, and applications
  publication-title: Res. Soc. Adm. Pharm.
  doi: 10.1016/j.sapharm.2012.04.004
– volume: 1
  start-page: 41
  year: 1972
  ident: ref_61
  article-title: Gatekeeping: Mass media systems and information control
  publication-title: Curr. Perspect. Mass Commun. Res.
– volume: 76
  start-page: 333
  year: 2019
  ident: ref_60
  article-title: The concept of neutrality: A new approach
  publication-title: J. Doc.
  doi: 10.1108/JD-05-2019-0102
– volume: 17
  start-page: 100
  year: 2014
  ident: ref_27
  article-title: Bias in research
  publication-title: Evid. Based Nurs.
  doi: 10.1136/eb-2014-101946
– volume: 2
  start-page: e0000278
  year: 2023
  ident: ref_29
  article-title: Bias in artificial intelligence algorithms and recommendations for mitigation
  publication-title: PLoS Digit. Health
  doi: 10.1371/journal.pdig.0000278
– volume: 85
  start-page: 156
  year: 2020
  ident: ref_14
  article-title: Commentary: Artificial Intelligence: The Marketer’s Dilemma
  publication-title: J. Mark.
  doi: 10.1177/0022242920972933
– ident: ref_18
  doi: 10.1109/SMAP59435.2023.10255219
– ident: ref_11
  doi: 10.18653/v1/2024.acl-long.760
– ident: ref_45
  doi: 10.1145/3716554.3716620
– volume: 376
  start-page: 134120
  year: 2022
  ident: ref_22
  article-title: How can artificial intelligence impact sustainability: A systematic literature review
  publication-title: J. Clean. Prod.
  doi: 10.1016/j.jclepro.2022.134120
– ident: ref_19
– volume: 58
  start-page: 635
  year: 2004
  ident: ref_28
  article-title: Bias
  publication-title: J. Epidemiol. Community Health
  doi: 10.1136/jech.2003.008466
– ident: ref_57
– volume: 126
  start-page: 104030
  year: 2025
  ident: ref_49
  article-title: Unpacking the impact of AI vs. human-generated review summary on hotel booking intentions
  publication-title: Int. J. Hosp. Manag.
  doi: 10.1016/j.ijhm.2024.104030
– ident: ref_8
  doi: 10.3233/FAIA240191
– volume: 32
  start-page: 1310
  year: 2011
  ident: ref_50
  article-title: The impact of online reviews on hotel booking intentions and perception of trust
  publication-title: Tour. Manag.
  doi: 10.1016/j.tourman.2010.12.011
SSID ssj0002512430
Score 2.3128521
Snippet As large language models (LLMs) gain traction among researchers and practitioners, particularly in digital marketing for tasks such as customer feedback...
SourceID doaj
proquest
crossref
SourceType Open Website
Aggregation Database
Enrichment Source
Index Database
StartPage 178
SubjectTerms AI bias
Algorithms
annotation bias
Annotations
Artificial intelligence
aspect classification
Automation
Batch processing
Behavior
Bias
Chatbots
Datasets
Decision making
Hate speech
Human error
Labeling
Labels
Large language models
Marketing
Sentences
Sentiment analysis
Synthetic data
topic modeling
User generated content
SummonAdditionalLinks – databaseName: Advanced Technologies & Aerospace Database
  dbid: P5Z
  link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1NT9wwELXKlgOX0pYittBqVPXQS0ScOE58QmwBcdgi1BaEeonir9VKyKG725X494wdJ1Si4tJr7EiWxp55Mx6_R8hnK4zm1tJEqwoTFF7JRGjJkiaV6C61kHmgUrqelhcX1c2NuIwFt2Vsq-x9YnDUulW-Rn6IgT8XHMMZP7r7nXjVKH-7GiU0NshLz5LgpRsui19DjcXHbpanHaFQjtn9YTPnCJGoF1X7KwwFtv4nzjhEmLPt_13ba_IqYks47jbDG_LCuLdku9dtgHiMd8h6MsfYpUHew0lo4DiC00j67WbgB6FxGibDC3446bs3YO5gOv0Gx8613SU-tBa-I9pMQltO-PHHvUNYiYuA83ZlbqG7f1i-I1dnpz-_nidRfyFRjNNV0shSV6zBpIxzpVihqaGqkoVBVMBZaTWiCVsKKxHSFLphvkgnWWqpaCqNeUy-S0audWaPgODUMpuzBuERs5hTyiLlQkldpkVllB6TL705ahXJyb1Gxm2NSYq3XD1Ybkw-DVPvOkaOf02aeJsOEzyJdvjQLmZ1PJN1VnKqiiIzJhNMUU-NqIWtMlUylVNbjslBb-46nuxl_Wjr988P75OtzGsFh2bBAzJaLf6YD2RTrVfz5eJj2KgPOhzyKw
  priority: 102
  providerName: ProQuest
Title Biased by Design? Evaluating Bias and Behavioral Diversity in LLM Annotation of Real-World and Synthetic Hotel Reviews
URI https://www.proquest.com/docview/3243968226
https://doaj.org/article/2761c552ee294c10877d9f82c74c31f7
Volume 6
WOSCitedRecordID wos001557107900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVAON
  databaseName: DOAJ Directory of Open Access Journals
  customDbUrl:
  eissn: 2673-2688
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0002512430
  issn: 2673-2688
  databaseCode: DOA
  dateStart: 20200101
  isFulltext: true
  titleUrlDefault: https://www.doaj.org/
  providerName: Directory of Open Access Journals
– providerCode: PRVHPJ
  databaseName: ROAD: Directory of Open Access Scholarly Resources
  customDbUrl:
  eissn: 2673-2688
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0002512430
  issn: 2673-2688
  databaseCode: M~E
  dateStart: 20190101
  isFulltext: true
  titleUrlDefault: https://road.issn.org
  providerName: ISSN International Centre
– providerCode: PRVPQU
  databaseName: Advanced Technologies & Aerospace Database
  customDbUrl:
  eissn: 2673-2688
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0002512430
  issn: 2673-2688
  databaseCode: P5Z
  dateStart: 20200101
  isFulltext: true
  titleUrlDefault: https://search.proquest.com/hightechjournals
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: ProQuest Central
  customDbUrl:
  eissn: 2673-2688
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0002512430
  issn: 2673-2688
  databaseCode: BENPR
  dateStart: 20200101
  isFulltext: true
  titleUrlDefault: https://www.proquest.com/central
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: Publicly Available Content Database
  customDbUrl:
  eissn: 2673-2688
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0002512430
  issn: 2673-2688
  databaseCode: PIMPY
  dateStart: 20200101
  isFulltext: true
  titleUrlDefault: http://search.proquest.com/publiccontent
  providerName: ProQuest
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LS-wwFA6iLu7G173i-OIgLu6m2LRp2qzE0RGFcSjeq6ib0rxkQDLijIL_3pO0MwwouHHTRZrScnKS833NyXcIObTCaG4tjbQqkKDwQkZCSxbVscTlUguZBiml234-GBR3d6KcK_Xlc8IaeeDGcEcJ8myVZYkxiWCKev06LWyRqJyplNpwjjzOxRyZ8muwj9osjRspoRR5_VE95AiOqC-nNheAgk7_p2U4xJbzNbLSgkI4aT5mnSwYt0FWpwUXoJ1_v8lbd4hBR4N8h7OQeXEMvVat2z2Cvwm109CdHb2Hs2naBQwd9PtXcOLcqNl9h5GFa4SJUcinCQ_-e3eIB_Ej4GI0MU_QbByM_5Cb897_04uoLZwQKcbpJKplrgtWI5viXCmWaWqoKmRmMJxzlluNMMDmwkrEIpmumf-7JllsqagLjQQk3SSLbuTMFgHBqWU2ZTXiGmaRDMos5kJJncdZYZTukL9Ta1aqVRX3xS2eKmQX3vDVzPAdcjDr-txIaXzVqeuHZNbBq1-HBvSJqvWJ6juf6JDd6YBW7ZQcV4gcU8ERD_Htn3jHDvmV-FLAIRdwlyxOXl7NHllWb5Ph-GWfLHV7g_J6P3glXsvsAdvKy6vy_gMRWOdg
linkProvider Directory of Open Access Journals
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V3LbtNAFL0qKRJsKE-RPmCEQGJj1WOPx54FqhrSKlGdKIKCysp4XlWkym6T0Co_xTf2jl8ggdh1wdYeW34c33PvzPU5AG-tMJpbSz2tEixQeCI9oSXzcl9iuNRChpWU0tc0nk6TszMx24Cf7b8wrq2yjYlVoNalcnPk-0j8oeBIZ_zg8spzrlFudbW10KhhcWLWN1iyLT-Mh_h-3wXB8dHpx5HXuAp4inG68nIZ64TlWGpwrhSLNDVUJTIyyHWcxVYjR9pYWIlEHemcuaknyXxLRZ5ozM5DPO892GQO7D3YnI0ns2_drI7LFljo1xJGYSj8_XzOMSmjzsbtN-Kr_AH-CP8Vpx1v_W9P4zE8arJncljD_QlsmOIpbLXOFKQJVM_gejBHdtZErsmwalE5IEeNrHlxTtxOkheaDDqNAjJs-1PIvCBpOiGHRVHWbQqktOQT5tNe1XhUHfh5XWDijBdBRuXKXJB6hWX5HL7cyd2_gF5RFuYlEMGpZTZkOSaAzGLVLCOfCyV17EeJUboP79vXn6lGft25gFxkWIY5pGQdUvrwpht6WWuO_G3QwGGoG-BkwqsN5eI8a6JOFsScqigKjAkEU9SJP2phk0DFTIXUxn3YbeGVNbFrmf3C1va_d7-GB6PTSZql4-nJDjwMnDNy1Rq5C73V4ofZg_vqejVfLl41nwmB73eNxVu5lU-N
linkToPdf http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V3LbtNAFL0qBSE2lKcIFBghkNhY8djjsWeBqoY0atUQRbxUsXE9rypSNS5JKMqv8XXcGT9AArHrgq09tvw4vufemeNzAV5aYTS3lkZaFVig8EJGQksWVbHEcKmFTIOV0udpPpsVJydivgU_un9hvKyyi4khUOta-TnyIRJ_KjjSGR_aVhYxH0_2Lr5GvoOUX2nt2mk0EDk2m-9Yvq3eHI3xXb9KksnBx7eHUdthIFKM03VUyVwXrMKyg3OlWKapoaqQmUHe4yy3GvnS5sJKJO1MV8xPQ0kWWyqqQmOmnuJ5r8H1HGtMLyecZ1_6-R2fN7A0bsyM0lTEw2rBMT2jvqHbbxQYOgX8QQSB3SY7__NzuQO325ya7DcfwV3YMu4e7HT9Kkgbvu7D5WiBnK2J3JBxEK7skYPW7NydEb-TVE6TUe9cQMadaoUsHJlO35F95-pGvEBqS95jlh0FOVI48MPGYTqNF0EO67U5J826y-oBfLqSu38I26525hEQwallNmUVpoXMYi0ts5gLJXUeZ4VRegCvOyiUqjVl971Bzksszjxqyh41A3jRD71onEj-Nmjk8dQP8ObhYUO9PCvbWFQmOacqyxJjEsEU9ZaQWtgiUTlTKbX5AHY7qJVtRFuVv3D2-N-7n8NNBGA5PZodP4FbiW-XHPSSu7C9Xn4zT-GGulwvVstn4XshcHrVQPwJ1uxW8A
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Biased+by+Design%3F+Evaluating+Bias+and+Behavioral+Diversity+in+LLM+Annotation+of+Real-World+and+Synthetic+Hotel+Reviews&rft.jtitle=AI+%28Basel%29&rft.au=Maria+C.+Voutsa&rft.au=Nicolas+Tsapatsoulis&rft.au=Constantinos+Djouvas&rft.date=2025-08-01&rft.pub=MDPI+AG&rft.eissn=2673-2688&rft.volume=6&rft.issue=8&rft.spage=178&rft_id=info:doi/10.3390%2Fai6080178&rft.externalDBID=DOA&rft.externalDocID=oai_doaj_org_article_2761c552ee294c10877d9f82c74c31f7
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2673-2688&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2673-2688&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2673-2688&client=summon