Biased by Design? Evaluating Bias and Behavioral Diversity in LLM Annotation of Real-World and Synthetic Hotel Reviews

As large language models (LLMs) gain traction among researchers and practitioners, particularly in digital marketing for tasks such as customer feedback analysis and automated communication, concerns remain about the reliability and consistency of their outputs. This study investigates annotation bi...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	AI (Basel) Jg. 6; H. 8; S. 178
Hauptverfasser:	Voutsa, Maria C., Tsapatsoulis, Nicolas, Djouvas, Constantinos
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	Basel MDPI AG 01.08.2025
Schlagworte:	AI bias Algorithms annotation bias Annotations Artificial intelligence aspect classification Automation Batch processing Behavior Bias Chatbots Datasets Decision making Hate speech Human error Labeling Labels Large language models Marketing Sentences Sentiment analysis Synthetic data topic modeling User generated content
ISSN:	2673-2688, 2673-2688
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Abstract	As large language models (LLMs) gain traction among researchers and practitioners, particularly in digital marketing for tasks such as customer feedback analysis and automated communication, concerns remain about the reliability and consistency of their outputs. This study investigates annotation bias in LLMs by comparing human and AI-generated annotation labels across sentiment, topic, and aspect dimensions in hotel booking reviews. Using the HRAST dataset, which includes 23,114 real user-generated review sentences and a synthetically generated corpus of 2000 LLM-authored sentences, we evaluate inter-annotator agreement between a human expert and three LLMs (ChatGPT-3.5, ChatGPT-4, and ChatGPT-4-mini) as a proxy for assessing annotation bias. Our findings show high agreement among LLMs, especially on synthetic data, but only moderate to fair alignment with human annotations, particularly in sentiment and aspect-based sentiment analysis. LLMs display a pronounced neutrality bias, often defaulting to neutral sentiment in ambiguous cases. Moreover, annotation behavior varies notably with task design, as manual, one-to-one prompting produces higher agreement with human labels than automated batch processing. The study identifies three distinct AI biases—repetition bias, behavioral bias, and neutrality bias—that shape annotation outcomes. These findings highlight how dataset complexity and annotation mode influence LLM behavior, offering important theoretical, methodological, and practical implications for AI-assisted annotation and synthetic content generation.
AbstractList	As large language models (LLMs) gain traction among researchers and practitioners, particularly in digital marketing for tasks such as customer feedback analysis and automated communication, concerns remain about the reliability and consistency of their outputs. This study investigates annotation bias in LLMs by comparing human and AI-generated annotation labels across sentiment, topic, and aspect dimensions in hotel booking reviews. Using the HRAST dataset, which includes 23,114 real user-generated review sentences and a synthetically generated corpus of 2000 LLM-authored sentences, we evaluate inter-annotator agreement between a human expert and three LLMs (ChatGPT-3.5, ChatGPT-4, and ChatGPT-4-mini) as a proxy for assessing annotation bias. Our findings show high agreement among LLMs, especially on synthetic data, but only moderate to fair alignment with human annotations, particularly in sentiment and aspect-based sentiment analysis. LLMs display a pronounced neutrality bias, often defaulting to neutral sentiment in ambiguous cases. Moreover, annotation behavior varies notably with task design, as manual, one-to-one prompting produces higher agreement with human labels than automated batch processing. The study identifies three distinct AI biases—repetition bias, behavioral bias, and neutrality bias—that shape annotation outcomes. These findings highlight how dataset complexity and annotation mode influence LLM behavior, offering important theoretical, methodological, and practical implications for AI-assisted annotation and synthetic content generation.
Author	Djouvas, Constantinos Tsapatsoulis, Nicolas Voutsa, Maria C.
Author_xml	– sequence: 1 givenname: Maria C. orcidid: 0000-0001-7889-3804 surname: Voutsa fullname: Voutsa, Maria C. – sequence: 2 givenname: Nicolas orcidid: 0000-0002-6739-8602 surname: Tsapatsoulis fullname: Tsapatsoulis, Nicolas – sequence: 3 givenname: Constantinos orcidid: 0000-0003-1215-7294 surname: Djouvas fullname: Djouvas, Constantinos
BookMark	eNplUcluFDEQtVCQCCEH_sASJw6d2G63lxPKBok0CIlFHK1qLxOPGjvYnkbz9_RkACE4VaneUk9Vz9FRyskj9JKSs77X5ByiIIpQqZ6gYyZk3zGh1NFf_TN0WuuGEMIGynhPjtF8GaF6h8cdvvY1rtMbfDPDtIUW0xrvQQzJ4Ut_D3PMBSZ8HWdfamw7HBNerd7ji5RyW_g54RzwRw9T9zWXyT0KP-1Su_ctWnybm58WeI7-R32BngaYqj_9VU_Ql7c3n69uu9WHd3dXF6vOckFbB6N0igNTTAhr-eCop1aNg6eKCi6DU1QHqcOopRgccC51P3ISqAblBin6E3R38HUZNuahxG9QdiZDNI-DXNYGypJu8oZJQe0wMO-Z5pYSJaXTQTErue1pkIvXq4PXQ8nft742s8nbkpb4pl-OqYVibL_x_MCyJddafDA2Hq7TCsTJUGL2vzJ_frUoXv-j-J3zf-5Pa-OUGg
CitedBy_id	crossref_primary_10_3390_ai6090236
Cites_doi	10.1145/3613904.3641960 10.2196/preprints.48399 10.1109/TCSS.2019.2914080 10.18653/v1/2023.eacl-main.130 10.1145/3627508.3638322 10.1017/CBO9781139062367 10.1007/978-3-031-25752-0_26 10.3390/ai6050092 10.1057/s41270-025-00379-6 10.1093/OED/4832698884 10.1109/ACCESS.2024.3402809 10.3390/ai6030056 10.18653/v1/2023.emnlp-main.647 10.3390/tourhosp6010036 10.18653/v1/2024.hcinlp-1.8 10.1007/s11042-012-0995-2 10.3390/ai6050106 10.1371/journal.pdig.0000082 10.1609/aaai.v35i16.17671 10.1136/bmj.320.7237.768 10.1177/1609406919899220 10.3115/1613715.1613751 10.1145/1743384.1743478 10.1007/s10462-024-10903-2 10.1007/978-3-031-93429-2_25 10.1080/0960085X.2021.1927212 10.2196/43251 10.1109/IC3INA48034.2019.8949568 10.1109/ICMLA58977.2023.00089 10.1007/978-3-030-75178-4 10.1609/aaai.v34i01.5385 10.2307/2529310 10.1145/3308560.3317590 10.1609/icwsm.v19i1.35837 10.1145/3341161.3342931 10.1079/9781800626607.0011 10.18653/v1/2022.findings-acl.58 10.1080/13683500.2023.2300030 10.18653/v1/D19-1107 10.1016/j.sapharm.2012.04.004 10.1108/JD-05-2019-0102 10.1136/eb-2014-101946 10.1371/journal.pdig.0000278 10.1177/0022242920972933 10.1109/SMAP59435.2023.10255219 10.18653/v1/2024.acl-long.760 10.1145/3716554.3716620 10.1016/j.jclepro.2022.134120 10.1136/jech.2003.008466 10.1016/j.ijhm.2024.104030 10.3233/FAIA240191 10.1016/j.tourman.2010.12.011
ContentType	Journal Article
Copyright	2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml	– notice: 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID	AAYXX CITATION 8FE 8FG ABUWG AFKRA ARAPS AZQEC BENPR BGLVJ CCPQU DWQXO HCIFZ P5Z P62 PHGZM PHGZT PIMPY PKEHL PQEST PQGLB PQQKQ PQUKI PRINS DOA
DOI	10.3390/ai6080178
DatabaseName	CrossRef ProQuest SciTech Collection ProQuest Technology Collection ProQuest Central (Alumni) ProQuest Central UK/Ireland Advanced Technologies & Computer Science Collection ProQuest Central Essentials ProQuest Central ProQuest Technology Collection ProQuest One ProQuest Central Korea SciTech Premium Collection Advanced Technologies & Aerospace Database ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Premium ProQuest One Academic (New) Publicly Available Content Database ProQuest One Academic Middle East (New) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China DOAJ Directory of Open Access Journals
DatabaseTitle	CrossRef Publicly Available Content Database Advanced Technologies & Aerospace Collection Technology Collection ProQuest One Academic Middle East (New) ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Essentials ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Technology Collection ProQuest SciTech Collection ProQuest Central China ProQuest Central Advanced Technologies & Aerospace Database ProQuest One Applied & Life Sciences ProQuest One Academic UKI Edition ProQuest Central Korea ProQuest Central (New) ProQuest One Academic ProQuest One Academic (New)
DatabaseTitleList	Publicly Available Content Database CrossRef
Database_xml	– sequence: 1 dbid: DOA name: Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website – sequence: 2 dbid: PIMPY name: Publicly Available Content Database url: http://search.proquest.com/publiccontent sourceTypes: Aggregation Database
DeliveryMethod	fulltext_linktorsrc
Discipline	Computer Science
EISSN	2673-2688
ExternalDocumentID	oai_doaj_org_article_2761c552ee294c10877d9f82c74c31f7 10_3390_ai6080178
GroupedDBID	AADQD AAYXX ABDBF ACUHS AFFHD AFKRA AFZYC ALMA_UNASSIGNED_HOLDINGS ARAPS BENPR BGLVJ CCPQU CITATION GROUPED_DOAJ HCIFZ IAO ICD IGS ISR ITC MODMG M~E OK1 PHGZM PHGZT PIMPY PQGLB 8FE 8FG ABUWG AZQEC DWQXO P62 PKEHL PQEST PQQKQ PQUKI PRINS
ID	FETCH-LOGICAL-c461t-ab7d84a28266cc45d1e1c8b5e181647fd819f79fb9765da44793b40f19a8d5763
IEDL.DBID	DOA
ISICitedReferencesCount	2
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001557107900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	2673-2688
IngestDate	Fri Oct 03 12:44:07 EDT 2025 Sat Nov 01 15:11:00 EDT 2025 Sat Nov 29 07:13:57 EST 2025 Tue Nov 18 21:46:39 EST 2025
IsDoiOpenAccess	true
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	8
Language	English
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c461t-ab7d84a28266cc45d1e1c8b5e181647fd819f79fb9765da44793b40f19a8d5763
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0003-1215-7294 0000-0001-7889-3804 0000-0002-6739-8602
OpenAccessLink	https://doaj.org/article/2761c552ee294c10877d9f82c74c31f7
PQID	3243968226
PQPubID	5046920
ParticipantIDs	doaj_primary_oai_doaj_org_article_2761c552ee294c10877d9f82c74c31f7 proquest_journals_3243968226 crossref_citationtrail_10_3390_ai6080178 crossref_primary_10_3390_ai6080178
PublicationCentury	2000
PublicationDate	2025-08-01
PublicationDateYYYYMMDD	2025-08-01
PublicationDate_xml	– month: 08 year: 2025 text: 2025-08-01 day: 01
PublicationDecade	2020
PublicationPlace	Basel
PublicationPlace_xml	– name: Basel
PublicationTitle	AI (Basel)
PublicationYear	2025
Publisher	MDPI AG
Publisher_xml	– name: MDPI AG
References	Smith (ref_27) 2014; 17 ref_58 ref_57 ref_12 ref_56 ref_11 ref_55 ref_10 ref_54 Nazer (ref_29) 2023; 2 Chen (ref_7) 2023; 25 ref_19 ref_18 ref_17 ref_16 ref_15 ref_59 Llorca (ref_28) 2004; 58 Giannoulakis (ref_37) 2019; 6 Ntalianis (ref_36) 2014; 69 Landis (ref_53) 1977; 33 Nasution (ref_21) 2024; 12 ref_24 ref_67 ref_66 ref_65 ref_20 ref_64 ref_63 ref_62 Reason (ref_34) 2000; 320 Bacalhau (ref_23) 2025; 13 Kar (ref_22) 2022; 376 ref_26 Kozinets (ref_14) 2020; 85 Jia (ref_49) 2025; 126 Sparks (ref_50) 2011; 32 Varsha (ref_25) 2023; 3 ref_35 Macdonald (ref_60) 2019; 76 ref_33 ref_32 ref_31 ref_30 Donohue (ref_61) 1972; 1 ref_39 ref_38 Wang (ref_2) 2024; 28 Gonzales (ref_13) 2023; 2 Joffe (ref_51) 2020; 19 Akter (ref_5) 2021; 60 ref_46 ref_45 Kordzadeh (ref_6) 2021; 31 ref_44 ref_43 ref_42 ref_41 ref_40 ref_1 ref_3 Vilares (ref_47) 2024; 57 ref_48 ref_9 Gisev (ref_52) 2013; 9 ref_8 ref_4
References_xml	– ident: ref_43 doi: 10.1145/3613904.3641960 – ident: ref_44 doi: 10.2196/preprints.48399 – volume: 6 start-page: 592 year: 2019 ident: ref_37 article-title: Filtering Instagram Hashtags Through Crowdtagging and the HITS Algorithm publication-title: IEEE Trans. Comput. Soc. Syst. doi: 10.1109/TCSS.2019.2914080 – ident: ref_9 doi: 10.18653/v1/2023.eacl-main.130 – ident: ref_41 doi: 10.1145/3627508.3638322 – ident: ref_35 doi: 10.1017/CBO9781139062367 – ident: ref_1 doi: 10.1007/978-3-031-25752-0_26 – ident: ref_30 doi: 10.3390/ai6050092 – volume: 13 start-page: 308 year: 2025 ident: ref_23 article-title: A bibliometric analysis of AI bias in marketing: Field evolution and future research agenda publication-title: J. Mark. Anal. doi: 10.1057/s41270-025-00379-6 – ident: ref_26 doi: 10.1093/OED/4832698884 – ident: ref_39 – volume: 12 start-page: 71876 year: 2024 ident: ref_21 article-title: Chatgpt label: Comparing the quality of human-generated and llm-generated annotations in low-resource language nlp tasks publication-title: IEEE Access doi: 10.1109/ACCESS.2024.3402809 – ident: ref_66 doi: 10.3390/ai6030056 – ident: ref_58 – ident: ref_15 doi: 10.18653/v1/2023.emnlp-main.647 – ident: ref_3 doi: 10.3390/tourhosp6010036 – ident: ref_32 doi: 10.18653/v1/2024.hcinlp-1.8 – volume: 69 start-page: 397 year: 2014 ident: ref_36 article-title: Automatic annotation of image databases based on implicit crowdsourcing, visual concept modeling and evolution publication-title: Multimed. Tools Appl. doi: 10.1007/s11042-012-0995-2 – ident: ref_31 doi: 10.3390/ai6050106 – volume: 2 start-page: e0000082 year: 2023 ident: ref_13 article-title: Synthetic data in health care: A narrative review publication-title: PLoS Digit. Health doi: 10.1371/journal.pdig.0000082 – volume: 3 start-page: 100165 year: 2023 ident: ref_25 article-title: How can we manage biases in artificial intelligence systems—A systematic literature review publication-title: Int. J. Inf. Manag. Data Insights – ident: ref_56 – ident: ref_10 – ident: ref_17 – ident: ref_48 doi: 10.1609/aaai.v35i16.17671 – volume: 60 start-page: 102387 year: 2021 ident: ref_5 article-title: Algorithmic bias in data-driven innovation in the age of AI publication-title: Int. J. Inf. Manag. – ident: ref_20 – ident: ref_59 – volume: 320 start-page: 768 year: 2000 ident: ref_34 article-title: Human error: Models and management publication-title: BMJ doi: 10.1136/bmj.320.7237.768 – volume: 19 start-page: 1609406919899220 year: 2020 ident: ref_51 article-title: Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines publication-title: Int. J. Qual. Methods doi: 10.1177/1609406919899220 – ident: ref_63 doi: 10.3115/1613715.1613751 – ident: ref_64 doi: 10.1145/1743384.1743478 – volume: 57 start-page: 265 year: 2024 ident: ref_47 article-title: Contrasting linguistic patterns in human and LLM-generated news text publication-title: Artif. Intell. Rev. doi: 10.1007/s10462-024-10903-2 – ident: ref_40 doi: 10.1007/978-3-031-93429-2_25 – volume: 31 start-page: 388 year: 2021 ident: ref_6 article-title: Algorithmic bias: Review, synthesis, and future research directions publication-title: Eur. J. Inf. Syst. doi: 10.1080/0960085X.2021.1927212 – volume: 25 start-page: e43251 year: 2023 ident: ref_7 article-title: Human-Centered Design to Address Biases in Artificial Intelligence publication-title: J. Med. Internet Res. doi: 10.2196/43251 – ident: ref_65 doi: 10.1109/IC3INA48034.2019.8949568 – ident: ref_42 doi: 10.1109/ICMLA58977.2023.00089 – ident: ref_16 doi: 10.1007/978-3-030-75178-4 – ident: ref_62 doi: 10.1609/aaai.v34i01.5385 – ident: ref_67 – volume: 33 start-page: 159 year: 1977 ident: ref_53 article-title: The Measurement of Observer Agreement for Categorical Data publication-title: Biometrics doi: 10.2307/2529310 – ident: ref_24 doi: 10.1145/3308560.3317590 – ident: ref_12 doi: 10.1609/icwsm.v19i1.35837 – ident: ref_33 doi: 10.1145/3341161.3342931 – ident: ref_4 doi: 10.1079/9781800626607.0011 – ident: ref_55 doi: 10.18653/v1/2022.findings-acl.58 – volume: 28 start-page: 527 year: 2024 ident: ref_2 article-title: Personalizing guest experience with generative AI in the hotel industry: There’s more to it than meets a Kiwi’s eye publication-title: Curr. Issues Tour. doi: 10.1080/13683500.2023.2300030 – ident: ref_38 doi: 10.18653/v1/D19-1107 – ident: ref_54 – ident: ref_46 – volume: 9 start-page: 330 year: 2013 ident: ref_52 article-title: Interrater agreement and interrater reliability: Key concepts, approaches, and applications publication-title: Res. Soc. Adm. Pharm. doi: 10.1016/j.sapharm.2012.04.004 – volume: 1 start-page: 41 year: 1972 ident: ref_61 article-title: Gatekeeping: Mass media systems and information control publication-title: Curr. Perspect. Mass Commun. Res. – volume: 76 start-page: 333 year: 2019 ident: ref_60 article-title: The concept of neutrality: A new approach publication-title: J. Doc. doi: 10.1108/JD-05-2019-0102 – volume: 17 start-page: 100 year: 2014 ident: ref_27 article-title: Bias in research publication-title: Evid. Based Nurs. doi: 10.1136/eb-2014-101946 – volume: 2 start-page: e0000278 year: 2023 ident: ref_29 article-title: Bias in artificial intelligence algorithms and recommendations for mitigation publication-title: PLoS Digit. Health doi: 10.1371/journal.pdig.0000278 – volume: 85 start-page: 156 year: 2020 ident: ref_14 article-title: Commentary: Artificial Intelligence: The Marketer’s Dilemma publication-title: J. Mark. doi: 10.1177/0022242920972933 – ident: ref_18 doi: 10.1109/SMAP59435.2023.10255219 – ident: ref_11 doi: 10.18653/v1/2024.acl-long.760 – ident: ref_45 doi: 10.1145/3716554.3716620 – volume: 376 start-page: 134120 year: 2022 ident: ref_22 article-title: How can artificial intelligence impact sustainability: A systematic literature review publication-title: J. Clean. Prod. doi: 10.1016/j.jclepro.2022.134120 – ident: ref_19 – volume: 58 start-page: 635 year: 2004 ident: ref_28 article-title: Bias publication-title: J. Epidemiol. Community Health doi: 10.1136/jech.2003.008466 – ident: ref_57 – volume: 126 start-page: 104030 year: 2025 ident: ref_49 article-title: Unpacking the impact of AI vs. human-generated review summary on hotel booking intentions publication-title: Int. J. Hosp. Manag. doi: 10.1016/j.ijhm.2024.104030 – ident: ref_8 doi: 10.3233/FAIA240191 – volume: 32 start-page: 1310 year: 2011 ident: ref_50 article-title: The impact of online reviews on hotel booking intentions and perception of trust publication-title: Tour. Manag. doi: 10.1016/j.tourman.2010.12.011
SSID	ssj0002512430
Score	2.3128521
Snippet	As large language models (LLMs) gain traction among researchers and practitioners, particularly in digital marketing for tasks such as customer feedback...
SourceID	doaj proquest crossref
SourceType	Open Website Aggregation Database Enrichment Source Index Database
StartPage	178
SubjectTerms	AI bias Algorithms annotation bias Annotations Artificial intelligence aspect classification Automation Batch processing Behavior Bias Chatbots Datasets Decision making Hate speech Human error Labeling Labels Large language models Marketing Sentences Sentiment analysis Synthetic data topic modeling User generated content
SummonAdditionalLinks	– databaseName: Advanced Technologies & Aerospace Database dbid: P5Z link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1La9wwEBZt2kMvTZ9007QMpYdeTFaWLFmnkG0SckhD6ANCL8Z6OCwEOd3dBvLvMyPLbqGll14tGQSj0XwzGn0fY--5nWuPYauojMAEJQhbmFb4orRocGI818ElsQl9dlZfXJjzXHBb57bK8UxMB7XvHdXI9zDwC6MwnKn96x8FqUbR7WqW0LjPHhBLAkk3nFffpxoLxW4p5gOhkMDsfq9dKoRInETVfgtDia3_j8M4RZjj7f9d2xP2OGNLOBg2w1N2L8RnbHvUbYDsxs_ZzWKJscuDvYXD1MCxD0eZ9DteAg1CGz0sphf8cDh2b8AywunpJziIsR8u8aHv4DOizSK15aQfv9xGhJW4CDjpN-EKhvuH9Qv27fjo68eTIusvFE4qvilaq30tW0zKlHJOVp4H7mpbBUQFSurOI5rotOksQprKt5KKdFbOO27a2mMeI16yrdjH8IqB6ZTF1IheulrpDDcecZd2lSiNF1LUM_ZhNEfjMjk5aWRcNZikkOWayXIz9m6aej0wcvxt0oJsOk0gEu30oV9dNtknm1Ir7qqqDKE00nGiRvSmq0unpRO80zO2O5q7yZ69bn7Zeuffw6_Zo5K0glOz4C7b2qx-hjfsobvZLNert2mj3gFtSPB1 priority: 102 providerName: ProQuest
Title	Biased by Design? Evaluating Bias and Behavioral Diversity in LLM Annotation of Real-World and Synthetic Hotel Reviews
URI	https://www.proquest.com/docview/3243968226 https://doaj.org/article/2761c552ee294c10877d9f82c74c31f7
Volume	6
WOSCitedRecordID	wos001557107900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVAON databaseName: Directory of Open Access Journals customDbUrl: eissn: 2673-2688 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0002512430 issn: 2673-2688 databaseCode: DOA dateStart: 20200101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources (ISSN International Center) customDbUrl: eissn: 2673-2688 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0002512430 issn: 2673-2688 databaseCode: M~E dateStart: 20190101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre – providerCode: PRVPQU databaseName: ProQuest advanced technologies & aerospace journals customDbUrl: eissn: 2673-2688 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0002512430 issn: 2673-2688 databaseCode: P5Z dateStart: 20200101 isFulltext: true titleUrlDefault: https://search.proquest.com/hightechjournals providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Central customDbUrl: eissn: 2673-2688 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0002512430 issn: 2673-2688 databaseCode: BENPR dateStart: 20200101 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest – providerCode: PRVPQU databaseName: Publicly Available Content Database customDbUrl: eissn: 2673-2688 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0002512430 issn: 2673-2688 databaseCode: PIMPY dateStart: 20200101 isFulltext: true titleUrlDefault: http://search.proquest.com/publiccontent providerName: ProQuest
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LS8QwEA6iHrz4FtcXg3jwUtw0adOcxNUVBV2KD1AvpXlUFpasuKvgv3eSdpcFBS9eekhTWiaTzPc1k28IOaKqLQyGrSiRDAmKZSqSJTNRrHDAveK5sDoUmxC9Xvb0JPOZUl8-J6yWB64NdxIjz9ZJElsbS66p168zsspiLbhmtArnyNtCzpApvwb7qM1Zu5YSYsjrT8p-iuCI-nJqMwEo6PT_WIZDbLlcJcsNKISz-mPWyJx162RlUnABmvm3QT47fQw6BtQXXITMi1PoNmrd7hX8TSidgc706D1cTNIuoO_g5uYWzpwb1rvvMKzgDmFiFPJpwoP3Xw7xIH4EXA3HdgD1xsFokzxedh_Or6KmcEKkeUrHUamEyXiJbCpNteaJoZbqTCUWw3nKRWUQBlRCVgqxSGJK7v-uKd6uqCwzgwSEbZF5N3R2m4CsUoWcxh9RVVxLKg0CJqETFkvDOMta5HhizUI3quK-uMWgQHbhDV9MDd8ih9Oub7WUxm-dOn5Iph28-nVoQJ8oGp8o_vKJFtmbDGjRTMlRgciRyRTxULrzH-_YJUuxLwUccgH3yPz4_cPuk0X9Oe6P3g_IQqfby-8OglfiNU9esC2_vs2fvwEjwOWq
linkProvider	Directory of Open Access Journals
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1Nb9NAEB21KVK5tHyKlAIrBFIvVrPe9a73gKqGtErUJIpokcrJ9X64ilTZJQlF-VP8Rmb9BRKIWw9c7bVle5_nzazH7wG8o7onLdJWECmGBYpjOlAps0GoccK94rl0pjSbkNNpfHmpZhvwo_kXxrdVNjGxDNS2MH6N_BCJnymBdCaObr8G3jXKf11tLDQqWJy59Xcs2ZYfRgOc3_dheHpy8XEY1K4CgeGCroJUSxvzFEsNIYzhkaWOmlhHDrlOcJlZ5MhMqkwjUUc25X7pSfNeRlUaW8zOGZ53E7a4B3sHtmajyexLu6rjswXOepWEEWOqd5jOBSZl1Nu4_UZ8pT_AH-G_5LTT3f_taTyCnTp7JscV3B_DhsufwG7jTEHqQPUU7vpzZGdL9JoMyhaVI3JSy5rn18TvJGluSb_VKCCDpj-FzHMyHk_IcZ4XVZsCKTLyCfPpoGw8Kg88X-eYOONFkGGxcjek-sKyfAaf7-Xun0MnL3L3AojKhMbiz__Lq7lRVFnMLKWJWKgs4yzuwkEz_Ymp5de9C8hNgmWYR0rSIqULb9uht5XmyN8G9T2G2gFeJrzcUCyukzrqJKEU1ERR6FyouKFe_NGqLA6N5IbRTHZhv4FXUseuZfILW3v_3v0GtocXk3EyHk3PXsLD0Dsjl62R-9BZLb65V_DA3K3my8Xr-jUhcHXfWPwJTORN1w
linkToPdf	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1Lb9QwEB6VglAvlKe6pQULgcQl2vUjcXxAVZftqlWX1YqHVHEJ8SPVSlXS7i5F-9f4dR07D5BA3HrgGjtREn-eb8YezwfwmuqBtEhbUaw4BiiO60jl3EZM44D7iufSmSA2IafT9OxMzTbgZ3sWxqdVtjYxGGpbGb9G3kfi5ypBOkv6RZMWMRuNDy6vIq8g5XdaWzmNGiKnbv0Dw7flu5MRjvUbxsZHn98fR43CQGREQldRrqVNRY5hR5IYI2JLHTWpjh3yXiJkYZEvC6kKjaQd21z4ZSgtBgVVeWrRU-f43DtwV2KM6dMJZ_HXbn3H-w2CD-piRpyrQT-fJ-ieUS_o9hsFBqWAP4ggsNt4-3_-Lw_hQeNTk8N6EjyCDVc-hu1Wr4I05usJXA_nyNmW6DUZhcSVA3LUFDsvz4lvJHlpybCrXEBGbdYKmZdkMvlADsuyqpMXSFWQj-hlRyEdKdz4aV2iO40vQY6rlbsg9b7L8il8uZWvfwabZVW6HSCqSDSGhP6ErxZGUWXR35Qm5kxZLnjag7ctFDLTFGX32iAXGQZnHjVZh5oevOq6XtaVSP7Waejx1HXwxcPDhWpxnjW2KGMyoSaOmXNMCUN9SUiripQZKQynhezBXgu1rLFoy-wXznb_3fwS7iMAs8nJ9PQ5bDEvlxzyJfdgc7X47vbhnrlezZeLF2G-EPh220C8AVm1VTo
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Biased+by+Design%3F+Evaluating+Bias+and+Behavioral+Diversity+in+LLM+Annotation+of+Real-World+and+Synthetic+Hotel+Reviews&rft.jtitle=AI+%28Basel%29&rft.au=Voutsa%2C+Maria+C.&rft.au=Tsapatsoulis%2C+Nicolas&rft.au=Djouvas%2C+Constantinos&rft.date=2025-08-01&rft.issn=2673-2688&rft.eissn=2673-2688&rft.volume=6&rft.issue=8&rft.spage=178&rft_id=info:doi/10.3390%2Fai6080178&rft.externalDBID=n%2Fa&rft.externalDocID=10_3390_ai6080178
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2673-2688&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2673-2688&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2673-2688&client=summon