Safeguarding Online Communications using DistilRoBERTa for Detection of Terrorism and Offensive Chats

People use social media for both good and distasteful purposes. When used with malicious intent, it raises significant concerns as it involves the use of offensive language and hate speech that promote terrorism and other negative behaviors. To create a safe, secure and pleasant environment, these c...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Journal of Information Security and Cybercrimes Research (Online) Jg. 7; H. 1; S. 93 - 107
Hauptverfasser: Shah, Mohamed Safwan Saalik, Abuaieta, Amr Mohamed, Almazrouei, Shaima Saeed
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Naif University Publishing House 30.06.2024
Schlagworte:
distilroberta model
large language models
offensive language
social media
ISSN:1658-7782, 1658-7790
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Abstract People use social media for both good and distasteful purposes. When used with malicious intent, it raises significant concerns as it involves the use of offensive language and hate speech that promote terrorism and other negative behaviors. To create a safe, secure and pleasant environment, these communications must be closely monitored to prevent severe problems, associated risks and other pertinent issues. With the help of AI, specifically Large Language Models (LLM), we can quickly analyze text and speech to determine whether the communications promote the dangers identified here above not to mention other toxic elements. For this research, the LLM used is the DistilRoBERTa model from the Transformers library using Hugging Face. The DistilRoBERTa model was trained on datasets consisting of terrorism-related conversations, offensive-related conversations, and neutral conversations. These datasets were obtained from publicly available sources. The results of the experimentation show that the model achieved 99% accuracy, precision, recall, F1 score, and ROC curve. To improve the robustness of the model, it must be continuously fine-tuned to predict dynamic communication behavior since real conversations are inaccessible due to restrictions. A drag-and-drop interface is used to upload the files and get the categorical output, ensuring seamless and easy interaction.
AbstractList People use social media for both good and distasteful purposes. When used with malicious intent, it raises significant concerns as it involves the use of offensive language and hate speech that promote terrorism and other negative behaviors. To create a safe, secure and pleasant environment, these communications must be closely monitored to prevent severe problems, associated risks and other pertinent issues. With the help of AI, specifically Large Language Models (LLM), we can quickly analyze text and speech to determine whether the communications promote the dangers identified here above not to mention other toxic elements. For this research, the LLM used is the DistilRoBERTa model from the Transformers library using Hugging Face. The DistilRoBERTa model was trained on datasets consisting of terrorism-related conversations, offensive-related conversations, and neutral conversations. These datasets were obtained from publicly available sources. The results of the experimentation show that the model achieved 99% accuracy, precision, recall, F1 score, and ROC curve. To improve the robustness of the model, it must be continuously fine-tuned to predict dynamic communication behavior since real conversations are inaccessible due to restrictions. A drag-and-drop interface is used to upload the files and get the categorical output, ensuring seamless and easy interaction.
Author Shah, Mohamed Safwan Saalik
Almazrouei, Shaima Saeed
Abuaieta, Amr Mohamed
Author_xml – sequence: 1
  givenname: Mohamed Safwan Saalik
  surname: Shah
  fullname: Shah, Mohamed Safwan Saalik
  organization: Middlesex University Dubai, Dubai, UAE
– sequence: 2
  givenname: Amr Mohamed
  surname: Abuaieta
  fullname: Abuaieta, Amr Mohamed
  organization: Digital Forensics Expert at International Center for Forensic Science and Criminology, Dubai Police, Dubai, UAE
– sequence: 3
  givenname: Shaima Saeed
  surname: Almazrouei
  fullname: Almazrouei, Shaima Saeed
  organization: Digital Forensics Expert at International Center for Forensic Science and Criminology, Dubai Police, Dubai, UAE
BookMark eNplkEtPQjEQhRuDiYgs_AfdurjS3gdtlwr4SIgkSNg2vdMWa-5tTXsx8d_LQ1zoaiYzZ745OZeo54M3CF1TcpuPWVGN1i_rZc4EPUN9Oq54xpggvd-e5xdomJKrSUVYUTBS9pF5VdZstipq5zd44RvnDZ6Ett16B6pzwSe8Tfvd1KXONctwP1uuFLYh4qnpDOwlOFi8MjGG6FKLldd4Ya3xyX3uWG-qS1fo3KommeFPHaDVw2w1ecrmi8fnyd08A8oqmjHKK0I4WG0t1zXRNae0pKIUWtScAQVrwBA93lm3vKZAcqIJlFQVhYC8GKDnI1YH9S4_omtV_JJBOXkYhLiRKnYOGiM5FYIb0FDXoqyBi9zsvjEBpRa0EHvW6MiCGFKKxkpw3SGQLirXSErkIXN5ynx3cfPn4uTgv_YbakSEwQ
CitedBy_id crossref_primary_10_1109_ACCESS_2025_3576853
crossref_primary_10_3390_info16050342
crossref_primary_10_12688_openreseurope_19612_1
crossref_primary_10_1109_ACCESS_2024_3470901
Cites_doi 10.3390/electronics12183785
10.1109/bigdata52589.2021.9671823
10.1109/discover55800.2022.9974631
10.37394/23209.2023.20.2
10.32604/cmc.2022.019189
10.1155/2023/4563145
10.1109/access.2022.3227962
10.1109/idicaiei58380.2023.10406633
10.3390/app10238614
10.3390/app10186527
10.1109/icaect60202.2024.10469467
10.1109/access.2021.3068313
10.7717/peerj-cs.1934/table-7
10.1145/3625007.3627317
ContentType Journal Article
DBID AAYXX
CITATION
DOA
DOI 10.26735/VNVR2791
DatabaseName CrossRef
DOAJ Directory of Open Access Journals
DatabaseTitle CrossRef
DatabaseTitleList CrossRef
Database_xml – sequence: 1
  dbid: DOA
  name: DOAJ Directory of Open Access Journals
  url: https://www.doaj.org/
  sourceTypes: Open Website
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 1658-7790
EndPage 107
ExternalDocumentID oai_doaj_org_article_81998ecdcbb94bc892edb879c4d91392
10_26735_VNVR2791
GroupedDBID AAYXX
ALMA_UNASSIGNED_HOLDINGS
CITATION
GROUPED_DOAJ
ID FETCH-LOGICAL-c1751-7185008cfdff8db0db81141949d9b87c1cfece0d6704f8b1c020d0c41a339c23
IEDL.DBID DOA
ISSN 1658-7782
IngestDate Fri Oct 03 12:52:31 EDT 2025
Sat Nov 29 03:29:11 EST 2025
Tue Nov 18 21:51:26 EST 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 1
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c1751-7185008cfdff8db0db81141949d9b87c1cfece0d6704f8b1c020d0c41a339c23
OpenAccessLink https://doaj.org/article/81998ecdcbb94bc892edb879c4d91392
PageCount 15
ParticipantIDs doaj_primary_oai_doaj_org_article_81998ecdcbb94bc892edb879c4d91392
crossref_citationtrail_10_26735_VNVR2791
crossref_primary_10_26735_VNVR2791
PublicationCentury 2000
PublicationDate 2024-6-30
PublicationDateYYYYMMDD 2024-06-30
PublicationDate_xml – month: 06
  year: 2024
  text: 2024-6-30
  day: 30
PublicationDecade 2020
PublicationTitle Journal of Information Security and Cybercrimes Research (Online)
PublicationYear 2024
Publisher Naif University Publishing House
Publisher_xml – name: Naif University Publishing House
References ref13
ref12
ref15
ref14
ref20
ref11
ref10
ref21
ref0
ref2
ref1
ref17
ref16
ref19
ref18
ref8
ref7
ref9
ref4
ref3
ref6
ref5
References_xml – ident: ref5
  doi: 10.3390/electronics12183785
– ident: ref9
  doi: 10.1109/bigdata52589.2021.9671823
– ident: ref12
  doi: 10.1109/discover55800.2022.9974631
– ident: ref1
– ident: ref7
– ident: ref8
  doi: 10.37394/23209.2023.20.2
– ident: ref20
– ident: ref6
  doi: 10.32604/cmc.2022.019189
– ident: ref11
  doi: 10.1155/2023/4563145
– ident: ref0
  doi: 10.1109/access.2022.3227962
– ident: ref10
  doi: 10.1109/idicaiei58380.2023.10406633
– ident: ref21
– ident: ref14
  doi: 10.3390/app10238614
– ident: ref2
  doi: 10.3390/app10186527
– ident: ref19
– ident: ref3
  doi: 10.1109/icaect60202.2024.10469467
– ident: ref4
  doi: 10.1109/access.2021.3068313
– ident: ref18
  doi: 10.7717/peerj-cs.1934/table-7
– ident: ref16
– ident: ref13
  doi: 10.1145/3625007.3627317
– ident: ref17
– ident: ref15
SSID ssib050733704
ssj0002910551
Score 2.2607446
Snippet People use social media for both good and distasteful purposes. When used with malicious intent, it raises significant concerns as it involves the use of...
SourceID doaj
crossref
SourceType Open Website
Enrichment Source
Index Database
StartPage 93
SubjectTerms distilroberta model
large language models
offensive language
social media
Title Safeguarding Online Communications using DistilRoBERTa for Detection of Terrorism and Offensive Chats
URI https://doaj.org/article/81998ecdcbb94bc892edb879c4d91392
Volume 7
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVAON
  databaseName: DOAJ Directory of Open Access Journals
  customDbUrl:
  eissn: 1658-7790
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0002910551
  issn: 1658-7782
  databaseCode: DOA
  dateStart: 20180101
  isFulltext: true
  titleUrlDefault: https://www.doaj.org/
  providerName: Directory of Open Access Journals
– providerCode: PRVHPJ
  databaseName: ROAD: Directory of Open Access Scholarly Resources
  customDbUrl:
  eissn: 1658-7790
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssib050733704
  issn: 1658-7782
  databaseCode: M~E
  dateStart: 20180101
  isFulltext: true
  titleUrlDefault: https://road.issn.org
  providerName: ISSN International Centre
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LS8NAEF6kePDiW6wvFvHgJTS72SS7R_vCg1appfQWsq8qlETa6tHf7uwmLRUEL15yCMsSvszOfAOz34fQDTdpwqCyBEQyEbBYJ0GehjagcZSHkhghc6-u_5AOBnwyEc8bVl9uJqySB66Aa3F3CcworaQUTCouqNGSp0Ix7RQtffYF1rPRTEEkxc6KMK0bDZeTqXBGkL77gpILlJLTSmaIJmkUt8aD8ZB6nc6N4rSh4e-LTX8f7dYsEd9VX3eAtkxxiPZWDgy4PpBHyLzk1kz9Xy6muJINxT_ufCywm2yf4q47y7Nh2e4NRzkGpoq7ZunHsApcWjwy83np5ARxXmj8ZG011447r_lycYxG_d6ocx_UxgmBAjZAAqg3MdR2ZbW1XMsQAIO2hwgmtADsFFHWKBPqBCCyXBIFnFGHipE8ioSi0QlqFGVhThEmlGoWWZIITZjimrtdpYEsaWySMNtEtyvAMlWLijtvi1kGzYXHNlth20TX66XvlZLGb4vaDvX1Aid-7V9ASGR1SGR_hcTZf2xyjnYo8JdqNPACNZbzD3OJttXn8m0xv_LRBs_Hr943rnbZUw
linkProvider Directory of Open Access Journals
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Safeguarding+Online+Communications+using+DistilRoBERTa+for+Detection+of+Terrorism+and+Offensive+Chats&rft.jtitle=Journal+of+Information+Security+and+Cybercrimes+Research+%28Online%29&rft.au=Mohamed+Safwan+Saalik+Shah&rft.au=Amr+Mohamed+Abuaieta&rft.au=Shaima+Saeed+Almazrouei&rft.date=2024-06-30&rft.pub=Naif+University+Publishing+House&rft.issn=1658-7782&rft.eissn=1658-7790&rft.volume=7&rft.issue=1&rft.spage=93&rft.epage=107&rft_id=info:doi/10.26735%2FVNVR2791&rft.externalDBID=DOA&rft.externalDocID=oai_doaj_org_article_81998ecdcbb94bc892edb879c4d91392
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1658-7782&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1658-7782&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1658-7782&client=summon