An inter-modal attention-based deep learning framework using unified modality for multimodal fake news, hate speech and offensive language detection
Fake news, hate speech and offensive language are related evil triplets currently affecting modern societies. Text modality for the computational detection of these phenomena has been widely used. In recent times, multimodal studies in this direction are attracting a lot of interests because of the...
Uloženo v:
| Vydáno v: | Information systems (Oxford) Ročník 123; s. 102378 |
|---|---|
| Hlavní autoři: | , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Elsevier Ltd
01.07.2024
|
| Témata: | |
| ISSN: | 0306-4379, 1873-6076 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Fake news, hate speech and offensive language are related evil triplets currently affecting modern societies. Text modality for the computational detection of these phenomena has been widely used. In recent times, multimodal studies in this direction are attracting a lot of interests because of the potentials offered by other modalities in contributing to the detection of these menaces. However, a major problem in multimodal content understanding is how to effectively model the complementarity of the different modalities due to their diverse characteristics and features. From a multimodal point of view, the three tasks have been studied mainly using image and text modalities. Improving the effectiveness of the diverse multimodal approaches is still an open research topic. In addition to the traditional text and image modalities, we consider image–texts which are rarely used in previous studies but which contain useful information for enhancing the effectiveness of a prediction model. In order to ease multimodal content understanding and enhance prediction, we leverage recent advances in computer vision and deep learning for these tasks. First, we unify the modalities by creating a text representation of the images and image–texts, in addition to the main text. Secondly, we propose a multi-layer deep neural network with inter-modal attention mechanism to model the complementarity among these modalities. We conduct extensive experiments involving three standard datasets covering the three tasks. Experimental results show that detection of fake news, hate speech and offensive language can benefit from this approach. Furthermore, we conduct robust ablation experiments to show the effectiveness of our approach. Our model predominantly outperforms prior works across the datasets.
•A unified deep learning model can be used for multimodal fake news, hate speech and offensive language detection.•Unifying modalities is useful for multimodal content understanding.•Inter-modal attention mechanism is effective for multimodal-based deep learning models.•The inter-modal attention deep learning framework is effective for fake news, hate speech and offensive language detection.•Incorporation of image-texts as additional modality improves performance. The model can be tuned to use desired number of modalities. |
|---|---|
| AbstractList | Fake news, hate speech and offensive language are related evil triplets currently affecting modern societies. Text modality for the computational detection of these phenomena has been widely used. In recent times, multimodal studies in this direction are attracting a lot of interests because of the potentials offered by other modalities in contributing to the detection of these menaces. However, a major problem in multimodal content understanding is how to effectively model the complementarity of the different modalities due to their diverse characteristics and features. From a multimodal point of view, the three tasks have been studied mainly using image and text modalities. Improving the effectiveness of the diverse multimodal approaches is still an open research topic. In addition to the traditional text and image modalities, we consider image–texts which are rarely used in previous studies but which contain useful information for enhancing the effectiveness of a prediction model. In order to ease multimodal content understanding and enhance prediction, we leverage recent advances in computer vision and deep learning for these tasks. First, we unify the modalities by creating a text representation of the images and image–texts, in addition to the main text. Secondly, we propose a multi-layer deep neural network with inter-modal attention mechanism to model the complementarity among these modalities. We conduct extensive experiments involving three standard datasets covering the three tasks. Experimental results show that detection of fake news, hate speech and offensive language can benefit from this approach. Furthermore, we conduct robust ablation experiments to show the effectiveness of our approach. Our model predominantly outperforms prior works across the datasets.
•A unified deep learning model can be used for multimodal fake news, hate speech and offensive language detection.•Unifying modalities is useful for multimodal content understanding.•Inter-modal attention mechanism is effective for multimodal-based deep learning models.•The inter-modal attention deep learning framework is effective for fake news, hate speech and offensive language detection.•Incorporation of image-texts as additional modality improves performance. The model can be tuned to use desired number of modalities. |
| ArticleNumber | 102378 |
| Author | Ayetiran, Eniafe Festus Özgöbek, Özlem |
| Author_xml | – sequence: 1 givenname: Eniafe Festus orcidid: 0000-0002-6816-2781 surname: Ayetiran fullname: Ayetiran, Eniafe Festus email: eniafe.ayetiran@ntnu.no organization: Department of Computer Science, Norwegian University of Science and Technology, Trondheim, Norway – sequence: 2 givenname: Özlem orcidid: 0000-0003-2612-2009 surname: Özgöbek fullname: Özgöbek, Özlem email: ozlem.ozgobek@ntnu.no organization: Department of Computer Science, Norwegian University of Science and Technology, Trondheim, Norway |
| BookMark | eNp9kM1OwzAQhC0EEqVw5-gHIMVO0qTlhhB_EhIXOFsbZ7fdNnUq2wXxHjwwDuWEBKfVSPPt7M6JOHS9QyHOtZpopavL1YTDJFd5mWRe1LMDMdKzusgqVVeHYqQKVWVlUc-PxUkIK6VUPp3PR-Lz2kl2EX226VvoJMSILnLvsgYCtrJF3MoOwTt2C0keNvje-7XchUHvHBMn1zfL8UNS7-Vm10XebyNYo3T4Hi7kEiLKsEW0SwmulT0RusBvKDtwix0sMGVFtEP2qTgi6AKe_cyxeL27fbl5yJ6e7x9vrp8yWxRVzBpFDc51U4MFymnaltDUClWpCW1uc43JYbGpdPKUDWByaCI1o5aUnlIxFtV-r_V9CB7JWI4wXBA9cGe0MkO3ZmU4mKFbs-82geoXuPW8Af_xH3K1RzA99MboTbCMzmLLPn1t2p7_hr8AECWYMw |
| CitedBy_id | crossref_primary_10_1145_3748326 crossref_primary_10_1016_j_eswa_2025_129756 crossref_primary_10_3390_e26121114 crossref_primary_10_1051_itmconf_20257802017 crossref_primary_10_2478_nor_2025_0015 crossref_primary_10_1007_s13042_025_02715_9 crossref_primary_10_1109_ACCESS_2024_3406258 crossref_primary_10_1007_s42979_024_03280_8 crossref_primary_10_1016_j_asoc_2025_113277 crossref_primary_10_1007_s44379_025_00033_z crossref_primary_10_1016_j_asoc_2024_112358 crossref_primary_10_1016_j_knosys_2025_113249 crossref_primary_10_1108_DTA_06_2023_0230 crossref_primary_10_1016_j_aej_2025_03_071 crossref_primary_10_1016_j_rineng_2025_104752 crossref_primary_10_1016_j_inffus_2025_103628 crossref_primary_10_1016_j_neucom_2025_131118 crossref_primary_10_1007_s12559_024_10356_3 |
| Cites_doi | 10.1016/j.knosys.2021.106902 10.1162/neco.1997.9.8.1735 10.1016/j.inffus.2022.12.016 10.1016/0031-3203(82)90024-3 10.1016/j.neucom.2020.10.042 10.1016/j.knosys.2022.109409 10.1089/big.2020.0062 10.1016/j.ipm.2021.102610 10.1016/j.ipm.2023.103474 |
| ContentType | Journal Article |
| Copyright | 2024 The Author(s) |
| Copyright_xml | – notice: 2024 The Author(s) |
| DBID | 6I. AAFTH AAYXX CITATION |
| DOI | 10.1016/j.is.2024.102378 |
| DatabaseName | ScienceDirect Open Access Titles Elsevier:ScienceDirect:Open Access CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Computer Science |
| EISSN | 1873-6076 |
| ExternalDocumentID | 10_1016_j_is_2024_102378 S030643792400036X |
| GroupedDBID | --K --M -~X .DC .~1 0R~ 13V 1B1 1~. 1~5 29I 4.4 457 4G. 5GY 5VS 63O 6I. 7-5 71M 77K 8P~ 9JN 9JO AAAKF AAAKG AACTN AAEDT AAEDW AAFTH AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AARIN AAXUO AAYFN ABBOA ABFNM ABKBG ABMAC ABMVD ABTAH ABUCO ABXDB ABYKQ ACDAQ ACGFS ACHRH ACNNM ACNTT ACRLP ACZNC ADBBV ADEZE ADJOM ADMUD AEBSH AEKER AENEX AFFNX AFKWA AFTJW AGHFR AGJBL AGUBO AGUMN AGYEJ AHHHB AHZHX AI. AIALX AIEXJ AIKHN AITUG AJOXV AKRWK ALEQD ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD APLSM ASPBG AVWKF AXJTR AZFZN BKOJK BLXMC BNSAS CS3 DU5 EBS EFJIC EJD EO8 EO9 EP2 EP3 F5P FDB FEDTE FGOYB FIRID FNPLU FYGXN G-Q G8K GBLVA GBOLZ HAMUX HF~ HLZ HVGLF HZ~ H~9 IHE J1W KOM LG9 M41 MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. PQQKQ Q38 R2- RIG RNS ROL RPZ SBC SDF SDG SDP SES SEW SPC SPCBC SSB SSD SSL SSV SSZ T5K TN5 UHS VH1 WUQ XSW ZCG ZY4 ~G- 77I 9DU AATTM AAXKI AAYWO AAYXX ABDPE ABJNI ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO ADVLN AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKYEP ANKPU APXCP CITATION EFKBS EFLBG ~HD |
| ID | FETCH-LOGICAL-c336t-b0fbe91b7acaf2f5d4ab70e041fec2c21eb0fceb611b74bae5d41ff08fdf015f3 |
| ISICitedReferencesCount | 17 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001217561900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0306-4379 |
| IngestDate | Tue Nov 18 21:18:50 EST 2025 Sat Nov 29 06:20:20 EST 2025 Sat May 04 15:44:47 EDT 2024 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Inter-modal attention Unified modality BiLSTM-CNN Hate speech Multimodal fusion Offensive language Fake news Multimodal content understanding |
| Language | English |
| License | This is an open access article under the CC BY license. |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c336t-b0fbe91b7acaf2f5d4ab70e041fec2c21eb0fceb611b74bae5d41ff08fdf015f3 |
| ORCID | 0000-0003-2612-2009 0000-0002-6816-2781 |
| OpenAccessLink | https://dx.doi.org/10.1016/j.is.2024.102378 |
| ParticipantIDs | crossref_citationtrail_10_1016_j_is_2024_102378 crossref_primary_10_1016_j_is_2024_102378 elsevier_sciencedirect_doi_10_1016_j_is_2024_102378 |
| PublicationCentury | 2000 |
| PublicationDate | July 2024 2024-07-00 |
| PublicationDateYYYYMMDD | 2024-07-01 |
| PublicationDate_xml | – month: 07 year: 2024 text: July 2024 |
| PublicationDecade | 2020 |
| PublicationTitle | Information systems (Oxford) |
| PublicationYear | 2024 |
| Publisher | Elsevier Ltd |
| Publisher_xml | – name: Elsevier Ltd |
| References | Maity, Jha, Saha, Bhattacharyya (b29) 2022 Pramanick, Sharma, Dimitrov, Akhtar, Nakov, Chakraborty (b34) 2021 Koutlis, Schinas, Papadopoulos (b8) 2023 Radford, Kim, Hallacy, Ramesh, Goh, Agarwal, Sastry, Askell, Mishkin, Clark, Krueger, Sutskever (b37) 2021; vol. 139 Chen, Wang, Liu, Lew (b10) 2021; 426 Hosseinmardi, Rafiq, Han, Lv, Mishra (b26) 2016 Chen, Li, Zhang, Sui, Lv, Lu, Shang (b23) 2022 Suryawanshi, Chakravarthi, Arcan, Buitelaar (b4) 2020 Fukushima, Miyake (b40) 1982; 15 Lahat, Adali, Jutten (b13) 2014 Khattar, Goud, Gupta, Varma (b3) 2019 Bozarth, Budak (b9) 2020 Li, Li, Xiong, Hoi (b38) 2022; vol. 162 Rizzi, Gasparini, Saibene, Rosso, Fersini (b32) 2023; 60 Srivastava, Hinton, Krizhevsky, Sutskever, Salakhutdinov (b48) 2014; 15 Giachanou, Rosso (b1) 2020 Giachanou, Zhang, Rosso (b19) 2020; vol. 12284 Zhong, Wang, Liu (b35) 2022; vol. 13141 Xiong, Zhang, Batra, Xi, Shi, Liu (b25) 2023; 93 Alam, Cresci, Chakraborty, Silvestri, Dimitrov, Martino, Shaar, Firooz, Nakov (b14) 2022 Kim (b41) 2014 Wang, Ma, Jin, Yuan, Xun, Jha, Su, Gao (b15) 2018 Wu, Zhan, Zhang, Wang, Xu (b22) 2021; ACL/IJCNLP 2021 Segura-Bedmar, Alonso-Bartolome (b12) 2022; 13 Xue, Wang, Tian, Li, Shi, Wei (b21) 2021; 58 European Foundation for South Asian Studies (b2) 2021 Fersini, Gasparini, Rizzi, Saibene, Chulvi, Rosso, Lees, Sorensen (b31) 2022 Pennington, Socher, Manning (b46) 2014 D. Kiela, H. Firooz, A. Mohan, V. Goswami, A. Singh, P. Ringshia, D. Testuggine, The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes, in: H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, H. Lin (Eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, Virtual, 2020. Graves, Jaitly, Mohamed (b11) 2013 Ayetiran (b43) 2022; 252 Yang, Zhu, Liu, Han, Hu (b7) 2022 K. Simonyan, A. Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition, in: Y. Bengio, Y. LeCun (Eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. Yang, Peng, Ghosh, Shilon, Ma, Moore, Predovic (b27) 2019 T. Dozat, Incorporating nesterov momentum into adam, in: 4th International Conference on Learning Representations, ICLR 2016 Workshop Track, San Juan, Puerto Rico, USA, May 2-4, 2016, Conference Track Proceedings, 2016. Ayetiran, Sojka, Novotný (b44) 2021; 219 Li, Li, Le, Wang, Savarese, Hoi (b36) 2022 D.P. Kingma, J. Ba, Adam: A Method for Stochastic Optimization, in: Y. Bengio, Y. LeCun (Eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. Hochreiter, Schmidhuber (b39) 1997; 9 Lee, Cao, Fan, Jiang, Chong (b33) 2021 Giachanou, Zhang, Rosso (b20) 2020 Luong, Pham, Manning (b42) 2015 Singhal, Shah, Chakraborty, Kumaraguru, Satoh (b16) 2019 Yang, Zhu, Liu, Han, Hu (b30) 2022 Gomez, Gibert, Gómez, Karatzas (b6) 2020 Ioffe, Szegedy (b49) 2015; vol. 37 Shu, Mahudeswaran, Wang, Lee, Liu (b45) 2020; 8 Zhou, Wu, Zafarani (b5) 2020; 12085 Devlin, Chang, Lee, Toutanova (b17) 2019 Zhang, Giachanou, Rosso (b24) 2024 Xue (10.1016/j.is.2024.102378_b21) 2021; 58 Radford (10.1016/j.is.2024.102378_b37) 2021; vol. 139 Xiong (10.1016/j.is.2024.102378_b25) 2023; 93 Yang (10.1016/j.is.2024.102378_b27) 2019 Maity (10.1016/j.is.2024.102378_b29) 2022 Fersini (10.1016/j.is.2024.102378_b31) 2022 Luong (10.1016/j.is.2024.102378_b42) 2015 Bozarth (10.1016/j.is.2024.102378_b9) 2020 Chen (10.1016/j.is.2024.102378_b23) 2022 Gomez (10.1016/j.is.2024.102378_b6) 2020 Suryawanshi (10.1016/j.is.2024.102378_b4) 2020 Hochreiter (10.1016/j.is.2024.102378_b39) 1997; 9 Yang (10.1016/j.is.2024.102378_b7) 2022 Zhang (10.1016/j.is.2024.102378_b24) 2024 Li (10.1016/j.is.2024.102378_b38) 2022; vol. 162 Li (10.1016/j.is.2024.102378_b36) 2022 Lee (10.1016/j.is.2024.102378_b33) 2021 Zhong (10.1016/j.is.2024.102378_b35) 2022; vol. 13141 Ayetiran (10.1016/j.is.2024.102378_b43) 2022; 252 Srivastava (10.1016/j.is.2024.102378_b48) 2014; 15 10.1016/j.is.2024.102378_b47 Rizzi (10.1016/j.is.2024.102378_b32) 2023; 60 Shu (10.1016/j.is.2024.102378_b45) 2020; 8 Singhal (10.1016/j.is.2024.102378_b16) 2019 Giachanou (10.1016/j.is.2024.102378_b1) 2020 Giachanou (10.1016/j.is.2024.102378_b19) 2020; vol. 12284 Giachanou (10.1016/j.is.2024.102378_b20) 2020 Khattar (10.1016/j.is.2024.102378_b3) 2019 10.1016/j.is.2024.102378_b18 Lahat (10.1016/j.is.2024.102378_b13) 2014 Graves (10.1016/j.is.2024.102378_b11) 2013 Wang (10.1016/j.is.2024.102378_b15) 2018 Chen (10.1016/j.is.2024.102378_b10) 2021; 426 Pramanick (10.1016/j.is.2024.102378_b34) 2021 Kim (10.1016/j.is.2024.102378_b41) 2014 10.1016/j.is.2024.102378_b50 Fukushima (10.1016/j.is.2024.102378_b40) 1982; 15 Pennington (10.1016/j.is.2024.102378_b46) 2014 Ioffe (10.1016/j.is.2024.102378_b49) 2015; vol. 37 Alam (10.1016/j.is.2024.102378_b14) 2022 Koutlis (10.1016/j.is.2024.102378_b8) 2023 10.1016/j.is.2024.102378_b28 Ayetiran (10.1016/j.is.2024.102378_b44) 2021; 219 Hosseinmardi (10.1016/j.is.2024.102378_b26) 2016 Devlin (10.1016/j.is.2024.102378_b17) 2019 Zhou (10.1016/j.is.2024.102378_b5) 2020; 12085 European Foundation for South Asian Studies (10.1016/j.is.2024.102378_b2) 2021 Wu (10.1016/j.is.2024.102378_b22) 2021; ACL/IJCNLP 2021 Yang (10.1016/j.is.2024.102378_b30) 2022 Segura-Bedmar (10.1016/j.is.2024.102378_b12) 2022; 13 |
| References_xml | – start-page: 11 year: 2019 end-page: 18 ident: b27 article-title: Exploring deep multimodal fusion of text and photo for hate speech classification publication-title: Proceedings of the Third Workshop on Abusive Language Online – start-page: 60 year: 2020 end-page: 71 ident: b9 article-title: Toward a better performance evaluation framework for fake news classification publication-title: Proceedings of the Fourteenth International AAAI Conference on Web and Social Media, ICWSM 2020, Held Virtually, Original Venue: Atlanta, Georgia, USA, June 8-11, 2020 – volume: 9 start-page: 1735 year: 1997 end-page: 1780 ident: b39 article-title: Long short-term memory publication-title: Neural Comput. – start-page: 6625 year: 2022 end-page: 6643 ident: b14 article-title: A survey on multimodal disinformation detection publication-title: Proceedings of the 29th International Conference on Computational Linguistics – start-page: 39 year: 2019 end-page: 47 ident: b16 article-title: SpotFake: A multi-modal framework for fake news detection publication-title: Fifth IEEE International Conference on Multimedia Big Data, BigMM 2019, Singapore, September 11-13, 2019 – volume: vol. 37 start-page: 448 year: 2015 end-page: 456 ident: b49 article-title: Batch normalization: Accelerating deep network training by reducing internal covariate shift publication-title: Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015 – volume: vol. 139 start-page: 8748 year: 2021 end-page: 8763 ident: b37 article-title: Learning transferable visual models from natural language supervision publication-title: Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event – year: 2021 ident: b2 article-title: The role of fake news in fueling hate speech and extremism online; promoting adequate measures for tackling the phenomenon – start-page: 849 year: 2018 end-page: 857 ident: b15 article-title: EANN: event adversarial neural networks for multi-modal fake news detection publication-title: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, August 19-23, 2018 – start-page: 1412 year: 2015 end-page: 1421 ident: b42 article-title: Effective approaches to attention-based neural machine translation publication-title: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015 – volume: 426 start-page: 195 year: 2021 end-page: 215 ident: b10 article-title: New ideas and trends in deep multimodal content understanding: A review publication-title: Neurocomputing – start-page: 2915 year: 2019 end-page: 2921 ident: b3 article-title: MVAE: multimodal variational autoencoder for fake news detection publication-title: The World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019 – volume: 13 start-page: 284 year: 2022 ident: b12 article-title: Multimodal fake news detection publication-title: Inf. – start-page: 4439 year: 2021 end-page: 4455 ident: b34 article-title: MOMENTA: A multimodal framework for detecting harmful memes and their targets publication-title: Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021 – start-page: 1746 year: 2014 end-page: 1751 ident: b41 article-title: Convolutional neural networks for sentence classification publication-title: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, a Meeting of SIGDAT, a Special Interest Group of the ACL – volume: 8 start-page: 171 year: 2020 end-page: 188 ident: b45 article-title: FakeNewsNet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media publication-title: Big Data – reference: D.P. Kingma, J. Ba, Adam: A Method for Stochastic Optimization, in: Y. Bengio, Y. LeCun (Eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. – reference: D. Kiela, H. Firooz, A. Mohan, V. Goswami, A. Singh, P. Ringshia, D. Testuggine, The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes, in: H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, H. Lin (Eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, Virtual, 2020. – volume: 252 year: 2022 ident: b43 article-title: Attention-based aspect sentiment classification using enhanced learning through CNN-BiLSTM networks publication-title: Knowl.-Based Syst. – start-page: 4505 year: 2022 end-page: 4514 ident: b7 article-title: Multimodal hate speech detection via cross-domain knowledge transfer publication-title: MM ’22: The 30th ACM International Conference on Multimedia, Lisboa, Portugal, October 10 - 14, 2022 – volume: ACL/IJCNLP 2021 start-page: 2560 year: 2021 end-page: 2569 ident: b22 article-title: Multimodal fusion with co-attention networks for fake news detection publication-title: Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021 – start-page: 32 year: 2020 end-page: 41 ident: b4 article-title: Multimodal meme dataset (multiOFF) for identifying offensive content in image and text publication-title: Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying – reference: T. Dozat, Incorporating nesterov momentum into adam, in: 4th International Conference on Learning Representations, ICLR 2016 Workshop Track, San Juan, Puerto Rico, USA, May 2-4, 2016, Conference Track Proceedings, 2016. – start-page: 533 year: 2022 end-page: 549 ident: b31 article-title: SemEval-2022 task 5: Multimedia automatic misogyny identification publication-title: Proceedings of the 16th International Workshop on Semantic Evaluation, SemEval@NAACL 2022, Seattle, Washington, United States, July 14-15, 2022 – start-page: 647 year: 2020 end-page: 654 ident: b20 article-title: Multimodal multi-image fake news detection publication-title: 7th IEEE International Conference on Data Science and Advanced Analytics, DSAA 2020, Sydney, Australia, October 6-9, 2020 – volume: 219 year: 2021 ident: b44 article-title: EDS-MEMBED: multi-sense embeddings based on enhanced distributional semantic structures via a graph walk over word senses publication-title: Knowl.-Based Syst. – volume: 15 start-page: 455 year: 1982 end-page: 469 ident: b40 article-title: Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position publication-title: Pattern Recognit. – year: 2024 ident: b24 article-title: Scenefnd: Multimodal fake news detection by modelling scene context information publication-title: J. Inf. Sci. – start-page: 4171 year: 2019 end-page: 4186 ident: b17 article-title: BERT: pre-training of deep bidirectional transformers for language understanding publication-title: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers) – volume: vol. 162 start-page: 12888 year: 2022 end-page: 12900 ident: b38 article-title: BLIP: bootstrapping language-image pre-training for unified vision-language understanding and generation publication-title: International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA – start-page: 1739 year: 2022 end-page: 1749 ident: b29 article-title: A multitask framework for sentiment, emotion and sarcasm aware cyberbullying detection from multi-modal code-mixed memes publication-title: SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11 - 15, 2022 – volume: 60 year: 2023 ident: b32 article-title: Recognizing misogynous memes: Biased models and tricky archetypes publication-title: Inf. Process. Manag. – volume: 15 start-page: 1929 year: 2014 end-page: 1958 ident: b48 article-title: Dropout: A simple way to prevent neural networks from overfitting publication-title: J. Mach. Learn. Res. – volume: 93 start-page: 150 year: 2023 end-page: 158 ident: b25 article-title: TRIMOON: two-round inconsistency-based multi-modal fusion network for fake news detection publication-title: Inf. Fusion – start-page: 4505 year: 2022 end-page: 4514 ident: b30 article-title: Multimodal hate speech detection via cross-domain knowledge transfer publication-title: MM ’22: The 30th ACM International Conference on Multimedia, Lisboa, Portugal, October 10 - 14, 2022 – reference: K. Simonyan, A. Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition, in: Y. Bengio, Y. LeCun (Eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. – start-page: 1459 year: 2020 end-page: 1467 ident: b6 article-title: Exploring hate speech detection in multimodal publications publication-title: IEEE Winter Conference on Applications of Computer Vision, WACV 2020, Snowmass Village, CO, USA, March 1-5, 2020 – start-page: 101 year: 2014 end-page: 105 ident: b13 article-title: Challenges in multimodal data fusion publication-title: 22nd European Signal Processing Conference, EUSIPCO 2014, Lisbon, Portugal, September 1-5, 2014 – start-page: 3503 year: 2020 end-page: 3504 ident: b1 article-title: The battle against online harmful information: The cases of fake news and hate speech publication-title: CIKM ’20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020 – volume: vol. 12284 start-page: 30 year: 2020 end-page: 38 ident: b19 article-title: Multimodal fake news detection with textual, visual and semantic information publication-title: Text, Speech, and Dialogue - 23rd International Conference, TSD 2020, Brno, Czech Republic, September 8-11, 2020, Proceedings – start-page: 186 year: 2016 end-page: 192 ident: b26 article-title: Prediction of cyberbullying incidents in a media-based social network publication-title: 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2016, San Francisco, CA, USA, August 18-21, 2016 – start-page: 1532 year: 2014 end-page: 1543 ident: b46 article-title: Glove: Global vectors for word representation publication-title: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, a Meeting of SIGDAT, a Special Interest Group of the ACL – year: 2022 ident: b36 article-title: LAVIS: A library for language-vision intelligence – volume: 58 year: 2021 ident: b21 article-title: Detecting fake news by exploring the consistency of multimodal data publication-title: Inf. Process. Manag. – start-page: 586 year: 2023 end-page: 591 ident: b8 article-title: MemeFier: Dual-stage modality fusion for image meme classification publication-title: Proceedings of the 2023 ACM International Conference on Multimedia Retrieval, ICMR 2023, Thessaloniki, Greece, June 12-15, 2023 – volume: 12085 start-page: 354 year: 2020 end-page: 367 ident: b5 article-title: SAFE: similarity-aware multi-modal fake news detection publication-title: Advances in Knowledge Discovery and Data Mining - 24th Pacific-Asia Conference, PAKDD 2020, Singapore, May 11-14, 2020, Proceedings, Part II – start-page: 5138 year: 2021 end-page: 5147 ident: b33 article-title: Disentangling hate in online memes publication-title: MM ’21: ACM Multimedia Conference, Virtual Event, China, October 20 - 24, 2021 – volume: vol. 13141 start-page: 599 year: 2022 end-page: 611 ident: b35 article-title: Combining knowledge and multi-modal fusion for meme classification publication-title: MultiMedia Modeling - 28th International Conference, MMM 2022, Phu Quoc, Vietnam, June 6-10, 2022, Proceedings, Part I – start-page: 273 year: 2013 end-page: 278 ident: b11 article-title: Hybrid speech recognition with deep bidirectional LSTM publication-title: 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, Olomouc, Czech Republic, December 8-12, 2013 – start-page: 2897 year: 2022 end-page: 2905 ident: b23 article-title: Cross-modal ambiguity learning for multimodal fake news detection publication-title: WWW ’22: The ACM Web Conference 2022, Virtual Event, Lyon, France, April 25 - 29, 2022 – volume: vol. 162 start-page: 12888 year: 2022 ident: 10.1016/j.is.2024.102378_b38 article-title: BLIP: bootstrapping language-image pre-training for unified vision-language understanding and generation – start-page: 5138 year: 2021 ident: 10.1016/j.is.2024.102378_b33 article-title: Disentangling hate in online memes – start-page: 2897 year: 2022 ident: 10.1016/j.is.2024.102378_b23 article-title: Cross-modal ambiguity learning for multimodal fake news detection – year: 2024 ident: 10.1016/j.is.2024.102378_b24 article-title: Scenefnd: Multimodal fake news detection by modelling scene context information publication-title: J. Inf. Sci. – start-page: 4505 year: 2022 ident: 10.1016/j.is.2024.102378_b7 article-title: Multimodal hate speech detection via cross-domain knowledge transfer – start-page: 60 year: 2020 ident: 10.1016/j.is.2024.102378_b9 article-title: Toward a better performance evaluation framework for fake news classification – start-page: 586 year: 2023 ident: 10.1016/j.is.2024.102378_b8 article-title: MemeFier: Dual-stage modality fusion for image meme classification – volume: 219 year: 2021 ident: 10.1016/j.is.2024.102378_b44 article-title: EDS-MEMBED: multi-sense embeddings based on enhanced distributional semantic structures via a graph walk over word senses publication-title: Knowl.-Based Syst. doi: 10.1016/j.knosys.2021.106902 – start-page: 849 year: 2018 ident: 10.1016/j.is.2024.102378_b15 article-title: EANN: event adversarial neural networks for multi-modal fake news detection – start-page: 186 year: 2016 ident: 10.1016/j.is.2024.102378_b26 article-title: Prediction of cyberbullying incidents in a media-based social network – start-page: 1739 year: 2022 ident: 10.1016/j.is.2024.102378_b29 article-title: A multitask framework for sentiment, emotion and sarcasm aware cyberbullying detection from multi-modal code-mixed memes – volume: 9 start-page: 1735 issue: 8 year: 1997 ident: 10.1016/j.is.2024.102378_b39 article-title: Long short-term memory publication-title: Neural Comput. doi: 10.1162/neco.1997.9.8.1735 – start-page: 1412 year: 2015 ident: 10.1016/j.is.2024.102378_b42 article-title: Effective approaches to attention-based neural machine translation – start-page: 273 year: 2013 ident: 10.1016/j.is.2024.102378_b11 article-title: Hybrid speech recognition with deep bidirectional LSTM – volume: 93 start-page: 150 year: 2023 ident: 10.1016/j.is.2024.102378_b25 article-title: TRIMOON: two-round inconsistency-based multi-modal fusion network for fake news detection publication-title: Inf. Fusion doi: 10.1016/j.inffus.2022.12.016 – start-page: 101 year: 2014 ident: 10.1016/j.is.2024.102378_b13 article-title: Challenges in multimodal data fusion – start-page: 39 year: 2019 ident: 10.1016/j.is.2024.102378_b16 article-title: SpotFake: A multi-modal framework for fake news detection – volume: 15 start-page: 455 issue: 6 year: 1982 ident: 10.1016/j.is.2024.102378_b40 article-title: Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position publication-title: Pattern Recognit. doi: 10.1016/0031-3203(82)90024-3 – volume: vol. 12284 start-page: 30 year: 2020 ident: 10.1016/j.is.2024.102378_b19 article-title: Multimodal fake news detection with textual, visual and semantic information – start-page: 4505 year: 2022 ident: 10.1016/j.is.2024.102378_b30 article-title: Multimodal hate speech detection via cross-domain knowledge transfer – volume: 426 start-page: 195 year: 2021 ident: 10.1016/j.is.2024.102378_b10 article-title: New ideas and trends in deep multimodal content understanding: A review publication-title: Neurocomputing doi: 10.1016/j.neucom.2020.10.042 – year: 2021 ident: 10.1016/j.is.2024.102378_b2 – volume: 13 start-page: 284 issue: 6 year: 2022 ident: 10.1016/j.is.2024.102378_b12 article-title: Multimodal fake news detection publication-title: Inf. – volume: vol. 37 start-page: 448 year: 2015 ident: 10.1016/j.is.2024.102378_b49 article-title: Batch normalization: Accelerating deep network training by reducing internal covariate shift – start-page: 533 year: 2022 ident: 10.1016/j.is.2024.102378_b31 article-title: SemEval-2022 task 5: Multimedia automatic misogyny identification – ident: 10.1016/j.is.2024.102378_b28 – volume: vol. 13141 start-page: 599 year: 2022 ident: 10.1016/j.is.2024.102378_b35 article-title: Combining knowledge and multi-modal fusion for meme classification – start-page: 1459 year: 2020 ident: 10.1016/j.is.2024.102378_b6 article-title: Exploring hate speech detection in multimodal publications – start-page: 4439 year: 2021 ident: 10.1016/j.is.2024.102378_b34 article-title: MOMENTA: A multimodal framework for detecting harmful memes and their targets – start-page: 6625 year: 2022 ident: 10.1016/j.is.2024.102378_b14 article-title: A survey on multimodal disinformation detection – volume: 15 start-page: 1929 issue: 1 year: 2014 ident: 10.1016/j.is.2024.102378_b48 article-title: Dropout: A simple way to prevent neural networks from overfitting publication-title: J. Mach. Learn. Res. – start-page: 1746 year: 2014 ident: 10.1016/j.is.2024.102378_b41 article-title: Convolutional neural networks for sentence classification – start-page: 3503 year: 2020 ident: 10.1016/j.is.2024.102378_b1 article-title: The battle against online harmful information: The cases of fake news and hate speech – ident: 10.1016/j.is.2024.102378_b18 – year: 2022 ident: 10.1016/j.is.2024.102378_b36 – volume: 252 year: 2022 ident: 10.1016/j.is.2024.102378_b43 article-title: Attention-based aspect sentiment classification using enhanced learning through CNN-BiLSTM networks publication-title: Knowl.-Based Syst. doi: 10.1016/j.knosys.2022.109409 – volume: 8 start-page: 171 issue: 3 year: 2020 ident: 10.1016/j.is.2024.102378_b45 article-title: FakeNewsNet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media publication-title: Big Data doi: 10.1089/big.2020.0062 – volume: 12085 start-page: 354 year: 2020 ident: 10.1016/j.is.2024.102378_b5 article-title: SAFE: similarity-aware multi-modal fake news detection – start-page: 4171 year: 2019 ident: 10.1016/j.is.2024.102378_b17 article-title: BERT: pre-training of deep bidirectional transformers for language understanding – volume: ACL/IJCNLP 2021 start-page: 2560 year: 2021 ident: 10.1016/j.is.2024.102378_b22 article-title: Multimodal fusion with co-attention networks for fake news detection – start-page: 2915 year: 2019 ident: 10.1016/j.is.2024.102378_b3 article-title: MVAE: multimodal variational autoencoder for fake news detection – volume: 58 issue: 5 year: 2021 ident: 10.1016/j.is.2024.102378_b21 article-title: Detecting fake news by exploring the consistency of multimodal data publication-title: Inf. Process. Manag. doi: 10.1016/j.ipm.2021.102610 – ident: 10.1016/j.is.2024.102378_b50 – start-page: 32 year: 2020 ident: 10.1016/j.is.2024.102378_b4 article-title: Multimodal meme dataset (multiOFF) for identifying offensive content in image and text – start-page: 1532 year: 2014 ident: 10.1016/j.is.2024.102378_b46 article-title: Glove: Global vectors for word representation – volume: 60 issue: 5 year: 2023 ident: 10.1016/j.is.2024.102378_b32 article-title: Recognizing misogynous memes: Biased models and tricky archetypes publication-title: Inf. Process. Manag. doi: 10.1016/j.ipm.2023.103474 – volume: vol. 139 start-page: 8748 year: 2021 ident: 10.1016/j.is.2024.102378_b37 article-title: Learning transferable visual models from natural language supervision – start-page: 647 year: 2020 ident: 10.1016/j.is.2024.102378_b20 article-title: Multimodal multi-image fake news detection – ident: 10.1016/j.is.2024.102378_b47 – start-page: 11 year: 2019 ident: 10.1016/j.is.2024.102378_b27 article-title: Exploring deep multimodal fusion of text and photo for hate speech classification |
| SSID | ssj0002599 |
| Score | 2.476748 |
| Snippet | Fake news, hate speech and offensive language are related evil triplets currently affecting modern societies. Text modality for the computational detection of... |
| SourceID | crossref elsevier |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 102378 |
| SubjectTerms | BiLSTM-CNN Fake news Hate speech Inter-modal attention Multimodal content understanding Multimodal fusion Offensive language Unified modality |
| Title | An inter-modal attention-based deep learning framework using unified modality for multimodal fake news, hate speech and offensive language detection |
| URI | https://dx.doi.org/10.1016/j.is.2024.102378 |
| Volume | 123 |
| WOSCitedRecordID | wos001217561900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1873-6076 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0002599 issn: 0306-4379 databaseCode: AIEXJ dateStart: 19950301 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV3JbhNBEG2ZhAM5sAQQYVMduCAzMItnOxqUCDhESATJt1F3T1cywYwtexwlfAdHPpbqbWzCIkDiMrZavYxcz13V1a-qGHsSYZimnKeBkjUdUKK0CISI0yAuOYaY68LT0hSbyA8Pi8mkfDcYfPWxMGfTvG2L8_Ny_l9FTW0kbB06-xfi7ielBvpOQqcniZ2efyT4cWtyQCyCT7NaJwLoOstoDLTCqoe1UnNfK-J4iJ6bNVwZp8GqbVAbpWZs49ichnRoZ0P-UZkq5Fo0J2SnDpdzpaQNj5shOjq894LSap3herWbRrALgTLIs5mkjevXxi5uuiYuVNcsuCs20nBUwwNSYqv-GKBv-cvs87H-fJkJZXZ21zh1Pmnn0YhHPfvVudl8qM2a12TCu8Is0MkTreKyu3WRJ0EW2gIy_XZu45d_UA3WS3FKsz3Xi5qcFbZ60KWE2-_1UnolTa8lBT-5wrbjPC1pz9wev9mfvO01PR0dS3tLZV_NXYNb_uD36_zc7NkwZY5usuvuDAJji51bbKDaXXbD1_cAt93vsp2NZJW32ZdxCxvAgkvAAg0s8MCCHlhggAUOWOCBBSRrWAMLNLBAA-sZaFiBhRUQrKCHFXhYQQ-rO-zDwf7Rq9eBq-kRyCTJukCEKFQZiZxLjjGm9YiLPFThKEIlYxlHinpIJbKI-owEV9QjQgwLrJEsV0zusq121qp7DGTCsyzPVZEj9ZSJwJpnBaIoFdnYNd9jL_xPXkmX8F7XXZlWntl4WjXLSgupskLaY0_7EXOb7OU3fRMvxcoZq9YIrQhwvxx1_59GPWDX1v-Th2yrW6zUI3ZVnnXNcvHYofIb8ne97Q |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=An+inter-modal+attention-based+deep+learning+framework+using+unified+modality+for+multimodal+fake+news%2C+hate+speech+and+offensive+language+detection&rft.jtitle=Information+systems+%28Oxford%29&rft.au=Ayetiran%2C+Eniafe+Festus&rft.au=%C3%96zg%C3%B6bek%2C+%C3%96zlem&rft.date=2024-07-01&rft.pub=Elsevier+Ltd&rft.issn=0306-4379&rft.eissn=1873-6076&rft.volume=123&rft_id=info:doi/10.1016%2Fj.is.2024.102378&rft.externalDocID=S030643792400036X |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0306-4379&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0306-4379&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0306-4379&client=summon |