A Hybrid Deep Neural Network for Arabic Fake News Classification based on Temporal CNN and BiLSTM with Attention
Subject Areas : Natural Language Processing
Azhar Hadi
1
*
,
Abbas Hommadi
2
,
Hussein Ismael
3
1 - University of Babylon
2 - University of Babylon
3 - University of Babylon
Keywords: Arabic Fake News, Temporal Convolution Network, BiLSTM, Attention Mechanism, and Deep Learning.,
Abstract :
The proliferation of fake news in social media poses a significant threat to societal harmony, especially in languages like Arabic, which is distinguished by their complexity. Thus, Artificial intelligence techniques are necessary to mitigate the impact of misinformation. Many researchers have conducted this challenge by introducing machine and deep learning approaches. This study presents a new hybrid deep learning network, which combines three mechanisms: Temporal Convolutional Networks, Bidirectional Long Short-Term Memory, and Attention Mechanism. This mixture model produces a strong feature extraction process with temporal awareness. Moreover, the attention layer derives the model to focus only on relevant features and ignores irrelevant ones to concentrate on salient features. Also, several stages pre-processing Arabic text, representing words using a pre-trained word embedding model (Glove, Ara2Vec), and extracting features through advanced layers( BiLSTM, TCN, and Attention). Extensive experimentation on large-scale and well-known Arabic fake news datasets (AFND and AraNews) demonstrates the efficacy of the proposed model. The hybrid model shows superior performance compared to baseline and previous state-of-the-art studies by achieving an accuracy of 88% and 95% on the AFND and AraNews datasets, respectively. Our results show that the suggested model can be used as a powerful technique to restrain the widespread online misinformation in the Arabic language.
[1] M. Alkhair, K. Meftouh, K. Smaïli, and N. Othman, “An Arabic Corpus of Fake News: Collection, Analysis and Classification,” in Communications in Computer and Information Science, 2019. doi: 10.1007/978-3-030-32959-4_21.
[2] M. Al-Yahya, H. Al-Khalifa, H. Al-Baity, D. Alsaeed, and A. Essam, “Arabic Fake News Detection: Comparative Study of Neural Networks and Transformer-Based Approaches,” Complexity, vol. 2021, 2021, doi: 10.1155/2021/5516945.
[3] A. Khalil, M. Jarrah, M. Aldwairi, and Y. Jararweh, “Detecting Arabic Fake News Using Machine Learning,” in 2021 2nd International Conference on Intelligent Data Science Technologies and Applications, IDSTA 2021, 2021. doi: 10.1109/IDSTA53674.2021.9660811.
[4] Y. Almsrahad and N. Moghaddam Charkari, “Image Fake News Detection using Efficient NetB0 Model,” Journal of Information Systems and Telecommunication (JIST), vol. 12, no. 45, pp. 41–48, Mar. 2024, doi: 10.61186/jist.40976.12.45.41.
[5] P. Bahad, P. Saxena, and R. Kamal, “Fake News Detection using Bi-directional LSTM-Recurrent Neural Network,” in Procedia Computer Science, 2019. doi: 10.1016/j.procs.2020.01.072.
[6] A. Monnier, “Narratives of the Fake News Debate in France,” IAFOR Journal of Arts & Humanities, vol. 5, no. 2, 2018, doi: 10.22492/ijah.5.2.01.
[7] S. Senhadji and R. A. S. Ahmed, “Fake news detection using naïve Bayes and long short term memory algorithms,” IAES International Journal of Artificial Intelligence, vol. 11, no. 2, 2022, doi: 10.11591/ijai.v11.i2.pp746-752.
[8] Y. Bang, E. Ishii, S. Cahyawijaya, Z. Ji, and P. Fung, “Model Generalization on COVID-19 Fake News Detection,” in Communications in Computer and Information Science, 2021. doi: 10.1007/978-3-030-73696-5_13.
[9] J. A. Nasir, O. S. Khan, and I. Varlamis, “Fake news detection: A hybrid CNN-RNN based deep learning approach,” International Journal of Information Management Data Insights, vol. 1, no. 1, 2021, doi: 10.1016/j.jjimei.2020.100007.
[10] S. R. Sahoo and B. B. Gupta, “Multiple features based approach for automatic fake news detection on social networks using deep learning,” Appl Soft Comput, vol. 100, 2021, doi: 10.1016/j.asoc.2020.106983.
[11] A. Khalil, M. Jarrah, M. Aldwairi, and M. Jaradat, “AFND: Arabic fake news dataset for the detection and classification of articles credibility,” Data Brief, vol. 42, 2022, doi: 10.1016/j.dib.2022.108141.
[12] G. Jardaneh, H. Abdelhaq, M. Buzz, and D. Johnson, “Classifying Arabic tweets based on credibility using content and user features,” in 2019 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology, JEEIT 2019 - Proceedings, 2019. doi: 10.1109/JEEIT.2019.8717386.
[13] F. Aljwari, W. Alkaberi, A. Alshutayri, E. Aldhahri, N. Aljojo, and O. Abouola, “Multi-scale Machine Learning Prediction of the Spread of Arabic Online Fake News,” Postmodern Openings, vol. 13, no. 1 Sup1, 2022, doi: 10.18662/po/13.1sup1/411.
[14] E. M. B. Nagoudi, A. Elmadany, M. Abdul-Mageed, T. Alhindi, and H. Cavusoglu, “Machine Generation and Detection of Arabic Manipulated and Fake News,” Nov. 2020, [Online]. Available: http://arxiv.org/abs/2011.03092
[15] A. R. Mahlous and A. Al-Laith, “Fake News Detection in Arabic Tweets during the COVID-19 Pandemic,” International Journal of Advanced Computer Science and Applications, vol. 12, no. 6, 2021, doi: 10.14569/IJACSA.2021.0120691.
[16] S. E. Sorour and H. E. Abdelkader, “AFND: ARABIC FAKE NEWS DETECTION WITH AN ENSEMBLE DEEP CNN-LSTM MODEL,” J Theor Appl Inf Technol, vol. 100, no. 14, 2022.
[17] T. A. Wotaifi and B. N. Dhannoon, “An Effective Hybrid Deep Neural Network for Arabic Fake News Detection,” Baghdad Science Journal, vol. 20, no. 4, 2023, doi: 10.21123/bsj.2023.7427.
[18] T. A. Wotaifi and B. N. Dhannoon, “Attention Mechanism Based on a Pre-trained Model for Improving Arabic Fake News Predictions,” Iraqi Journal of Science, vol. 64, no. 11, 2023, doi: 10.24996/ijs.2023.64.11.45.
[19] H. Saadany, E. Mohamed, and C. Orasan, “Fake or Real? A Study of Arabic Satirical Fake News,” Nov. 2020, [Online]. Available: http://arxiv.org/abs/2011.00452
[20] R. M. Albalawi, A. T. Jamal, A. O. Khadidos, and A. M. Alhothali, “Multimodal Arabic Rumors Detection,” IEEE Access, vol. 11, 2023, doi: 10.1109/ACCESS.2023.3240373.
[21] S. F. Ahmed et al., “Deep learning modelling techniques: current progress, applications, advantages, and challenges,” Artif Intell Rev, vol. 56, no. 11, 2023, doi: 10.1007/s10462-023-10466-8.
[22] J. Kasubi, M. D. Huchaiah, I. Gad, and M. K. Hooshmand, “A Comparison Analysis of Conventional Classifiers and Deep Learning Model for Activity Recognition in Smart Homes based on Multi-label Classification,” Journal of Information Systems and Telecommunication (JIST), vol. 12, no. 46, pp. 127–137, Jun. 2024, doi: 10.61186/jist.36294.12.46.127.
[23] P. Kim, MATLAB Deep Learning: With Machine Learning, Neural Networks and Artificial Intelligence. 2017.
[24] K. Jindal and R. Aron, “A Hybrid Machine Learning Approach for Sentiment Analysis of Beauty Products Reviews,” Journal of Information Systems and Telecommunication, vol. 10, no. 37, 2022, doi: 10.52547/jist.15586.10.37.1.
[25] Q. Zhang, L. T. Yang, Z. Chen, and P. Li, “A survey on deep learning for big data,” 2018. doi: 10.1016/j.inffus.2017.10.006.
[26] J. Yu, A. de Antonio, and E. Villalba-Mora, “Deep Learning (CNN, RNN) Applications for Smart Homes: A Systematic Review,” 2022. doi: 10.3390/computers11020026.
[27] M. Schuster and K. K. Paliwal, “Bidirectional recurrent neural networks,” IEEE Transactions on Signal Processing, vol. 45, no. 11, 1997, doi: 10.1109/78.650093.
[28] Z. Huang, W. Xu, and K. Yu, “Bidirectional LSTM-CRF Models for Sequence Tagging,” Aug. 2015, [Online]. Available: http://arxiv.org/abs/1508.01991
[29] A. Graves and J. Schmidhuber, “Framewise phoneme classification with bidirectional LSTM and other neural network architectures,” Neural Networks, vol. 18, no. 5–6, pp. 602–610, 2005, doi: 10.1016/j.neunet.2005.06.042.
[30] D. Bahdanau, K. H. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” in 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, 2015.
[31] P. Zhou et al., “Attention-based bidirectional long short-term memory networks for relation classification,” in 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Short Papers, 2016. doi: 10.18653/v1/p16-2034.
[32] A. Vaswani et al., “Attention is all you need,” in Advances in Neural Information Processing Systems, 2017.
[33] C. Lea, R. Vidal, A. Reiter, and G. D. Hager, “Temporal convolutional networks: A unified approach to action segmentation,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2016. doi: 10.1007/978-3-319-49409-8_7.
[34] S. Vijayarani and M. P. Research Scholar, “Preprocessing Techniques for Text Mining-An Overview.”
[35] S. Akuma, T. Lubem, and I. T. Adom, “Comparing Bag of Words and TF-IDF with different models for hate speech detection from live tweets,” International Journal of Information Technology (Singapore), vol. 14, no. 7, 2022, doi: 10.1007/s41870-022-01096-4.
[36] M. SATHVIK, “Enhancing Machine Learning Algorithms using GPT Embeddings for Binary Classification,” Mar. 27, 2023. doi: 10.36227/techrxiv.22331053.v1.
[37] A. B. Soliman, K. Eissa, and S. R. El-Beltagy, “AraVec: A set of Arabic Word Embedding Models for use in Arabic NLP,” in Procedia Computer Science, 2017. doi: 10.1016/j.procs.2017.10.117.
[38] S. F. Sabbeh and H. A. Fasihuddin, “A Comparative Analysis of Word Embedding and Deep Learning for Arabic Sentiment Classification,” Electronics (Switzerland), vol. 12, no. 6, 2023, doi: 10.3390/electronics12061425.
[39] HANSON ER, “GloVe: Global Vectors for Word Representation Jeffrey,” AES: Journal of the Audio Engineering Society, vol. 19, no. 5, 1971.
[40] H. A. Ismael, N. H. Al-A’araji, and B. K. Shukur, “An Enhanced Diabetic Foot Ulcer Classification Approach Using GLCM and Deep Convolution Neural Network,” Karbala International Journal of Modern Science, vol. 8, no. 4, 2022, doi: 10.33640/2405-609X.3268.
[41] H. Himdi, G. Weir, F. Assiri, and H. Al-Barhamtoshy, “Arabic Fake News Detection Based on Textual Analysis,” Arab J Sci Eng, vol. 47, no. 8, 2022, doi: 10.1007/s13369-021-06449-y.
[42] S. Sharma, S. Sharma, and A. Athaiya, “ACTIVATION FUNCTIONS IN NEURAL NETWORKS,” International Journal of Engineering Applied Sciences and Technology, vol. 04, no. 12, 2020, doi: 10.33564/ijeast.2020.v04i12.054.
[43] H. Pratiwi et al., “Sigmoid Activation Function in Selecting the Best Model of Artificial Neural Networks,” in Journal of Physics: Conference Series, 2020. doi: 10.1088/1742-6596/1471/1/012010.
[44] H. A. Ismael, N. H. Al-A’araji, and B. K. Shukur, “Enhanced the prediction approach of diabetes using an autoencoder with regularization and deep neural network,” Periodicals of Engineering and Natural Sciences, vol. 10, no. 6, 2022, doi: 10.21533/pen.v10i6.3394.
[45] S. R. Durugkar, R. Raja, K. K. Nagwanshi, and S. Kumar, “Introduction to data mining,” 2022. doi: 10.1002/9781119792529.ch1.