A Transformer-based approach for Fake News detection using Time Series Analysis

Fake news is a growing problem in the digital age, spreading misinformation and affecting public opinion. Existing fake news detection is based on style analysis or news generator's behavior analysis, the former fails if the content is generated using existing news corpus while the latter requires a very specific data set. In this research, we aim to address the issue of fake news detection by employing deep learning-based time series analysis (TSA). We propose a TSA method to gauge the authenticity of the news based on previously available news content of a similar genre. We employed pre-trained models for encoding including GloVe and BERT and transformer-based sequence-to-sequence (Seq2Seq) for TSA. The results demonstrate 98% accuracy of pre-trained models, such as GloVe and BERT, over traditional encoding approaches having accuracy between 77% and 93%. Our study also compares the effectiveness of various deep learning methods, including Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU), LSTM with attention, GRU with attention, and Transformers with 8 and 16 multi-heads. The results show that Transformers with 8 and 16 multi-heads achieve 98% and 97% accuracy respectively as compared to LSTM (87%) and GRU (88%). Our work is useful for future research in TSA-based fake news detection and proposes to use GloVe and BERT-based encoding and multi-head transform architecture.

[1]  G. S,et al.  Fake News Detection Techniques for Diversified Datasets , 2023, Computational Intelligence and Machine Learning.

[2]  Linmei Hu,et al.  Deep learning for fake news detection: A comprehensive survey , 2022, AI Open.

[3]  Hassan Nazeer Chaudhry,et al.  A review of machine learning-based human activity recognition for diverse applications , 2022, Neural Computing and Applications.

[4]  Hassan Nazeer Chaudhry,et al.  Sentiment Analysis of before and after Elections: Twitter Data of U.S. Election 2020 , 2021, Electronics.

[5]  Ralf C. Staudemeyer,et al.  Understanding LSTM - a tutorial into Long Short-Term Memory Recurrent Neural Networks , 2019, ArXiv.

[6]  Francesco Marcelloni,et al.  A survey on fake news and rumour detection techniques , 2019, Inf. Sci..

[7]  Peng Wang,et al.  Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition , 2018, AAAI.

[8]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[9]  Vahid Kazemi,et al.  Show, Ask, Attend, and Answer: A Strong Baseline For Visual Question Answering , 2017, ArXiv.

[10]  Eros Gian Alessandro Pasero,et al.  EEG Based Eye State Classification using Deep Belief Network and Stacked AutoEncoder , 2016 .

[11]  Eros Gian Alessandro Pasero,et al.  A Hybrid Approach for Time Series Forecasting Using Deep Learning and Nonlinear Autoregressive Neural Networks , 2016 .

[12]  Kate Saenko,et al.  Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering , 2015, ECCV.

[13]  Quoc V. Le,et al.  Listen, Attend and Spell , 2015, ArXiv.

[14]  Matthew R. Walter,et al.  Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences , 2015, AAAI.

[15]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[16]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[17]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[18]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[19]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[20]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[21]  Xinyi Zhou,et al.  A Survey of Fake News , 2020, ACM Comput. Surv..

[22]  S. C. Kremer,et al.  Recurrent Neural Networks , 2019, Neural Networks and Statistical Learning.

[23]  Eros Gian Alessandro Pasero,et al.  Time Series Forecasting for Outdoor Temperature using Nonlinear Autoregressive Neural Network Models , 2016 .