Combat AI With AI: Counteract Machine-Generated Fake Restaurant Reviews on Social Media

Recent advances in generative models such as GPT may be used to fabricate indistinguishable fake customer reviews at a much lower cost, thus posing challenges for social media platforms to detect these machine-generated fake reviews. We propose to leverage the high-quality elite restaurant reviews verified by Yelp to generate fake reviews from the OpenAI GPT review creator and ultimately fine-tune a GPT output detector to predict fake reviews that significantly outperform existing solutions. We further apply the model to predict non-elite reviews and identify the patterns across several dimensions, such as review, user and restaurant characteristics, and writing style. We show that social media platforms are continuously challenged by machine-generated fake reviews, although they may implement detection systems to filter out suspicious reviews.

[1]  Mengxia Zhang,et al.  Can Consumer-Posted Photos Serve as a Leading Indicator of Restaurant Survival? Evidence from Yelp , 2018, Manag. Sci..

[2]  Jan-Willem van de Meent,et al.  Probabilistic program inference in network-based epidemiological simulations , 2022, PLoS Comput. Biol..

[3]  Mark Heitmann,et al.  More than a Feeling: Accuracy and Application of Sentiment Analysis , 2022, International Journal of Research in Marketing.

[4]  Bernard J. Jansen,et al.  Creating and detecting fake reviews of online products , 2022, Journal of Retailing and Consumer Services.

[5]  B. Quigley,et al.  The role of alcohol outlet visits derived from mobile phone location data in enhancing domestic violence prediction at the neighborhood level , 2021, Health & place.

[6]  D. Taylor,et al.  Human mobility data and machine learning reveal geographic differences in alcohol sales and alcohol outlet visits across U.S. states during COVID-19 , 2021, PloS one.

[7]  Jianxi Gao,et al.  Percolation of temporal hierarchical mobility networks during COVID-19 , 2021, Philosophical Transactions of the Royal Society A.

[8]  Philippe Laban,et al.  Can Transformer Models Measure Coherence In Text: Re-Thinking the Shuffle Test , 2021, ACL.

[9]  Alexander G. Nikolaev,et al.  Fake review detection on online E-commerce platforms: a systematic literature review , 2021, Data Mining and Knowledge Discovery.

[10]  Stella Biderman,et al.  GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow , 2021 .

[11]  Charles Foster,et al.  The Pile: An 800GB Dataset of Diverse Text for Language Modeling , 2020, ArXiv.

[12]  Anja Lambrecht,et al.  The Effect of Individual Online Reviews on Purchase Likelihood , 2020, Mark. Sci..

[13]  Anubha Agrawal,et al.  Leveraging GPT-2 for Classifying Spam Reviews with Limited Labeled Data via Adversarial Training , 2020, ArXiv.

[14]  Ben Charoenwong,et al.  Social connections with COVID-19–affected areas increase compliance with mobility restrictions , 2020, Science Advances.

[15]  Mark Chen,et al.  Language Models are Few-Shot Learners , 2020, NeurIPS.

[16]  Eric W.T. Ngai,et al.  Fake online reviews: Literature review, synthesis, and directions for future research , 2020, Decis. Support Syst..

[17]  Jessie Pallud,et al.  Illusions of truth—Experimental insights into human and algorithmic detections of fake online reviews , 2020 .

[18]  R. Law,et al.  Effects of online reviews and managerial responses from a review manipulation perspective , 2020, Current Issues in Tourism.

[19]  Xiaolong Wang,et al.  Opinion spam detection by incorporating multimodal embedded representation into a probabilistic review graph , 2019, Neurocomputing.

[20]  Alec Radford,et al.  Release Strategies and the Social Impacts of Language Models , 2019, ArXiv.

[21]  Jurui Zhang,et al.  What’s yours is mine: exploring customer voice on Airbnb using text-mining approaches , 2019, Journal of Consumer Marketing.

[22]  Andrew Owens,et al.  Detecting Photoshopped Faces by Scripting Photoshop , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[23]  Carlos Angel Iglesias,et al.  A framework for fake review detection in online consumer electronics retailers , 2019, Inf. Process. Manag..

[24]  Xu Zhuang,et al.  Enactment of Ensemble Learning for Review Spam Detection on Selected Features , 2019, Int. J. Comput. Intell. Syst..

[25]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[26]  Bo Pang,et al.  A unified framework for detecting author spamicity by modeling review deviation , 2018, Expert Syst. Appl..

[27]  Brandon Van Der Heide,et al.  How People Evaluate Online Reviews , 2018, Commun. Res..

[28]  Wen Zhang,et al.  DRI-RCNN: An approach to deceptive review identification using recurrent convolutional neural network , 2018, Inf. Process. Manag..

[29]  Nor Badrul Anuar,et al.  Detecting opinion spams through supervised boosting approach , 2018, PloS one.

[30]  Ling Peng,et al.  Manufactured opinions: The effect of manipulating online product reviews , 2018, Journal of Business Research.

[31]  C. Hall,et al.  The manager's dilemma: a conceptualization of online review manipulation strategies , 2018 .

[32]  Qingguo Ma,et al.  The Effects of Money on Fake Rating Behavior in E-Commerce: Electrophysiological Time Course Evidence From Consumers , 2018, Front. Neurosci..

[33]  John H. Summey,et al.  WHAT MOTIVATES CONSUMERS TO PARTAKE IN CYBER SHILLING? , 2018 .

[34]  Xiaowei Xu,et al.  GSLDA: LDA-based group spamming detection in product reviews , 2018, Applied Intelligence.

[35]  Alec Radford,et al.  Improving Language Understanding by Generative Pre-Training , 2018 .

[36]  Jie Cao,et al.  Detecting Spammer Groups From Product Reviews: A Partially Supervised Learning Model , 2018, IEEE Access.

[37]  Frank Hutter,et al.  Fixing Weight Decay Regularization in Adam , 2017, ArXiv.

[38]  Andrew Whinston,et al.  Sentiment Manipulation in Online Platforms: An Analysis of Movie Tweets , 2017 .

[39]  Ben Y. Zhao,et al.  Automated Crowdturfing Attacks and Defenses in Online Review Systems , 2017, CCS.

[40]  Taghi M. Khoshgoftaar,et al.  Improving detection of untrustworthy online reviews using ensemble learners combined with feature selection , 2017, Social Network Analysis and Mining.

[41]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[42]  Dong-Hong Ji,et al.  Neural networks for deceptive opinion spam detection: An empirical study , 2017, Inf. Sci..

[43]  Tao Zhang,et al.  Welfare economics of review information: Implications for the online selling platform owner , 2017 .

[44]  Joshua Fogel,et al.  Intentions to Use the Yelp Review Website and Purchase Behavior after Reading Reviews , 2017, J. Theor. Appl. Electron. Commer. Res..

[45]  Wonho Song,et al.  Information Quality of Online Reviews in the Presence of Potentially Fake Reviews , 2017 .

[46]  George Valkanas,et al.  The Impact of Fake Reviews on Online Visibility: A Vulnerability Assessment of the Hotel Industry , 2016, Inf. Syst. Res..

[47]  A. Agnihotri,et al.  Online Review Helpfulness: Role of Qualitative Factors , 2016 .

[48]  Andreas Munzel Assisting consumers in detecting fake reviews: The role of identity information disclosure and consensus , 2016 .

[49]  Dongsong Zhang,et al.  What Online Reviewer Behaviors Really Matter? Effects of Verbal and Nonverbal Behaviors on Detection of Fake Online Reviews , 2016, J. Manag. Inf. Syst..

[50]  Michael Luca Reviews, Reputation, and Revenue: The Case of Yelp.Com , 2016 .

[51]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[52]  Q. Ye,et al.  Analysis of the Perceived Value of Online Tourism Reviews: Influence of Readability and Reviewer Characteristics , 2016 .

[53]  Rico Sennrich,et al.  Neural Machine Translation of Rare Words with Subword Units , 2015, ACL.

[54]  Georgios Zervas,et al.  Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud , 2015, Manag. Sci..

[55]  Joan Lu,et al.  A framework investigating the online user reviews to measure the biasness for sentiment analysis , 2016 .

[56]  Arif Djunaidy,et al.  FAKE REVIEW DETECTION FROM A PRODUCT REVIEW USING MODIFIED METHOD OF ITERATIVE COMPUTATION FRAMEWORK , 2016 .

[57]  Raffaele Filieri,et al.  Why do travelers trust TripAdvisor? Antecedents of trust towards consumer-generated media and its influence on recommendation adoption and word of mouth , 2015 .

[58]  Joseph D. Prusa,et al.  Survey of review spam detection using machine learning techniques , 2015, Journal of Big Data.

[59]  Leman Akoglu,et al.  Collective Opinion Spam Detection: Bridging Review Networks and Metadata , 2015, KDD.

[60]  Veronica Liljander,et al.  Young consumers' responses to suspected covert and overt blog marketing , 2015, Internet Res..

[61]  Martin Ester,et al.  Detecting Singleton Review Spammers Using Semantic Similarity , 2015, WWW.

[62]  Kate Hunt Gaming the system: Fake online reviews v. consumer law , 2015 .

[63]  Snehasish Banerjee,et al.  Applauses in hotel reviews: Genuine or deceptive? , 2014, 2014 Science and Information Conference.

[64]  Hyun-Hwa Lee,et al.  Consumer responses toward online review manipulation , 2014 .

[65]  Alton Yeow-Kuan Chua,et al.  A theoretical framework to identify authentic online reviews , 2014, Online Inf. Rev..

[66]  Camilla Vásquez,et al.  The Discourse of Online Consumer Reviews , 2014 .

[67]  Eric Fang,et al.  Is Neutral Really Neutral? The Effects of Neutral User-Generated Content on Product Sales , 2014 .

[68]  Eric T. Anderson,et al.  Reviews without a Purchase: Low Ratings, Loyal Customers, and Deception , 2014 .

[69]  Y. Wang,et al.  A Short Analysis of Discourse Coherence , 2014 .

[70]  Yun Wan,et al.  THE RELIABILITY OF ONLINE REVIEW HELPFULNESS , 2014 .

[71]  Chuan-Hoo Tan,et al.  Helpfulness of Online Product Reviews as Seen by Consumers: Source and Content Features , 2013, Int. J. Electron. Commer..

[72]  Claire Cardie,et al.  Negative Deceptive Opinion Spam , 2013, NAACL.

[73]  Xifeng Yan,et al.  Synthetic review spamming and defense , 2013, WWW.

[74]  Yi Zhao,et al.  Modeling Consumer Learning from Online Product Reviews , 2012, Mark. Sci..

[75]  Philip S. Yu,et al.  Review spam detection via temporal pattern discovery , 2012, KDD.

[76]  Dina Mayzlin,et al.  Promotional Reviews: An Empirical Investigation of Online Review Manipulation , 2012 .

[77]  Christopher G. Harris Detecting Deceptive Opinion Spam Using Human Computation , 2012, HCOMP@AAAI.

[78]  Claire Cardie,et al.  Estimating the prevalence of deception in online review communities , 2012, WWW.

[79]  Ling Liu,et al.  Manipulation of online reviews: An analysis of ratings, readability, and sentiments , 2012, Decis. Support Syst..

[80]  Yi Yang,et al.  Learning to Identify Review Spam , 2011, IJCAI.

[81]  Claire Cardie,et al.  Finding Deceptive Opinion Spam by Any Stretch of the Imagination , 2011, ACL.

[82]  Ling Liu,et al.  Manipulation in digital word-of-mouth: A reality check for book reviews , 2011, Decis. Support Syst..

[83]  Qing Cao,et al.  Exploring determinants of voting for the "helpfulness" of online user reviews: A text mining approach , 2011, Decis. Support Syst..

[84]  R. Schindler,et al.  Perceived Helpfulness of Online Consumer Reviews: The Role of Message Content and Style , 2010 .

[85]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[86]  Kyung Hyan Yoo,et al.  Comparison of Deceptive and Truthful Travel Reviews , 2009, ENTER.

[87]  Olga Vechtomova,et al.  Book Review: Introduction to Information Retrieval by Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze , 2009, CL.

[88]  Frank Keller,et al.  Data from eye-tracking corpora as evidence for theories of syntactic processing complexity , 2008, Cognition.

[89]  Bin Gu,et al.  Do online reviews matter? - An empirical investigation of panel data , 2008, Decis. Support Syst..

[90]  Miguel-Ángel Sicilia,et al.  The Impact of Readability on the Usefulness of Online Product Reviews: A Case Study on an Online Bookstore , 2008, WSKS.

[91]  Bing Liu,et al.  Opinion spam and analysis , 2008, WSDM '08.

[92]  Ling Liu,et al.  Do online reviews affect product sales? The role of reviewer characteristics and temporal effects , 2008, Inf. Technol. Manag..

[93]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[94]  A. Tversky,et al.  Subjective Probability: A Judgment of Representativeness , 1972 .

[95]  R. Gunning The Fog Index After Twenty Years , 1969 .

[96]  E A Smith,et al.  Automated readability index. , 1967, AMRL-TR. Aerospace Medical Research Laboratories.

[97]  W. Youden,et al.  Index for rating diagnostic tests , 1950, Cancer.