Artificial Intelligence Security Threat, Crime, and Forensics: Taxonomy and Open Issues

Advances in Artificial Intelligence (AI) have influenced almost every field including computer science, robotics, social engineering, psychology, criminology and so on. Although AI has solved various challenges, potential security threats of AI algorithms and training data have been stressed by AI researchers. As AI system inherits security threats of traditional computer system, the concern about novel cyberattack enhanced by AI is also growing. In addition, AI is deeply connected to physical space (e.g. autonomous vehicle, intelligent virtual assistant), so AI-related crime can harm people physically, beyond the cyberspace. In this context, we represent a literature review of security threats and AI-related crime. Based on the literature review, this article defines the term AI crime and classifies AI crime into 2 categories: AI as tool crime and AI as target crime, inspired by a taxonomy of cybercrime: Computer as tool crime and Computer as tool crime. Through the proposed taxonomy, foreseeable AI crimes are systematically studied and related forensic techniques are also addressed. We also analyze the characteristics of the AI crimes and present challenges that are difficult to be solved with the traditional forensic techniques. Finally, open issues are presented, with emphasis on the need to establish novel strategies for AI forensics.

[1]  Robert M. Chesney,et al.  Deep Fakes: A Looming Crisis for National Security, Democracy and Privacy? , 2018 .

[2]  Payman Mohassel,et al.  SecureML: A System for Scalable Privacy-Preserving Machine Learning , 2017, 2017 IEEE Symposium on Security and Privacy (SP).

[3]  Guru Venkataramani,et al.  Machine Learning-Based Analysis of Program Binaries: A Comprehensive Study , 2019, IEEE Access.

[4]  Wei Cai,et al.  A Survey on Security Threats and Defensive Techniques of Machine Learning: A Data Driven View , 2018, IEEE Access.

[5]  Seyed-Mohsen Moosavi-Dezfooli,et al.  Universal Adversarial Perturbations , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Matthew C. Stamm,et al.  Forensic Similarity for Digital Images , 2019, IEEE Transactions on Information Forensics and Security.

[7]  Flora Amato,et al.  A semantic-based methodology for digital forensics analysis , 2020, J. Parallel Distributed Comput..

[8]  Ryan R. Curtin,et al.  Detecting Adversarial Samples from Artifacts , 2017, ArXiv.

[9]  Hao Chen,et al.  MagNet: A Two-Pronged Defense against Adversarial Examples , 2017, CCS.

[10]  R. Venkatesh Babu,et al.  Fast Feature Fool: A data independent approach to universal adversarial perturbations , 2017, BMVC.

[11]  Hyrum S. Anderson,et al.  The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation , 2018, ArXiv.

[12]  Yifei Zhang,et al.  #DebateNight: The Role and Influence of Socialbots on Twitter During the 1st 2016 U.S. Presidential Debate , 2018, ICWSM.

[13]  Anne E. James,et al.  CPS data streams analytics based on machine learning for Cloud and Fog Computing: A survey , 2019, Future Gener. Comput. Syst..

[14]  Victor R. Kebande,et al.  Novel digital forensic readiness technique in the cloud environment , 2018 .

[15]  Mariarosaria Taddeo,et al.  Artificial Intelligence Crime: An Interdisciplinary Analysis of Foreseeable Threats and Solutions , 2020, Sci. Eng. Ethics.

[16]  Ali Dehghantanha,et al.  Towards a Unified Forensic Investigation Framework of Smartphones , 2013 .

[17]  Ying Cai,et al.  Detecting Adversarial Examples Through Image Transformation , 2018, AAAI.

[18]  Yang Zhang,et al.  Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning , 2019, USENIX Security Symposium.

[19]  R. Gehl,et al.  Socialbots and Their Friends , 2016 .

[20]  Kang Li,et al.  Security Risks in Deep Learning Implementations , 2017, 2018 IEEE Security and Privacy Workshops (SPW).

[21]  Tiago Cruz,et al.  Adversarial Machine Learning Applied to Intrusion and Malware Scenarios: A Systematic Review , 2020, IEEE Access.

[22]  Miriam J. Metzger,et al.  The science of fake news , 2018, Science.

[23]  Dan Boneh,et al.  The Space of Transferable Adversarial Examples , 2017, ArXiv.

[24]  Peter A. Gloor,et al.  In the shades of the uncanny valley: An experimental study of human-chatbot interaction , 2018, Future Gener. Comput. Syst..

[25]  Robert Rowlingson,et al.  A Ten Step Process for Forensic Readiness , 2004, Int. J. Digit. EVid..

[26]  Kamini Dashora,et al.  Cyber Crime in the Society: Problems and Preventions , 2011 .

[27]  Patrick Lin,et al.  Robot Ethics 2.0: From Autonomous Cars to Artificial Intelligence , 2017 .

[28]  Mary L. Cummings,et al.  Artificial Intelligence and the Future of Warfare , 2017 .

[29]  David A. Wagner,et al.  Towards Evaluating the Robustness of Neural Networks , 2016, 2017 IEEE Symposium on Security and Privacy (SP).

[30]  Stefanos Gritzalis,et al.  A framework for designing cloud forensic-enabled services (CFeS) , 2018, Requirements Engineering.

[31]  Rodney McKemmish,et al.  When is Digital Evidence Forensically Sound? , 2008, IFIP Int. Conf. Digital Forensics.

[32]  Paul Barford,et al.  Data Poisoning Attacks against Autoregressive Models , 2016, AAAI.

[33]  Eduardo Fidalgo,et al.  Classifying suspicious content in tor darknet through Semantic Attention Keypoint Filtering , 2019, Digit. Investig..

[34]  Peter Stone,et al.  Autonomous agents modelling other agents: A comprehensive survey and open problems , 2017, Artif. Intell..

[35]  Onur Ozdemir,et al.  Automated Vulnerability Detection in Source Code Using Deep Representation Learning , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[36]  Yevgeniy Vorobeychik,et al.  Data Poisoning Attacks on Factorization-Based Collaborative Filtering , 2016, NIPS.

[37]  Fengyuan Xu,et al.  A Privacy-Preserving Deep Learning Approach for Face Recognition with Edge Computing , 2018, HotEdge.

[38]  Ping Wang,et al.  Offline Dictionary Attack on Password Authentication Schemes Using Smart Cards , 2013, ISC.

[39]  Kincho H. Law,et al.  A Standardized Representation of Convolutional Neural Networks for Reliable Deployment of Machine Learning Models in the Manufacturing Industry , 2019 .

[40]  Matthew U. Scherer.  Regulating Artificial Intelligence Systems: Risks, Challenges, Competencies, and Strategies , 2015 .

[41]  Jiliang Zhang,et al.  Adversarial Examples: Opportunities and Challenges , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[42]  Thomas Engel,et al.  A Car Hacking Experiment: When Connectivity Meets Vulnerability , 2015, 2015 IEEE Globecom Workshops (GC Wkshps).

[43]  Shouhuai Xu,et al.  VulDeePecker: A Deep Learning-Based System for Vulnerability Detection , 2018, NDSS.

[44]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[45]  Ishwarya,et al.  A Self-Diagnosis Medical Chatbot Using Artificial Intelligence , 2018 .

[46]  Battista Biggio.  Machine Learning under Attack: Vulnerability Exploitation and Security Measures , 2016, IH&MMSec.

[47]  Mohsen Guizani,et al.  Deep Learning for IoT Big Data and Streaming Analytics: A Survey , 2017, IEEE Communications Surveys & Tutorials.

[48]  Vassil Roussev,et al.  An evaluation of forensic similarity hashes , 2011, Digit. Investig..

[49]  Rajiv V. Dharaskar,et al.  Study of Mobile Botnets: An Analysis from the Perspective of Efficient Generalized Forensics Framework for Mobile Devices , 2012 .

[50]  Banu Diri,et al.  Investigating the effect of dataset size, metrics sets, and feature selection techniques on software fault prediction problem , 2009, Inf. Sci..

[51]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[52]  Ajmal Mian,et al.  Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey , 2018, IEEE Access.

[53]  Oliver Bendel.  The synthetization of human voices , 2017, AI & SOCIETY.

[54]  Guillermo L. Grinblat,et al.  Toward Large-Scale Vulnerability Discovery using Machine Learning , 2016, CODASPY.

[55]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[56]  Daniel S. Hoadley,et al.  Artificial Intelligence and National Security , 1986 .

[57]  Vassil Roussev,et al.  Data Fingerprinting with Similarity Digests , 2010, IFIP Int. Conf. Digital Forensics.

[58]  Eugene H. Spafford,et al.  An Event-Based Digital Forensic Investigation Framework , 2004 .

[59]  Avideh Zakhor,et al.  Sensor fusion for semantic segmentation of urban scenes , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[60]  David A. Wagner,et al.  Audio Adversarial Examples: Targeted Attacks on Speech-to-Text , 2018, 2018 IEEE Security and Privacy Workshops (SPW).

[61]  Ananthram Swami,et al.  Practical Black-Box Attacks against Machine Learning , 2016, AsiaCCS.

[62]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[63]  Imran Sarwar Bajwa,et al.  An Empirical Study on Forensic Analysis of Urdu Text Using LDA-Based Authorship Attribution , 2019, IEEE Access.

[64]  Wenchang Shi,et al.  A Survey on Digital Forensics in Internet of Things , 2020, IEEE Internet of Things Journal.

[65]  David Wagner,et al.  Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods , 2017, AISec@CCS.

[66]  Tao Zhang,et al.  An exploration on artificial intelligence application: From security, privacy and ethic perspective , 2017, 2017 IEEE 2nd International Conference on Cloud Computing and Big Data Analysis (ICCCBDA).

[67]  Sencun Zhu,et al.  Semantics-Based Obfuscation-Resilient Binary Code Similarity Comparison with Applications to Software and Algorithm Plagiarism Detection , 2017, IEEE Transactions on Software Engineering.

[68]  Eldad Eilam,et al.  Reversing: Secrets of Reverse Engineering , 2005 .

[69]  M. Gentzkow,et al.  Social Media and Fake News in the 2016 Election , 2017 .

[70]  Lujo Bauer,et al.  Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition , 2016, CCS.

[71]  Mourad Debbabi,et al.  MalDy: Portable, data-driven malware detection using natural language processing and machine learning techniques on behavioral analysis reports , 2018, Digit. Investig..

[72]  Ar Kar Kyaw,et al.  Dictionary attack on Wordpress: Security and forensic analysis , 2015, 2015 Second International Conference on Information Security and Cyber Forensics (InfoSec).

[73]  Hany Farid,et al.  Evading Deepfake-Image Detectors with White- and Black-Box Attacks , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[74]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[75]  Tatiana Tropina,et al.  Self- and Co-regulation in Cybercrime, Cybersecurity and National Security , 2015, SpringerBriefs in Cybersecurity.

[76]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[77]  Michael P. Wellman,et al.  SoK: Security and Privacy in Machine Learning , 2018, 2018 IEEE European Symposium on Security and Privacy (EuroS&P).

[78]  Cristiano André da Costa,et al.  Survey of conversational agents in health , 2019, Expert Syst. Appl..

[79]  David V. Gioe.  The History of Fake News , 2017 .

[80]  Ali Dehghantanha,et al.  Internet of Things security and forensics: Challenges and opportunities , 2018, Future Gener. Comput. Syst..

[81]  Konstantin Beznosov,et al.  Key Challenges in Defending Against Malicious Socialbots , 2012, LEET.

[82]  M. Yar,et al.  Book Review: Cybercrime and Society , 2006 .

[83]  Kui Ren,et al.  Distributionally Adversarial Attack , 2018, AAAI.

[84]  Hamid Reza Shahriari,et al.  Software Vulnerability Analysis and Discovery Using Machine-Learning and Data-Mining Techniques , 2017, ACM Comput. Surv..

[85]  Domenico Beneventano,et al.  Computing inter-document similarity with Context Semantic Analysis , 2018, Inf. Syst..

[86]  Bo An,et al.  Efficient Label Contamination Attacks Against Black-Box Learning Models , 2017, IJCAI.

[87]  Jason Yosinski,et al.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[88]  Abhishek Singhal,et al.  A literature survey on social engineering attacks: Phishing attack , 2016, 2016 International Conference on Computing, Communication and Automation (ICCCA).

[89]  Xin Li,et al.  Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[90]  Kristen Grauman,et al.  Supervoxel-Consistent Foreground Propagation in Video , 2014, ECCV.

[91]  Haralambos Mouratidis,et al.  A survey on cloud forensics challenges and solutions , 2016, Secur. Commun. Networks.

[92]  Jong Hyuk Park,et al.  Social network security: Issues, challenges, threats, and solutions , 2017, Inf. Sci..

[93]  David A. Wagner,et al.  Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples , 2018, ICML.

[94]  Fabio Roli,et al.  Adversarial attacks against intrusion detection systems: Taxonomy, solutions and open issues , 2013, Inf. Sci..

[95]  Patrick D. McDaniel,et al.  Adversarial Examples for Malware Detection , 2017, ESORICS.

[96]  David W. McDonald,et al.  Dissecting a Social Botnet: Growth, Content and Influence in Twitter , 2015, CSCW.

[97]  John Woods,et al.  Survey on Chatbot Design Techniques in Speech Conversation Systems , 2015 .

[98]  Antonella Santone,et al.  Car hacking identification through fuzzy logic algorithms , 2017, 2017 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[99]  Liang Tong,et al.  Improving Robustness of ML Classifiers against Realizable Evasion Attacks Using Conserved Features , 2017, USENIX Security Symposium.

[100]  Chang Liu,et al.  Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning , 2018, 2018 IEEE Symposium on Security and Privacy (SP).

[101]  Lynn Margaret Batten,et al.  Reproducibility of Digital Evidence in Forensic Investigations , 2005, DFRWS.

[102]  Filippo Menczer,et al.  BotOrNot: A System to Evaluate Social Bots , 2016, WWW.

[103]  Amina Adadi,et al.  Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) , 2018, IEEE Access.

[104]  Matthew Wilson,et al.  Collaborative intelligence: How human and artificial intelligence create value along the B2B sales funnel , 2020, Business Horizons.

[105]  Arun Lakhotia,et al.  Identifying Shared Software Components to Support Malware Forensics , 2014, DIMVA.

[106]  Vitaly Shmatikov,et al.  Membership Inference Attacks Against Machine Learning Models , 2016, 2017 IEEE Symposium on Security and Privacy (SP).

[107]  Nick Bostrom,et al.  Future Progress in Artificial Intelligence: A Survey of Expert Opinion , 2013, PT-AI.

[108]  Filippo Menczer,et al.  The rise of social bots , 2014, Commun. ACM.

[109]  Jan Hendrik Metzen,et al.  On Detecting Adversarial Perturbations , 2017, ICLR.

[110]  Francesco Colace,et al.  Chatbot: An Education Support System for Student , 2018, CSS.

[111]  Craig Valli,et al.  Malware Forensics: Discovery of the Intent of Deception , 2010, J. Digit. Forensics Secur. Law.

[112]  Hutan Ashrafian,et al.  AIonAI: A Humanitarian Law of Artificial Intelligence and Robotics , 2015, Sci. Eng. Ethics.

[113]  Scott E. Coull,et al.  Exploring Adversarial Examples in Malware Detection , 2018, 2019 IEEE Security and Privacy Workshops (SPW).

[114]  Atul Prakash,et al.  Robust Physical-World Attacks on Deep Learning Visual Classification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[115]  Lilian Mitrou,et al.  Data Protection, Artificial Intelligence and Cognitive Services: Is the General Data Protection Regulation (GDPR) ‘Artificial Intelligence-Proof’? , 2018 .

[116]  Somesh Jha,et al.  Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures , 2015, CCS.

[117]  Ming Zhou,et al.  SuperAgent: A Customer Service Chatbot for E-commerce Websites , 2017, ACL.

[118]  Matthew C. Stamm,et al.  Learned Forensic Source Similarity for Unknown Camera Models , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[119]  Radha Poovendran,et al.  Blocking Transferability of Adversarial Examples in Black-Box Learning Systems , 2017, ArXiv.

[120]  J. Clough.  Principles of Cybercrime , 2010, The Military Law and the Law of War Review.

[121]  Leonidas J. Guibas,et al.  A scalable active framework for region annotation in 3D shape collections , 2016, ACM Trans. Graph..

[122]  Jesse D. Kornblum.  Identifying almost identical files using context triggered piecewise hashing , 2006, Digit. Investig..

[123]  Yann LeCun,et al.  Road Scene Segmentation from a Single Image , 2012, ECCV.

[124]  Olga Ohrimenko,et al.  Contamination Attacks and Mitigation in Multi-Party Machine Learning , 2018, NeurIPS.

[125]  Patrick D. McDaniel,et al.  On the (Statistical) Detection of Adversarial Examples , 2017, ArXiv.

[126]  Ali Dehghantanha,et al.  A Survey on Digital Forensics Trends , 2014 .