New attacks on RNN based healthcare learning system and their detections

Abstract Advances in machine learning (ML) in recent years have enabled a wide range of applications such as data analytics, autonomous systems, and security diagnostics. For example, recurrent neural networks (RNNs), such as long short-term memory networks (LSTMs), serve as a fundamental building block for many sequence learning tasks. Many researchers construct learning based inference and decision making models using primitive learning modules (PLMs) hosted in popular developer platforms, e.g., github. However, most of the publicly available primitive learning modules (PLMs) are maintained by third parties and lack proper checking to ensure they have not been maliciously modified by adversaries. In this paper, we articulate a particular threat to Recurrent Neural Network (RNN) based ML systems by introducing a new attack, which adjusts the weights of a RNN-based model causing it to produce wrong prediction results. Via synthetic and real world datasets, we demonstrate that such an attack is feasible. Next, we propose a detection scheme which can be used to infer if a particular PLM used in a RNN-based ML system contains such malicious behaviors. Experimental results show that our RNN based attack algorithm decreases the system performance when weights of important features are modified during training. In addition, our results also show that our detection mechanism is useful in identifying such malicious PLM.