UTAF: A Universal Approach to Task-Agnostic Model Fingerprinting

Protecting the intellectual property (IP) of deep neural networks (DNN) becomes an urgent concern for IT corporations. For model piracy forensics, previous model fingerprinting schemes are commonly based on adversarial examples constructed for the owner’s model as the fingerprint, and verify whether a suspect model is indeed pirated from the original model by matching the behavioral pattern on the fingerprint examples between one another. However, these methods heavily rely on the characteristics of classification tasks which inhibits their application to more general scenarios. To address this issue, we present Universal Task-Agnostic Fingerprinting (UTAF), the first task-agnostic model fingerprinting framework which enables fingerprinting on a much wider range of DNNs independent from the downstream learning task, and exhibits strong robustness against a variety of ownership obfuscation techniques. Specifically, we generalize previous schemes into two critical design components in UTAF: the adaptive fingerprint and the meta-verifier, which are jointly optimized such that the meta-verifier learns to determine whether a suspect model is stolen based on the concatenated outputs of the suspect model on the adaptive fingerprint. As a key of being task-agnostic, the full process makes no assumption on the model internals in the ensemble only if they have the same input and output dimensions. Spanning classification, regression and generative modeling, extensive experimental results validate the substantially improved performance of UTAF over the state-of-the-art fingerprinting schemes and demonstrate the enhanced generality of UTAF for providing task-agnostic fingerprinting. For example, on fingerprinting ResNet-18 trained for skin cancer diagnosis, UTAF achieves simultaneously 100% true positives and 100% true negatives on a diverse test set of 70 suspect models, achieving an about 220% relative improvement in ARUC in comparison to the optimal baseline.

[1]  Mi Zhang,et al.  Enhancing diversity in Top-N recommendation , 2009, RecSys '09.

[2]  Cao Yuan,et al.  “Identity Bracelets” for Deep Neural Networks , 2020, IEEE Access.

[3]  Neil J. Hurley,et al.  Evaluating the Diversity of Top-N Recommendations , 2009, 2009 21st IEEE International Conference on Tools with Artificial Intelligence.

[4]  Kevin Fu,et al.  Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving , 2019, CCS.

[5]  Jan Hendrik Witte,et al.  Deep Learning for Finance: Deep Portfolios , 2016 .

[6]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[7]  Alok Aggarwal,et al.  Regularized Evolution for Image Classifier Architecture Search , 2018, AAAI.

[8]  Samuel Marchal,et al.  PRADA: Protecting Against DNN Model Stealing Attacks , 2018, 2019 IEEE European Symposium on Security and Privacy (EuroS&P).

[9]  Neil J. Hurley,et al.  A Double-Ranking Strategy for Long-Tail Product Recommendation , 2012, 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology.

[10]  Tsung-Yi Ho,et al.  CloudLeak: Large-Scale Deep Learning Models Stealing Through Adversarial Examples , 2020, NDSS.

[11]  Soheil Feizi,et al.  Compressing GANs using Knowledge Distillation , 2019, ArXiv.

[12]  Junbeom Hur,et al.  Neural Network Stealing via Meltdown , 2021, 2021 International Conference on Information Networking (ICOIN).

[13]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[14]  Josep Torrellas,et al.  Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architectures , 2018, USENIX Security Symposium.

[15]  Hanan Samet,et al.  Pruning Filters for Efficient ConvNets , 2016, ICLR.

[16]  Hui Wu,et al.  Protecting Intellectual Property of Deep Neural Networks with Watermarking , 2018, AsiaCCS.

[17]  David A. Wagner,et al.  Towards Evaluating the Robustness of Neural Networks , 2016, 2017 IEEE Symposium on Security and Privacy (SP).

[18]  Quoc V. Le,et al.  BAM! Born-Again Multi-Task Networks for Natural Language Understanding , 2019, ACL.

[19]  Geoffrey E. Hinton,et al.  Distilling the Knowledge in a Neural Network , 2015, ArXiv.

[20]  Yunxin Liu,et al.  ModelDiff: testing-based DNN similarity comparison for model reuse detection , 2021, ISSTA.

[21]  Shin'ichi Satoh,et al.  Embedding Watermarks into Deep Neural Networks , 2017, ICMR.

[22]  Neil J. Hurley,et al.  Statistical Modeling of Diversity in Top-N Recommender Systems , 2009, 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology.

[23]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[24]  Mi Zhang,et al.  Avoiding monotony: improving the diversity of recommendation lists , 2008, RecSys '08.

[25]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Zunping Cheng,et al.  Statistical attack detection , 2009, RecSys '09.

[27]  Neil J. Hurley,et al.  Novel Item Recommendation by User Profile Partitioning , 2009, 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology.

[28]  Florian Kerschbaum,et al.  Deep Neural Network Fingerprinting by Conferrable Adversarial Examples , 2019, ICLR.

[29]  Ilia Polian,et al.  Protecting artificial intelligence IPs: a survey of watermarking and fingerprinting for machine learning , 2021, CAAI Trans. Intell. Technol..

[30]  Benny Pinkas,et al.  Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring , 2018, USENIX Security Symposium.

[31]  Neil J. Hurley,et al.  Niche Product Retrieval in Top-N Recommendation , 2010, 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[32]  Jie Tang,et al.  Addressing cold start in recommender systems: a semi-supervised co-training algorithm , 2014, SIGIR.

[33]  Hui Xiong,et al.  Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting , 2020, AAAI.

[34]  Song Han,et al.  Learning both Weights and Connections for Efficient Neural Network , 2015, NIPS.

[35]  R. Altman,et al.  Pharmacogenomics Knowledge for Personalized Medicine , 2012, Clinical pharmacology and therapeutics.

[36]  Xudong Pan,et al.  Geographical Feature Extraction for Entities in Location-based Social Networks , 2018, WWW.

[37]  Franziska Boenisch,et al.  A Survey on Model Watermarking Neural Networks , 2020, ArXiv.

[38]  Yann LeCun,et al.  The Loss Surfaces of Multilayer Networks , 2014, AISTATS.

[39]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[40]  Jingjing Zhao,et al.  AFA: Adversarial fingerprinting authentication for deep neural networks , 2020, Comput. Commun..

[41]  Shouling Ji,et al.  Justinian's GAAvernor: Robust Distributed Learning with Gradient Aggregation Agent , 2020, USENIX Security Symposium.

[42]  Xudong Pan,et al.  Theoretical Analysis of Image-to-Image Translation with Adversarial Learning , 2018, ICML.

[43]  Fan Zhang,et al.  Stealing Machine Learning Models via Prediction APIs , 2016, USENIX Security Symposium.

[44]  Neil J. Hurley,et al.  Novelty and Diversity in Top-N Recommendation -- Analysis and Evaluation , 2011, TOIT.

[45]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[46]  Mi Zhang,et al.  Privacy Risks of General-Purpose Language Models , 2020, 2020 IEEE Symposium on Security and Privacy (SP).

[47]  Jianping Gou,et al.  Knowledge Distillation: A Survey , 2020, International Journal of Computer Vision.

[48]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[49]  Bingbing Ni,et al.  MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis , 2020, ArXiv.

[50]  Jinyuan Jia,et al.  IPGuard: Protecting the Intellectual Property of Deep Neural Networks via Fingerprinting the Classification Boundary , 2019, ArXiv.

[51]  Chip-Hong Chang,et al.  Fingerprinting Deep Neural Networks - a DeepFool Approach , 2021, 2021 IEEE International Symposium on Circuits and Systems (ISCAS).

[52]  Farinaz Koushanfar,et al.  DeepSigns: A Generic Watermarking Framework for IP Protection of Deep Learning Models , 2018, IACR Cryptol. ePrint Arch..

[53]  Min Yang,et al.  Enhancing State-of-the-art Classifiers with API Semantics to Detect Evolved Android Malware , 2020, CCS.

[54]  Shao-Yuan Li,et al.  BayDNN: Friend Recommendation with Bayesian Personalized Ranking Deep Neural Network , 2017, CIKM.

[55]  Xiangnan He,et al.  Modeling Extreme Events in Time Series Prediction , 2019, KDD.