A Tensor-Train Deep Computation Model for Industry Informatics Big Data Feature Learning

The deep computation model has been proved to be effective for big data hierarchical feature and representation learning in the tensor space. However, it requires expensively computational resources including high-performance computing units and large memory to train a deep computation model with a large number of parameters, limiting its effectiveness and efficiency for industry informatics big data feature learning. In this paper, a tensor-train deep computation model is presented for industry informatics big data feature learning. Specially, the tensor-train network is used to compress the parameters significantly by converting the dense weight tensors into the tensor-train format. Furthermore, a learning algorithm is implemented based on gradient descent and back-propagation to train the parameters of the presented tensor-train deep computation model. Extensive experiments are carried on STL-10, CUAVE, and SNAE2 to evaluate the presented model in terms of the approximation error, classification accuracy drop, parameters reduction, and speedup. Results demonstrate that the presented model can improve the training efficiency and save the memory space greatly for the deep computation model with small accuracy drops, proving its potential for industry informatics big data feature learning.

[1]  Xue-wen Chen,et al.  Big Data Deep Learning: Challenges and Perspectives , 2014, IEEE Access.

[2]  Victor C. M. Leung,et al.  Predicting Temporal Social Contact Patterns for Data Forwarding in Opportunistic Mobile Networks , 2017, IEEE Transactions on Vehicular Technology.

[3]  Laurence T. Yang,et al.  A Big Data-as-a-Service Framework: State-of-the-Art and Perspectives , 2018, IEEE Transactions on Big Data.

[4]  Ebru Arisoy,et al.  Low-rank matrix factorization for Deep Neural Network training with high-dimensional output targets , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[5]  Yueming Hu,et al.  Distributed Feature Selection for Efficient Economic Big Data Analysis , 2018, IEEE Transactions on Big Data.

[6]  Xianjun Deng,et al.  Localized Confident Information Coverage Hole Detection in Internet of Things for Radioactive Pollution Monitoring , 2017, IEEE Access.

[7]  Peng Li,et al.  Deep Convolutional Computation Model for Feature Learning on Big Data in Internet of Things , 2018, IEEE Transactions on Industrial Informatics.

[8]  Laurence T. Yang,et al.  An Improved Deep Computation Model Based on Canonical Polyadic Decomposition , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[9]  Yifan Gong,et al.  Restructuring of deep neural network acoustic models with singular value decomposition , 2013, INTERSPEECH.

[10]  Andrzej Cichocki,et al.  Era of Big Data Processing: A New Approach via Tensor Networks and Tensor Decompositions , 2014, ArXiv.

[11]  Laurence T. Yang,et al.  Energy-Efficient Scheduling for Real-Time Systems Based on Deep Q-Learning Model , 2019, IEEE Transactions on Sustainable Computing.

[12]  Yixin Chen,et al.  Compressing Neural Networks with the Hashing Trick , 2015, ICML.

[13]  Jie Wu,et al.  Dependable Structural Health Monitoring Using Wireless Sensor Networks , 2015, IEEE Transactions on Dependable and Secure Computing.

[14]  Md. Zakirul Alam Bhuiyan,et al.  A Survey on Deep Learning in Big Data , 2017, 22017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC).

[15]  Laurence T. Yang,et al.  A Tensor-Based Approach for Big Data Representation and Dimensionality Reduction , 2014, IEEE Transactions on Emerging Topics in Computing.

[16]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Robert H. Deng,et al.  Privacy-Preserving Outsourced Clinical Decision Support System in the Cloud , 2017, IEEE Transactions on Services Computing.

[18]  J.N. Gowdy,et al.  CUAVE: A new audio-visual database for multimodal human-computer interface research , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[19]  Laurence T. Yang,et al.  Deep Computation Model for Unsupervised Feature Learning on Big Data , 2016, IEEE Transactions on Services Computing.

[20]  Honglak Lee,et al.  An Analysis of Single-Layer Networks in Unsupervised Feature Learning , 2011, AISTATS.

[21]  Laurence T. Yang,et al.  Privacy-Preserving Double-Projection Deep Computation Model With Crowdsourcing on Cloud for Big Data Feature Learning , 2018, IEEE Internet of Things Journal.

[22]  Yingjiu Li,et al.  Lightweight Sharable and Traceable Secure Mobile Health System , 2020, IEEE Transactions on Dependable and Secure Computing.