Deep encrypted text categorization

Long short-term memory (LSTM) is a significant approach to capture the long-range temporal context in sequences of arbitrary length. This had shown astonishing performance in sentence and document modeling. To leverage this, we use LSTM network to the encrypted text categorization at character and word level of texts. These texts are transformed in to dense word-vectors by using bag-of-words embedding. Dense word vectors are fed in to recurrent layers to capture the contextual information and followed by dense and activation layer with nonlinear activation function such as softmax for classification. The optimal network architecture has found by conducting various experiments with varying network parameters and network structures. All the experiments are run up to 1000 epochs with learning rate in the range [0.01-0.5]. Most of the LSTM network structures substantially performed well in 5-fold cross-validation. Based on the 5-fold cross-validation results, we claim that the character level inputs are more efficient in dealing with the encrypted texts in comparison to word level, due to the fact that character level input keeps more information from low-level textual representations. Character level based LSTM models achieved highest accuracy as 0.99 and the word level achieved highest accuracy as 0.94 in the classification settings of 5-fold cross validation using LSTM networks. On the real-world test data of CDMC 2016 e-News categorization task, word level LSTM models attained its highest accuracy as 0.43.