Spam Filtering Using Regularized Neural Networks with Rectified Linear Units

The rapid growth of unsolicited and unwanted messages has inspired the development of many anti-spam methods. Machine-learning methods such as Naive Bayes NB, support vector machines SVMs or neural networks NNs have been particularly effective in categorizing spam /non-spam messages. They automatically construct word lists and their weights usually in a bag-of-words fashion. However, traditional multilayer perceptron MLP NNs usually suffer from slow optimization convergence to a poor local minimum and overfitting issues. To overcome this problem, we use a regularized NN with rectified linear units RANN-ReL for spam filtering. We compare its performance on three benchmark spam datasets Enron, SpamAssassin, and SMS spam collection with four machine algorithms commonly used in text classification, namely NB, SVM, MLP, and k-NN. We show that the RANN-ReL outperforms other methods in terms of classification accuracy, false negative and false positive rates. Notably, it classifies well both major legitimate and minor spam classes.