Implementation of discriminative and generative deep learning

Deep learning methods allow classifier to learn features through multiple layers training. Low level features are abstracted into high level features in learning process. In this paper, we proposed a probabilistic deep learning method which combine a discriminative model named Support Vector Machines (SVMs) with a generative model namely Gaussian Mixture Model (GMM). We propose a definition to deciding which part of the data is certain (which have been learned successfully in the current layer) and which is uncertain data that need to be passed to the next layer. Bayesian rule is used to represent the output data from the previous layer of SVM and GMM. The experiment results show that the proposed deep learning model can make it easier to classify uncertain data through multiple training and give more accurate results.