The appearance of deep neural networks for Side-Channel leads to strong power analysis techniques for detecting secret information of physical cryptography implementations. Generally, deep learning techniques do not suffer the difficulties of template attacks such as trace misalignment. However, the generalization of a trained deep neural network that can accurately predict Side-Channel leakages largely depends on its adjustable variables (parameters of a neural network). Although pre-training is no longer mandatory, it is needed for parameter selection of a deep neural network to improve the success rate and provide a better insight into the network’s inner functionality. In this paper, we propose a novel model via Twin support vector with a deep kernel approach when targeting a hardware implementation of AES-128. The proposed model is pre-trained with the Restricted Boltzmann Machine method in a layer-wise manner and then fine-tuned via gradient descent. Further, we used the grid search technique for the selection of each hyperparameter which is used to compute class probabilities of every test trace in our deep model based side-channel attack. Based on our analysis and experiments, this model empirically shows its effectiveness by outperforming some of its competitors in profiling attack methods such as convolutional neural networks and multilayer perceptron models. We also evaluate our model on both masked and unmasked AES implementation. The results indicate that the proposed approach has achieved a success rate of greater than 99% even with a single trace using Keras library with Tensorflow. We investigate the correct “key rank” according to the number of traces; our model reaches the key $ rank\leq 10 $ when attacking the third AES SBox.