Dyslexia is categorized as learning disorder that influence the ability of reading, writing and spelling. In Malaysia, “Instrumen Senarai Semak Disleksia (ISD)” that is provided by Ministry of Education is used to detect dyslexic student at early stage. However, such evaluations are time consuming, non-standardize and can lead to a biasing result since the evaluation is based on the teacher’s experiences with the student. Hence, this research focus on the development of dyslexic handwriting recognition. The purpose of this research is to develop a transfer learning of Dyslexia handwriting recognition by using Convolutional Neural Network (CNN) based on famous architecture of handwriting recognition using of LeNet-5. Data augmentation and pre-processing was employed to a total of 138,500 handwriting image dataset before feeding it into network. The hyper-parameter of the model was tuned and analyzed to classify the 3 classes of dyslexic handwriting. The developed CNN model has successfully achieved a remarkable accuracy of 95.34% in classifying 3 classes of dyslexic handwriting. From the result, the objective in developing the CNN model for dyslexia handwriting recognition was successfully achieved.