It is very important for the stable operation of spacecraft to realize the fault diagnosis of aluminum alloy stiffened plate, but it may be difficult to accurately identify the damage with some traditional methods due to its special rib structure. This paper presents a fault diagnosis method based on convolution neural network (CNN) to deal with impact damage recognition of aluminum alloy stiffened plates, which can accurately identify the impact location and damage degree (including no damage, pothole and perforation). For this purpose, the damage identification model was constructed by using CNN with double output, and the impact signal containing the impact position and damage degree information was obtained by combining the shock stress wave detection system and finite element simulation. Furthermore, the frequency data set is generated through the fast Fourier transform of the impact signal, and it is input into CNN to further extract the features. Finally, the position and damage degree are identified by two output layers respectively. To validate the effectiveness of the proposed model, the models of neural network and support vector machine are used for comparison. The results show that the method is well performed in accuracy and rapidity. The accuracies of this method reach 99.85% and 99.61% in impact location and damage degree.