Thyroid-associated ophthalmopathy (TAO)is one of the most common orbital diseases. The symptoms of the disease are not obvious in the early stage, using computers to assist doctors in TAO diagnosis has become an important method for the early examination in recent years. But it is difficult to diagnose the disease in the thyroid-associated ophthalmopathy CT images, the sample quantity is also small, which can not get good classification results. This paper proposes a recognition method based on transfer learning. Transfer learning can be applied to the data in the original domain, after the model is established, it can be used in other fields. This can save a lot of time and resources. This paper proposes a comprehensive detection network based on the doctor's routine diagnosis process and uses the transfer learning to construct the neural network and completes feature extraction and result classification. From the results, the method proposed in this paper can recognize and classify the diseases more effective compared with traditional classification algorithms, it can assist doctors in the early diagnosis of TAO, so as to help patients receive timely treatment.