Machine learning techniques have been widely adopted to assist in data analysis in a variety of Internet of Things (IoT) systems. To enable flexible use of trained learning models, one viable solution is to leverage all categories of data from different applications to train a general model, which can be further tuned for applications through the tuning process. This process incurs additional overhead at the start, but makes later revision and iteration faster and more flexible. Nonetheless, due to limited computing capabilities, IoT devices cannot handle the training process of large data sets. To address this issue, in this article, we propose a general framework to adopt transfer learning in Industrial IoT (IIoT) systems. In our study, we categorize the application space of applying transfer learning to IIoT systems into four generic scenarios: 1) centralized transfer learning with large data sets; 2) distributed transfer learning with large data sets; 3) centralized transfer learning with small data sets; and 4) distributed transfer learning with small data sets. According to the characteristics of each scenario, we design workflows to apply the transfer learning technique. To demonstrate the efficacy of the approach, we apply our transfer learning technique to the task of IIoT component recognition. We use the known VGG-16 model and leverage T-Less industrial data sets to evaluate the performance of our approach in different scenarios. Via performance evaluation, our experimental results confirm the efficacy of our approach, which can not only reduce training time but also achieve higher accuracy, compared with the classical convolutional neural network (CNN) approach.