It is an effective way to identify substation switch state using deep learning directly based on massive large image samples, which requires high-performance servers for off-line model training and high-quality industrial personal computer (IPC) for running models efficiently. The processing cost and delay will considerably increase by this means and the identify speed of robots for potential defects in the field reduces accordingly. Therefore, an image recognition method using only regular computers and IPC is proposed in this paper. Through target detection based on HSV (i.e. Hue, Saturation and Value) color space, this method firstly prefetch and preliminary screen the potential identifiers from large image samples, and subsequently trains a classification model with artifical neural network utilizing smaller samples labeled. Finally, target identifier can be located and identified through target detection, and then be used for recognizing switch state according to their relative positions. The experimental results show that with limit hardware resources, this method can process image samples efficiently and accurately based on robot vision. It is demonstrated to be a lightweight solution for precisely recognizing substation switch state.