With the rapid development of computer vision, techniques of machine vision and visual inspection have been applied into the inspection of catenary on high-speed railways. Visual inspection systems have been developed and super-high-resolution images are captured to check the status of catenary components. Automatic recognition of defects becomes very important since the number of images is too huge to be manually checked one by one. However, it is not easy for the development of recognition algorithms on catenary components. There are many types of defects to be checked on different kinds of catenary components, but the number of defect images is too small in real world. In this paper, a solution was proposed and implemented. An on-site data acquisition system was designed and developed, and different types of defects were manually made on different catenary components beforehand. Finally, a visual inspection database was successfully constructed, including plenty of different kinds of catenary components, different types of defects, in different inspection conditions. The visual inspection database will be of great use in the development and test of recognition algorithms for catenary.