One of the most critical challenges in the industry is to maintain assets in good condition to avoid breakdowns and downtimes. Therefore, effective automated visual inspection process is a key practice to proactively streamline maintenance, quality control and safety. Despite the fact that automated visual inspection is usually implemented with fixed-cameras or robotic arms, in several industrial cases, a fixed infrastructure is not possible. For those cases, one solution is to use mobile infrastructure with cameras manipulated by operators (robot or human). However, mobile infrastructure is highly impacted by aspects like occlusion, orientation, lighting conditions, dimensions and the overall localization and navigation of the operator. To address these issues, we present a guided visual inspection approach enabled by AI-based models to automate defect detection and ensure the quality of collected data and inspection results. The solution includes two methods: 1) guided data acquisition and 2) 3-level defect analysis. The guided data acquisition method facilitates multi-point inspection using 2D object detection models instead of traditional 3D reconstruction models. Our detection methods not only detect assets but also its conditions and localize objects with 92% accuracy and demonstrated to be 10-13x faster than a 3D model. The 3-level defect analysis method learns defect representations by combining object detection and classification models. The method reduces false-positive for small datasets, achieving accuracy by at least 89% with custom CNN and up to 98% with transfer learning. Finally, our approach provides a general framework applicable to multiple inspection domains and robot or human operators.