Vegetation encroaching on overhead power lines can cause short circuit faults and pose a major threat to the security and stability of power grids. Therefore, establishing an effective visual detection algorithm to oversee potential circuit failures of the power lines is critical to the ongoing inspection of vegetation encroachment. This paper establishes a deep learning-based detection framework that utilizes the images obtained from vision sensors mounted on power transmission towers. The proposed detection framework includes three cascaded modules: (1) detection of vegetation regions based on the Faster Region Convolution Neural Network (Faster R-CNN), (2) detection of power lines based on the Hough transform, and (3) detection of vegetation encroachment based on an advanced stereovision (SV) algorithm. In particular, the proposed SV algorithm converts the detected two-dimensional (2D) image data of the vegetation and power lines to three-dimensional (3D) height and location results in order to obtain precise geographical locations. Case studies using field captured images provided by a Transmission System Operator (TSO) demonstrate the effectiveness of the proposed framework in detecting vegetation failures, thus improving overall reliability and reducing economic loss.