Monocular Video-Based Trailer Coupler Detection Using Multiplexer Convolutional Neural Network

This paper presents an automated monocular-camera-based computer vision system for autonomous self-backing-up a vehicle towards a trailer, by continuously estimating the 3D trailer coupler position and feeding it to the vehicle control system, until the alignment of the tow hitch with the trailers coupler. This system is made possible through our proposed distance-driven Multiplexer-CNN method, which selects the most suitable CNN using the estimated coupler-to-vehicle distance. The input of the multiplexer is a group made of a CNN detector, trackers, and 3D localizer. In the CNN detector, we propose a novel algorithm to provide a presence confidence score with each detection. The score reflects the existence of the target object in a region, as well as how accurate is the 2D target detection. We demonstrate the accuracy and efficiency of the system on a large trailer database. Our system achieves an estimation error of 1.4 cm when the ball reaches the coupler, while running at 18.9 FPS on a regular PC.

[1]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Myoungho Sunwoo,et al.  Enhanced Road Boundary and Obstacle Detection Using a Downward-Looking LIDAR Sensor , 2012, IEEE Transactions on Vehicular Technology.

[3]  Rudolf Mester,et al.  Free Space Computation Using Stochastic Occupancy Grids and Dynamic Programming , 2008 .

[4]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[5]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[6]  Yi Li,et al.  DeepTrack: Learning Discriminative Feature Representations by Convolutional Neural Networks for Visual Tracking , 2014, BMVC.

[7]  David Cebon,et al.  A Visual Template-Matching Method for Articulation Angle Measurement , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[8]  Michelle Birdsall.  Google and ITE: The Road Ahead for Self-Driving Cars , 2014 .

[9]  Peter H. Tu,et al.  Facial Contour Labeling via Congealing , 2010, ECCV.

[10]  René Vidal,et al.  Projective Factorization of Multiple Rigid-Body Motions , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Andrea Vedaldi,et al.  MatConvNet: Convolutional Neural Networks for MATLAB , 2014, ACM Multimedia.

[12]  Zhe Chen,et al.  An Experimental Survey on Correlation Filter-based Tracking , 2015, ArXiv.

[13]  Shi-Huang Chen,et al.  Vision-Based Distance Estimation for Multiple Vehicles Using Single Optical Camera , 2011, 2011 Second International Conference on Innovations in Bio-inspired Computing and Applications.

[14]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Xiaoming Liu,et al.  On developing and enhancing plant-level disease rating systems in real fields , 2016, Pattern Recognit..

[16]  Sergiu Nedevschi,et al.  Processing Dense Stereo Data Using Elevation Maps: Road Surface, Traffic Isle, and Obstacle Detection , 2010, IEEE Transactions on Vehicular Technology.

[17]  Massimo Bertozzi,et al.  Real-time obstacle detection using stereo vision for autonomous ground vehicles: A survey , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[18]  Ming-Hsuan Yang,et al.  Long-term correlation tracking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Jorge L. Martínez,et al.  Driver assistance system for backward maneuvers in passive multi-trailer vehicles , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Xiaogang Wang,et al.  STCT: Sequentially Training Convolutional Networks for Visual Tracking , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Zhijiang Shao,et al.  Precise trajectory optimization for articulated wheeled vehicles in cluttered environments , 2016, Adv. Eng. Softw..

[22]  Ian H. Witten,et al.  Induction of model trees for predicting continuous classes , 1996 .

[23]  Michael Felsberg,et al.  Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking , 2016, ECCV.

[24]  Shiyu Song,et al.  Joint SFM and detection cues for monocular 3D localization in road scenes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Ryan M. Eustice,et al.  Visual localization within LIDAR maps for automated urban driving , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[27]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[28]  Hongdong Li,et al.  Robust Visual Tracking with Deep Convolutional Neural Network Based Object Proposals on PETS , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[29]  Rui Caseiro,et al.  High-Speed Tracking with Kernelized Correlation Filters , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Zhe Chen,et al.  MUlti-Store Tracker (MUSTer): A cognitive psychology inspired approach to object tracking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).