Vision-Based Parking-Slot Detection: A DCNN-Based Approach and a Large-Scale Benchmark Dataset

In the automobile industry, recent years have witnessed a growing interest in developing self-parking systems. For such systems, how to accurately and efficiently detect and localize the parking slots defined by regular line segments near the vehicle is a key and still unresolved issue. In fact, kinds of unfavorable factors, such as the diversity of ground materials, changes in illumination conditions, and unpredictable shadows caused by nearby trees, make the vision-based parking-slot detection much harder than it looks. In this paper, we attempt to solve this issue to some extent and our contributions are twofold. First, we propose a novel deep convolutional neural network (DCNN)-based parking-slot detection approach, namely, DeepPS, which takes the surround-view image as the input. There are two key steps in DeepPS, identifying all the marking points on the input image and classifying local image patterns formed by pairs of marking points. We formulate both of them as learning problems, which can be solved naturally by modern DCNN models. Second, to facilitate the study of vision-based parking-slot detection, a large-scale labeled dataset is established. This dataset is the largest in this field, comprising 12 165 surround-view images collected from typical indoor and outdoor parking sites. For each image, the marking points and parking slots are carefully labeled. The efficacy and efficiency of DeepPS have been corroborated on our collected dataset. To make our results fully reproducible, all the relevant source codes and the dataset have been made publicly available at https://cslinzhang.github.io/deepps/.

[1]  Ho Gi Jung,et al.  Uniform User Interface for Semiautomatic Parking Slot Marking Recognition , 2010, IEEE Transactions on Vehicular Technology.

[2]  Kwae-Hi Lee,et al.  Low cost design of parallel parking assist system based on an ultrasonic sensor , 2010 .

[3]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[4]  Fuchun Sun,et al.  HyperNet: Towards Accurate Region Proposal Generation and Joint Object Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  S. Deans The Radon Transform and Some of Its Applications , 1983 .

[6]  C. Vestri,et al.  3D parking assistant system , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Ho Gi Jung,et al.  Automatic Parking Space Detection and Tracking for Underground and Indoor Environments , 2016, IEEE Transactions on Industrial Electronics.

[10]  Ming Yang,et al.  Automatic Parking Based on a Bird's Eye View Vision System , 2014 .

[11]  Ho Gi Jung,et al.  Automatic free parking space detection by using motion stereo-based 3D reconstruction , 2010, Machine Vision and Applications.

[12]  Ho Gi Jung,et al.  3D VISION SYSTEM FOR THE RECOGNITION OF FREE PARKING SITE LOCATION , 2006 .

[13]  Shuicheng Yan,et al.  An HOG-LBP human detector with partial occlusion handling , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[14]  Seung-Woo Seo,et al.  Available Parking Slot Recognition Based on Slot Context Analysis , 2016 .

[15]  Seyed-Mohsen Moosavi-Dezfooli,et al.  DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Jürgen Dickmann,et al.  Detection of parked vehicles from a radar based occupancy grid , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[17]  N. Mattern,et al.  Free space determination for parking slots using a 3D PMD sensor , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[18]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Thomas Fechner,et al.  Parking lot measurement with 24 GHz short range automotive radar , 2015, 2015 16th International Radar Symposium (IRS).

[20]  Ho Gi Jung,et al.  Scanning Laser Radar-Based Target Position Designation for Parking Aid System , 2008, IEEE Transactions on Intelligent Transportation Systems.

[21]  Bernt Schiele,et al.  A Performance Evaluation of Single and Multi-feature People Detection , 2008, DAGM-Symposium.

[22]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[23]  Alois Knoll,et al.  Towards autonomous driving in a parking garage: Vehicle localization and tracking using environment-embedded LIDAR sensors , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[24]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[25]  Luc Van Gool,et al.  Seeking the strongest rigid detector , supplementary material , 2013 .

[26]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[27]  Hisashi Satonaka,et al.  DEVELOPMENT OF PARKING SPACE DETECTION USING AN ULTRASONIC SENSOR , 2006 .

[28]  Gunilla Borgefors,et al.  Hierarchical Chamfer Matching: A Parametric Edge Matching Algorithm , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Jason Yosinski,et al.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[32]  Byung-Sung Kim,et al.  Parking space detection using ultrasonic sensor in parking assistance system , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[33]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[34]  Ho Gi Jung,et al.  Parking Slot Markings Recognition for Automatic Parking Assist System , 2006, 2006 IEEE Intelligent Vehicles Symposium.

[35]  Larry S. Davis,et al.  Human detection using partial least squares analysis , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[36]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Ying Shen,et al.  Vision-Based Parking-Slot Detection: A Benchmark and A Learning-Based Approach , 2018, Symmetry.

[38]  Ming Xie,et al.  Vision-guided automatic parking for smart car , 2000, Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No.00TH8511).

[39]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[40]  David Barber,et al.  Bayesian reasoning and machine learning , 2012 .

[41]  Gamini Dissanayake,et al.  Simultaneous Localization and Mapping - Exactly Sparse Information Filters , 2011, New Frontiers in Robotics.

[42]  Ho Gi Jung,et al.  Full-automatic recognition of various parking slot markings using a hierarchical tree structure , 2013 .

[43]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[44]  Martial Hebert,et al.  Detection of parking spots using 2D range data , 2012, 2012 15th International IEEE Conference on Intelligent Transportation Systems.

[45]  Jürgen Dickmann,et al.  Parking space detection with hierarchical dynamic occupancy grids , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[46]  Ho Gi Jung,et al.  Sensor Fusion-Based Vacant Parking Slot Detection and Tracking , 2014, IEEE Transactions on Intelligent Transportation Systems.

[47]  Hui Chen,et al.  Surround view based parking lot detection and tracking , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[48]  Luis Salgado,et al.  Robust multiple lane road modeling based on perspective analysis , 2008, 2008 15th IEEE International Conference on Image Processing.

[49]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[50]  Pietro Perona,et al.  Pedestrian Detection: An Evaluation of the State of the Art , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Hideki Hashimoto,et al.  Development of advanced parking assistance system , 2003, IEEE Trans. Ind. Electron..

[52]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[53]  Ho Gi Jung,et al.  Structure Analysis Based Parking Slot Marking Recognition for Semi-automatic Parking System , 2006, SSPR/SPR.

[54]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Jiri Matas,et al.  Robust Detection of Lines Using the Progressive Probabilistic Hough Transform , 2000, Comput. Vis. Image Underst..

[56]  Ho Gi Jung,et al.  Light-Stripe-Projection-Based Target Position Designation for Intelligent Parking-Assist System , 2010, IEEE Transactions on Intelligent Transportation Systems.

[57]  Eric Wahl,et al.  Parking assistance using dense motion-stereo , 2011, Machine Vision and Applications.

[58]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[60]  Jonas Larsson,et al.  A semi-automated parallel parking system for passenger cars , 2006 .

[61]  N. Kaempchen,et al.  Stereo vision based pose estimation of parking lots using 3D vehicle models , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[62]  Kok Kiong Tan,et al.  Autonomous Reverse Parking System Based on Robust Path Generation and Improved Sliding Mode Control , 2015, IEEE Transactions on Intelligent Transportation Systems.

[63]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[64]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[65]  Toshiaki Kakinami,et al.  Evaluation of a Vision-Based Parking Assistance System , 2005 .