Rear-Stitched View Panorama: A Low-Power Embedded Implementation for Smart Rear-View Mirrors on Vehicles

Automobiles are currently equipped with a three-mirror system for rear-view visualization. The two side-view mirrors show close the periphery on the left and right sides of the vehicle, and the center rear-view mirror is typically adjusted to allow the driver to see through the vehicle's rear windshield. This three-mirror system, however, imposes safety concerns in requiring drivers to shift their attention and gaze to look in each mirror to obtain a full visualization of the rear-view surroundings, which takes attention off the scene in front of the vehicle. We present an alternative to the three-mirror rear-view system, which we call Rear-Stitched View Panorama (RSVP). The proposed system uses four rear-facing cameras, strategically placed to overcome the traditional blind spot problem, and stitches the feeds from each camera together to generate a single panoramic view, which can display the entire rear surroundings. We project individually captured frames onto a single virtual view using precomputed system calibration parameters. Then we determine optimal seam lines, along which the images are fused together to form the single RSVP view presented to the driver. Furthermore, we highlight techniques that enable efficient embedded implementation of the system and showcase a real-time system utilizing under 2W of power, making it suitable for in-cabin deployment in vehicles.

[1]  Richard Szeliski,et al.  Eliminating ghosting and exposure artifacts in image mosaics , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[2]  Uwe Franke,et al.  Efficient representation of traffic scenes by means of dynamic stixels , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[3]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  Santokh Singh,et al.  Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey , 2015 .

[5]  Jean-Philippe Tarel,et al.  Real time obstacle detection in stereovision on non flat road geometry through "v-disparity" representation , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[6]  Arturo de la Escalera,et al.  Traffic sign recognition and analysis for intelligent vehicles , 2003, Image Vis. Comput..

[7]  Sergiu Nedevschi,et al.  Processing Dense Stereo Data Using Elevation Maps: Road Surface, Traffic Isle, and Obstacle Detection , 2010, IEEE Transactions on Vehicular Technology.

[8]  Luis Salgado,et al.  Real-time lane tracking using Rao-Blackwellized particle filter , 2012, Journal of Real-Time Image Processing.

[9]  Xiaohui Liu,et al.  Real-time traffic sign recognition from video by class-specific discriminative features , 2010, Pattern Recognit..

[10]  Kwanghoon Sohn,et al.  Gradient-Enhancing Conversion for Illumination-Robust Lane Detection , 2013, IEEE Transactions on Intelligent Transportation Systems.

[11]  Dariu Gavrila,et al.  The Issues , 2011 .

[12]  James Davis,et al.  Mosaics of scenes with moving objects , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[13]  Massimo Bertozzi,et al.  Stereo vision-based vehicle detection , 2000, Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No.00TH8511).

[14]  David Salesin,et al.  Interactive digital photomontage , 2004, SIGGRAPH 2004.

[15]  Fan Zhang,et al.  Parallax-Tolerant Image Stitching , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Xiaohong W. Gao,et al.  Recognition of traffic signs based on their colour and shape features extracted using human vision models , 2006, J. Vis. Commun. Image Represent..

[17]  Sergiu Nedevschi,et al.  Stereo Image Processing for ADAS and Pre-Crash Systems , .

[18]  Mohan M. Trivedi,et al.  Panoramic stitching for driver assistance and applications to motion saliency-based risk analysis , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[19]  Yong-Sheng Chen,et al.  Bird's-Eye View Vision System for Vehicle Surrounding Monitoring , 2008, RobVis.

[20]  Marco Campani,et al.  Robust method for road sign detection and recognition , 1996, Image Vis. Comput..

[21]  David Gerónimo Gómez,et al.  Survey of Pedestrian Detection for Advanced Driver Assistance Systems , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Jannik Fritsch,et al.  A new performance measure and evaluation benchmark for road detection algorithms , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[23]  George Bebis,et al.  Temporal consistent fast stereo matching for advanced driver assistance systems (ADAS) , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[24]  Alan C. Bovik,et al.  Virtual top-view camera calibration for accurate object representation , 2016, 2016 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI).

[25]  Massimo Bertozzi,et al.  GOLD: a parallel real-time stereo vision system for generic obstacle and lane detection , 1998, IEEE Trans. Image Process..

[26]  Tim J. Dennis,et al.  Epipolar line estimation and rectification for stereo image pairs , 1996, IEEE Trans. Image Process..

[27]  Vikram V. Appia,et al.  A Diverse Low Cost High Performance Platform for Advanced Driver Assistance System (ADAS) Applications , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[28]  Sergiu Nedevschi,et al.  Real-time obstacle detection using dense stereo vision and dense optical flow , 2010, Proceedings of the 2010 IEEE 6th International Conference on Intelligent Computer Communication and Processing.

[29]  Thomas B. Moeslund,et al.  Vision-Based Traffic Sign Detection and Analysis for Intelligent Driver Assistance Systems: Perspectives and Survey , 2012, IEEE Transactions on Intelligent Transportation Systems.