Motorcycles that See: Multifocal Stereo Vision Sensor for Advanced Safety Systems in Tilting Vehicles

Advanced driver assistance systems, ADAS, have shown the possibility to anticipate crash accidents and effectively assist road users in critical traffic situations. This is not the case for motorcyclists, in fact ADAS for motorcycles are still barely developed. Our aim was to study a camera-based sensor for the application of preventive safety in tilting vehicles. We identified two road conflict situations for which automotive remote sensors installed in a tilting vehicle are likely to fail in the identification of critical obstacles. Accordingly, we set two experiments conducted in real traffic conditions to test our stereo vision sensor. Our promising results support the application of this type of sensors for advanced motorcycle safety applications.

[1]  Gaurav S. Sukhatme,et al.  Simultaneous mapping and stereo extrinsic parameter calibration using GPS measurements , 2011, 2011 IEEE International Conference on Robotics and Automation.

[2]  Uwe Franke,et al.  6D-Vision: Fusion of Stereo and Motion for Robust Environment Perception , 2005, DAGM-Symposium.

[3]  Marco Pierini,et al.  Is stereo vision a suitable remote sensing approach for motorcycle safety? An analysis of LIDAR, RADAR, and machine vision technologies subjected to the dynamics of a tilting vehicle. , 2018 .

[4]  P M Panchal,et al.  A Comparison of SIFT and SURF , 2013 .

[5]  Marco Pierini,et al.  Advanced accident research system based on a medical and engineering data in the metropolitan area of Florence , 2013, BMC Emergency Medicine.

[6]  Marco Pierini,et al.  Obstacle detection test in real-word traffic contexts for the purposes of motorcycle autonomous emergency braking (MAEB) , 2017, ArXiv.

[7]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Rama Chellappa,et al.  Statistical Error Propagation in 3D Modeling From Monocular Video , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[9]  Marco Dozza,et al.  How do drivers overtake cyclists? , 2016, Accident; analysis and prevention.

[10]  Sanshiro Shishido,et al.  4.7 A 2.1Mpixel organic-film stacked RGB-IR image sensor with electrically controllable IR sensitivity , 2017, 2017 IEEE International Solid-State Circuits Conference (ISSCC).

[11]  I Isaksson-Hellman,et al.  Evaluation of rear-end collision avoidance technologies based on real world crash data , 2015 .

[12]  Emile A. Hendriks,et al.  Adaptive stereo similarity fusion using confidence measures , 2015, Comput. Vis. Image Underst..

[13]  P.-E Dziadek,et al.  Eco-friendly application of EVs for home-to-work and home-to-education transports , 2013, 2013 IEEE International Conference on Industrial Technology (ICIT).

[14]  N. Suganuma,et al.  Obstacle detection using Virtual Disparity Image for non-flat road , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[15]  Larry H. Matthies,et al.  Error modeling in stereo navigation , 1986, IEEE J. Robotics Autom..

[16]  Fabrice Meriaudeau,et al.  A SURVEY ON OUTDOOR WATER HAZARD DETECTION , 2009 .

[17]  Joachim Weickert,et al.  Priors for Stereo Vision under Adverse Weather Conditions , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[18]  Sergiu Nedevschi,et al.  Processing Dense Stereo Data Using Elevation Maps: Road Surface, Traffic Isle, and Obstacle Detection , 2010, IEEE Transactions on Vehicular Technology.

[19]  Massimo Bertozzi,et al.  Real-time obstacle detection using stereo vision for autonomous ground vehicles: A survey , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[20]  Marco Pierini,et al.  Inevitable Collision States for Motorcycle-to-Car Collision Scenarios , 2016, IEEE Transactions on Intelligent Transportation Systems.

[21]  R. Danescu,et al.  High accuracy stereo vision system for far distance obstacle detection , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[22]  Atsushi Imiya,et al.  On Performance Analysis of Optical Flow Algorithms , 2011, Theoretical Foundations of Computer Vision.

[23]  Stefano Mattoccia,et al.  Quantitative Evaluation of Confidence Measures in a Machine Learning World , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Irene Isaksson-Hellman,et al.  Evaluation of the crash mitigation effect of low-speed automated emergency braking systems based on insurance claims data , 2016, Traffic injury prevention.

[25]  Richard Szeliski,et al.  Sampling the disparity space image , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Uwe Franke,et al.  The Stixel World - A Compact Medium Level Representation of the 3D-World , 2009, DAGM-Symposium.

[27]  Shmuel Peleg,et al.  Image sequence enhancement using sub-pixel displacements , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Jurgen Hasch,et al.  Feasibility of automotive radar at frequencies beyond 100 GHz , 2012, International Journal of Microwave and Wireless Technologies.

[29]  N. Suganuma,et al.  An Obstacle Extraction Method Using Virtual Disparity Image , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[30]  Sergiu Nedevschi,et al.  Stereovision Approach for Obstacle Detection on Non-Planar Roads , 2004, ICINCO.

[31]  Simon Lacroix,et al.  Simultaneous Localization and Mapping with Stereovision , 2003, ISRR.

[32]  Prithvi Simha,et al.  Disruptive Innovation on Two Wheels: Chinese Urban Transportation and Electrification of the Humble Bike , 2016 .

[33]  Niccolò Baldanzini,et al.  Are automatic systems the future of motorcycle safety? A novel methodology to prioritize potential safety solutions based on their projected effectiveness , 2017, Traffic injury prevention.

[34]  Luigi di Stefano,et al.  Learning confidence measures in the wild , 2017, BMVC.

[35]  Marco Pierini,et al.  Further Development of Motorcycle Autonomous Emergency Braking (MAEB), What Can In-Depth Studies Tell Us? A Multinational Study , 2014, Traffic injury prevention.

[36]  Rahul Nair,et al.  Ensemble Learning for Confidence Measures in Stereo Vision , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Jongchul Lee,et al.  Improved census transform for noise robust stereo matching , 2016 .

[38]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[39]  Sanshiro Shishido,et al.  A 2.1-Mpixel Organic Film-Stacked RGB-IR Image Sensor With Electrically Controllable IR Sensitivity , 2018, IEEE Journal of Solid-State Circuits.

[40]  Andrea Tonoli,et al.  Urban and Extra Urban Vehicles: Re-Thinking the Vehicle Design , 2011 .

[41]  Allen R. Hanson,et al.  Obstacle Detection Based on Qualitative and Quantitative 3D Reconstruction , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Benjamin Tseng,et al.  Towards the Development of the IEEE P1858 CPIQ Standard – A Validation Study , 2017 .

[43]  Yann LeCun,et al.  Computing the stereo matching cost with a convolutional neural network , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  A. Barth,et al.  Where will the oncoming vehicle be the next second? , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[45]  U. Franke,et al.  Monocular Video serves RADAR-based Emergency Braking , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[46]  Eric Wahl,et al.  Efficient stereo matching for moving cameras and decalibrated rigs , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[47]  Kanchan Cecil,et al.  An Analytical Study of SIFT and SURF in Image Registration , 2014 .

[48]  Riender Happee,et al.  The Deployment of Advanced Driver Assistance Systems in Europe , 2015 .

[49]  Yi Ruan,et al.  Government׳s role in disruptive innovation and industry emergence: The case of the electric bike in China , 2014 .

[50]  Horst Bischof,et al.  Using Self-Contradiction to Learn Confidence Measures in Stereo Vision , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Tran Duc Chung,et al.  An automobile detection algorithm development for automated emergency braking system , 2014, 2014 51st ACM/EDAC/IEEE Design Automation Conference (DAC).

[52]  Z. Hu,et al.  U-V-disparity: an efficient algorithm for stereovision based scene analysis , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[53]  Marco Pierini,et al.  Evaluation of the end-of-life performance of a hybrid scooter with the application of recyclability and recoverability assessment methods , 2016 .

[54]  Michael M. Zavlanos,et al.  Exact bias correction and covariance estimation for stereo vision , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[56]  Martin Lauer,et al.  Online stereo camera calibration from scratch , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[57]  Raúl Rojas,et al.  Weighted Semi-Global Matching and Center-Symmetric Census Transform for Robust Driver Assistance , 2013, CAIP.

[58]  Niccolò Baldanzini,et al.  Analysis of the minimum swerving distance for the development of a motorcycle autonomous braking system. , 2013, Accident; analysis and prevention.

[59]  Sebastian Ramos,et al.  Detecting unexpected obstacles for self-driving cars: Fusing deep learning and geometric modeling , 2016, 2017 IEEE Intelligent Vehicles Symposium (IV).

[60]  Luigi di Stefano,et al.  Real-Time Stereo within the VIDET Project , 2002, Real Time Imaging.

[61]  Stefan K. Gehrig,et al.  Exploiting the Power of Stereo Confidences , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[62]  Alberto Broggi,et al.  Obstacle Detection with Stereo Vision for Off-Road Vehicle Navigation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[63]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[64]  T. Williamson.  A high-performance stereo vision system for obstacle detection , 1998 .

[65]  Krystian Mikolajczyk,et al.  Evaluation of local detectors and descriptors for fast feature matching , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[66]  A A Parker,et al.  In Europe 250 watt pedelecs reduce pollution and improve the safety and mobility of young and elderly riders , 2011 .

[67]  Clark F. Olson,et al.  Stereo ego-motion improvements for robust rover navigation , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[68]  N Haworth,et al.  Powered two wheelers in a changing world-challenges and opportunities. , 2012, Accident; analysis and prevention.

[69]  Matthew N. Dailey,et al.  Simultaneous Localization and Mapping with Stereo Vision , 2006, 2006 9th International Conference on Control, Automation, Robotics and Vision.

[70]  Alex Zelinsky,et al.  Learning OpenCV---Computer Vision with the OpenCV Library (Bradski, G.R. et al.; 2008)[On the Shelf] , 2009, IEEE Robotics & Automation Magazine.

[71]  Andreas Geiger,et al.  Automatic camera and range sensor calibration using a single shot , 2012, 2012 IEEE International Conference on Robotics and Automation.

[72]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[73]  Heiko Hirschmüller,et al.  Evaluation of Stereo Matching Costs on Images with Radiometric Differences , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[74]  Kostas Daniilidis,et al.  Understanding noise sensitivity in structure from motion , 1996 .

[75]  Peter I. Corke,et al.  Quantitative Evaluation of Matching Methods and Validity Measures for Stereo Vision , 2001, Int. J. Robotics Res..

[76]  Pierre Vandergheynst,et al.  FREAK: Fast Retina Keypoint , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[77]  Masahiro Kobayashi,et al.  4.5 A 1.8erms− temporal noise over 110dB dynamic range 3.4µm pixel pitch global shutter CMOS image sensor with dual-gain amplifiers, SS-ADC and multiple-accumulation shutter , 2017, 2017 IEEE International Solid-State Circuits Conference (ISSCC).

[78]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[79]  Uwe Franke,et al.  Improving Stereo Sub-Pixel Accuracy for Long Range Stereo , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[80]  Richard I. Hartley,et al.  In Defense of the Eight-Point Algorithm , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[81]  Marco Pierini,et al.  A Pilot Study of an Integrated Accident Research System Based on a Medical and Engineering Data in the Metropolitan Area of Florence , 2012 .

[82]  Rama Chellappa,et al.  Statistical Analysis of Inherent Ambiguities in Recovering 3-D Motion from a Noisy Flow Field , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[83]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[84]  Jessica B. Cicchino,et al.  Effectiveness of forward collision warning and autonomous emergency braking systems in reducing front-to-rear crash rates. , 2017, Accident; analysis and prevention.

[85]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[86]  H. Hirschmüller.  Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information , 2005, CVPR.

[87]  Gaurav S. Sukhatme,et al.  Bias Reduction and Filter Convergence for Long Range Stereo , 2005, ISRR.

[88]  Emile A. Hendriks,et al.  Stereo Similarity Metric Fusion Using Stereo Confidence , 2014, 2014 22nd International Conference on Pattern Recognition.

[89]  Olivier D. Faugeras,et al.  The fundamental matrix: Theory, algorithms, and stability analysis , 2004, International Journal of Computer Vision.

[90]  A Lie,et al.  Effectiveness of low speed autonomous emergency braking in real-world rear-end crashes. , 2015, Accident; analysis and prevention.

[91]  Brett Browning,et al.  Online continuous stereo extrinsic parameter estimation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[92]  Sajid Saleem,et al.  A Performance Evaluation of SIFT and SURF for Multispectral Image Matching , 2012, ICIAR.

[93]  Robert C. Bolles,et al.  Generalizing Epipolar-Plane Image Analysis on the spatiotemporal surface , 2004, International Journal of Computer Vision.

[94]  N. Qian.  Binocular Disparity and the Perception of Depth , 1997, Neuron.

[95]  Nicholas J Garber,et al.  Infrastructure Countermeasures to Mitigate Motorcyclist Crashes in Europe , 2012 .

[96]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[97]  Shaojie Shen,et al.  High-precision online markerless stereo extrinsic calibration , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[98]  Marco Pierini,et al.  Electric L-category Vehicles for Smart Urban Mobility , 2016 .

[99]  Nathan J. Short,et al.  Modal-Based Camera Correction for Large Pitch Stereo Imaging , 2011 .

[100]  Alberto Broggi,et al.  The Single Frame Stereo Vision System for Reliable Obstacle Detection Used during the 2005 DARPA Grand Challenge on TerraMax , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[101]  Marco Pierini,et al.  Satellite markers: a simple method for ground truth car pose on stereo video , 2018, International Conference on Machine Vision.

[102]  Sergiu Nedevschi,et al.  Design of Interpolation Functions for Subpixel-Accuracy Stereo-Vision Systems , 2012, IEEE Transactions on Image Processing.

[103]  Masatoshi Okutomi,et al.  Precise subpixel estimation on area-based matching , 2002, Syst. Comput. Jpn..

[104]  Ben Upcroft,et al.  Online calibration of stereo rigs for long-term autonomy , 2013, 2013 IEEE International Conference on Robotics and Automation.

[105]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[106]  Stefan Gehrig,et al.  Stereo matching in the presence of sub-pixel calibration errors , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[107]  Rudolf Mester,et al.  Free Space Computation Using Stochastic Occupancy Grids and Dynamic Programming , 2008 .

[108]  J. Dekoster,et al.  Cycling : the way ahead for towns and cities , 1999 .

[109]  Janne Heikkilä,et al.  A four-step camera calibration procedure with implicit image correction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[110]  Timo Scharwächter,et al.  Stixel-Based Target Existence Estimation under Adverse Conditions , 2013, GCPR.

[111]  Martial Hebert,et al.  A behavior-based system for off-road navigation , 1994, IEEE Trans. Robotics Autom..

[112]  Taro Sekine.  Utilization of probe powered two-wheeler vehicles to realize a safe mobile society , 2014 .

[113]  Euntai Kim,et al.  Wet area and puddle detection for Advanced Driver Assistance Systems (ADAS) using a stereo camera , 2016, International Journal of Control, Automation and Systems.

[114]  Uwe Franke,et al.  Fast stereo based object detection for stop&go traffic , 1996, Proceedings of Conference on Intelligent Vehicles.

[115]  Bernard Rougé,et al.  Satellite attitude instability effects on stereo images , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[116]  Frank Will,et al.  Tomorrow's Car - For Today's People: Can Tilting Three Wheeled Vehicles be a Solution for the Problems of Today and the Future? , 2011 .

[117]  Sergiu Nedevschi,et al.  SORT-SGM: Subpixel Optimized Real-Time Semiglobal Matching for Intelligent Vehicles , 2012, IEEE Transactions on Vehicular Technology.