Roadmark reconstruction from stereo-images acquired by a ground-based mobile mapping system. (Reconstruction de marquages routiers à partir d'images terrestres)

Despite advances in ground-based Mobile Mapping System (MMS), automatic feature reconstruction seems far from being reached. In this thesis, we focus on 3D roadmark reconstruction from images acquired by road looking cameras of a MMS stereo-rig in dense urban context. A new approach is presented, that uses 3D geometric knowledge of roadmarks and provides a centimetric 3D accuracy with a low level of generalisation. Two classes of roadmarks are studied: zebra-crossing and dashed-lines. The general strategy consists in three main steps. The first step provides 3D linked-edges. Edges are extracted in the left and right images. Then a matching algorithm that is based on dynamic programming optimisation matches the edges between the two images. A sub-pixel matching is computed by post processing and 3D linked-edges are provided by classical photogrammetric triangulation. The second step uses the known specification of roadmarks to perform a signature based filtering of 3D linked-edges. This step provides hypothetical candidates for roadmark objects. The last step can be seen as a validation step that rejects or accepts the candidates. The validated candidates are finely reconstructed. The adopted model consists of a quasi parallelogram for each strip of zebra-crossing or dashed-line. Each strip is constrained to be flat but the roadmark as a whole is not planar. The method is evaluated on a set of 150 stereo-pairs acquired in a real urban area under normal traffic conditions. The results show the validity of the approach in terms of robustness, completeness and geometric accuracy. The method is robust and deals properly with partially occluded roadmarks as well as damaged or eroded ones. The detection rate reaches 90% and the 3D accuracy is about 2-4 cm. Finally an application of reconstructed roadmarks is presented. They are used in georeferencing of the system. Most of the MMSs use direct georeferencing devices such as GPS/INS for their localisation. However in urban areas masks and multi-path errors corrupt the measurements and provide only 50 cm accuracy. In order to improve the localisation quality, we aim at matching ground-based images with calibrated aerial images of the same area. For this purpose roadmarks are used as matching objects. The validity of this method is demonstrated on a zebra-crossing example

[1]  Massimo Bertozzi,et al.  AN EXTENSION TO THE INVERSE PERSPECTIVE MAPPING TO HANDLE NON-FLAT ROADS , 1998 .

[2]  Christian Früh,et al.  3D model generation for cities using aerial photographs and ground level laser scans , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Jing-Yu Yang,et al.  A vision based road detection algorithm , 2004, IEEE Conference on Robotics, Automation and Mechatronics, 2004..

[4]  R Risack.  ROBUST LANE RECOGNITION EMBEDDED IN A REAL-TIME DRIVER ASSISTANCE SYSTEM , 1998 .

[5]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[6]  C Baillard,et al.  A STEREO MATCHING ALGORITHM FOR URBAN DIGITAL ELEVATION MODELS , 2000 .

[7]  H.-H. Nagel,et al.  Texture-based segmentation of road images , 1994, Proceedings of the Intelligent Vehicles '94 Symposium.

[8]  Jean-Philippe Tarel,et al.  On the design of a single lane-markings detectors regardless the on-board camera's position , 2003, IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No.03TH8683).

[9]  Laurent Moll,et al.  Real time correlation-based stereo: algorithm, implementations and applications , 1993 .

[10]  Cláudio Rosito Jung,et al.  A robust linear-parabolic model for lane following , 2004, Proceedings. 17th Brazilian Symposium on Computer Graphics and Image Processing.

[11]  I. Laptev,et al.  AUTOMATIC ROAD EXTRACTION BASED ON MULTI-SCALE MODELING, CONTEXT, AND SNAKES , 2002 .

[12]  G. Maillet,et al.  3D CITY MODELS: AN OPERATIONAL APPROACH USING AERIAL IMAGES AND CADASTRAL MAPS , 2003 .

[13]  Massimo Bertozzi,et al.  Stereo inverse perspective mapping: theory and applications , 1998, Image Vis. Comput..

[14]  Estimating Sensor Pose from Images of a Stereo Rig , 2004 .

[15]  Nicolas Simond,et al.  Trajectography of an uncalibrated stereo rig in urban environments , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[16]  Christian Heipke,et al.  Hierarchical approach to automatic road extraction from aerial imagery , 1995, Defense, Security, and Sensing.

[17]  M.M. Trivedi,et al.  An integrated, robust approach to lane marking detection and lane tracking , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[18]  Christian Früh,et al.  Data Processing Algorithms for Generating Textured 3D Building Facade Meshes from Laser Scans and Camera Images , 2005, International Journal of Computer Vision.

[19]  Jan Skaloud,et al.  A mobile mapping system for road data capture via a single camera , 2003 .

[20]  Minoru Okada,et al.  Road extraction by snake with inertia and differential features , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[21]  Cordelia Schmid,et al.  Automatic line matching across views , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  E. Baltsavias,et al.  A TEST OF AUTOMATIC ROAD EXTRACTION APPROACHES , 2006 .

[23]  R. Danescu,et al.  Increased Accuracy Stereo Approach for 3D Lane Detection , 2006, 2006 IEEE Intelligent Vehicles Symposium.

[24]  Jitendra Malik,et al.  An integrated stereo-based approach to automatic vehicle guidance , 1995, Proceedings of IEEE International Conference on Computer Vision.

[25]  Konrad Schindler,et al.  MetropoGIS: A semi-automatic city documentation system , 2002 .

[26]  Tomaso Poggio,et al.  A Theory of Human Stereo Vision , 1977 .

[27]  Klaus Dietmayer,et al.  Roadway detection and lane detection using multilayer laserscanner , 2005 .

[28]  Roberto Cipolla,et al.  Camera Calibration from Vanishing Points in Image of Architectural Scenes , 1999, BMVC.

[29]  Hong Wang,et al.  Stereo-Vision Based Real time Obstacle Detection for Urban Environments , 2003 .

[30]  Nicolas Simond,et al.  Détection robuste du plan de la route en milieu urbain , 2004 .

[31]  Ayman Habib.  Matching Road Edges in Stereo-Image Sequences Using Data Association Techniques , 2000 .

[32]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[33]  Michel Bilodeau,et al.  Road tracking, lane segmentation and obstacle recognition by mathematical morphology , 1992, Proceedings of the Intelligent Vehicles `92 Symposium.

[34]  S. Nedevschi,et al.  3D lane detection system based on stereovision , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[35]  C. Lindner,et al.  Consumer graphics cards for fast image processing based on the Pixel Shader 3 . 0 standard , 2005 .

[36]  Pierre Charbonnier,et al.  Road boundaries detection using color saturation , 1998, 9th European Signal Processing Conference (EUSIPCO 1998).

[37]  Ryosuke Shibasaki,et al.  Reconstructing a textured CAD model of an urban environment using vehicle-borne laser range scanners and line cameras , 2003, Machine Vision and Applications.

[38]  Gérard G. Medioni,et al.  Parallel Multiscale Stereo Matching Using Adaptive Smoothing , 1990, ECCV.

[39]  Aly A. Farag,et al.  Threshold selection based on histogram modeling , 1992, [Proceedings] 1992 IEEE International Conference on Systems, Man, and Cybernetics.

[40]  Josiane Zerubia,et al.  A Gibbs Point Process for Road Extraction from Remotely Sensed Images , 2004, International Journal of Computer Vision.

[41]  Steven W. Zucker,et al.  Continuous Relaxation and Local Maxima Selection: Conditions for Equivalence , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Andreas Koschan,et al.  3D reconstruction of road surfaces using an integrated multi-sensory approach , 2007 .

[43]  Ryosuke Shibasaki,et al.  Reconstructing urban 3D model using vehicle-borne laser range scanners , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[44]  P. Garnesson,et al.  MESSIE: un système multi spécialistes en vision. Application à l'interprétation en imagerie aérienne , 1992 .

[45]  John Laurent,et al.  Road surface inspection using laser scanners adapted for the high precision 3D measurements of large flat surfaces , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[46]  Christopher M. Kreucher,et al.  LANA: a lane extraction algorithm that uses frequency domain features , 1999, IEEE Trans. Robotics Autom..

[47]  N. H. C. Yung,et al.  Lane detection by orientation and length discrimination , 2000, IEEE Trans. Syst. Man Cybern. Part B.

[48]  Edwin R. Hancock,et al.  Grouping Line-segments using Eigenclustering , 2000, BMVC.

[49]  Fadi Dornaika,et al.  Efficient On-Board Stereo Vision Pose Estimation , 2007, EUROCAST.

[50]  Sergiu Nedevschi,et al.  Efficient and robust classification method using combined feature vector for lane detection , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[51]  Jin Gon Kim,et al.  Efficient extraction of road information for car navigation applications using road pavement markings obtained from aerial images , 2006 .

[52]  Junichi Maruyama,et al.  Robust estimation of planar regions for visual navigation using sequential stereo images , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[53]  Rongxing Li,et al.  AUTOMATIC RECONSTRUCTION OF ROAD CENTERLINES FROM MOBILE MAPPING IMAGE SEQUENCES , 1998 .

[54]  Massimo Bertozzi,et al.  GOLD: a parallel real-time stereo vision system for generic obstacle and lane detection , 1998, IEEE Trans. Image Process..

[55]  Ayman Habib,et al.  Automatic Extraction of Road Signs from Terrestrial Color Imagery , 1999 .

[56]  Michel Dhome,et al.  Outdoor autonomous navigation using monocular vision , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[57]  M. Cannon,et al.  The use of INS/GPS in a highway survey system , 1990, IEEE Symposium on Position Location and Navigation. A Decade of Excellence in the Navigation Sciences.

[58]  Ingemar J. Cox,et al.  A maximum-flow formulation of the N-camera stereo correspondence problem , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[59]  Dinggang Shen,et al.  Lane detection and tracking using B-Snake , 2004, Image Vis. Comput..

[60]  Ki-Sang Hong,et al.  Fast line segment grouping method for finding globally more favorable line segments , 2002, Pattern Recognit..

[61]  John E. W. Mayhew,et al.  Psychophysical and Computational Studies Towards a Theory of Human Stereopsis , 1981, Artif. Intell..

[62]  P. Charbonnier,et al.  Robust visual recognition of colour images , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[63]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[64]  Tomaso Poggio,et al.  Computational vision and regularization theory , 1985, Nature.

[65]  Karl C. Kluge,et al.  Statistical characterization of the visual characteristics of painted lane markings , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[66]  Chunsun Zhang,et al.  Updating of cartographic road databases by image analysis , 2001 .

[67]  Galen C. Hunt,et al.  Lineal Feature Extraction by Parallel Stick Growing , 1996, IRREGULAR.

[68]  J. D. Bossler,et al.  Development and Application of the Highway Mapping System of Ohio State University , 1995 .

[69]  Radu Horaud,et al.  Stereo Correspondence Through Feature Grouping and Maximal Cliques , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[70]  Jefferey A. Shufelt,et al.  PERFORMANCE EVALUATION FOR AUTOMATIC FEATURE EXTRACTION , 2000 .

[71]  Sridhar Lakshmanan,et al.  A deformable-template approach to lane detection , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[72]  N. El-Sheimy,et al.  VIASAT - A mobile highway survey system of high accuracy , 1993, Proceedings of VNIS '93 - Vehicle Navigation and Information Systems Conference.

[73]  Rachid Deriche,et al.  From Noisy Edge Points to 3D Reconstruction of a Scene: A Robust Approach and its Uncertainty Analysis , 1992 .

[74]  Jan Skaloud,et al.  Photogrammetry-derived Navigation Parameters for INS Kalman Filter Updates , 2004 .

[75]  Uwe Weidner,et al.  A new approach towards quantitative quality evaluation of 3D building models , 2003 .

[76]  Jon P. Mills,et al.  ISPRS Commission I Symposium: From Sensors to Imagery , 2006 .

[77]  M. Clabian,et al.  Single camera lane detection and tracking , 2005, Proceedings. 2005 IEEE Intelligent Transportation Systems, 2005..

[78]  Fawzi Nashashibi,et al.  Digitizing and 3D modeling of urban environments and roads using vehicle-borne laser scanner system , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[79]  Thomas O. Binford,et al.  Depth from Edge and Intensity Based Stereo , 1981, IJCAI.

[80]  C. Hilario,et al.  Detection and classification of road lanes with a frequency analysis , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[81]  Jean-Philippe Tarel,et al.  Real time obstacle detection in stereovision on non flat road geometry through "v-disparity" representation , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[82]  F. Goulette,et al.  An integrated on-board laser range sensing system for on-the-way city and road modelling , 2006 .

[83]  Marc Berthod,et al.  Subpixel contour matching using continuous dynamic programming , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[84]  David H. Douglas,et al.  ALGORITHMS FOR THE REDUCTION OF THE NUMBER OF POINTS REQUIRED TO REPRESENT A DIGITIZED LINE OR ITS CARICATURE , 1973 .

[85]  Joon Hee Han,et al.  Contour Matching Using Epipolar Geometry , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[86]  J P Frisby,et al.  PMF: A Stereo Correspondence Algorithm Using a Disparity Gradient Limit , 1985, Perception.

[87]  Naser El-Sheimy,et al.  ARVEE : AUTOMATIC ROAD GEOMETRY EXTRACTION SYSTEM FOR MOBILE MAPPING , 2007 .

[88]  Michael Brady,et al.  Road feature detection and estimation , 2003, Machine Vision and Applications.