Rapid 3D object detection and modeling using range data from 3D range imaging camera for heavy equipment operation

Automated detection and modeling of 3D objects located in a construction work environment is critical for autonomous heavy equipment operation. Such automation allows for accurate, efficient, and autonomous operation of heavy equipment in a broad range of construction tasks by providing interactive background information. This paper proposes a 3D object detection and modeling system which utilizes range data obtained by 3D range imaging camera to generate 3D object models with an acceptable level of accuracy in a few seconds. The proposed system consists of four steps: data acquisition, pre-processing, object segmentation, and 3D model generation. The system was tested on the modeling of different classes of construction objects on actual construction sites. The results show that the proposed 3D object detection and modeling system achieves a good balance between speed and accuracy, and hence could be used to enhance efficiency and productivity in the autonomous operation of heavy equipment.

[1]  Carlos H. Caldas,et al.  Modeling job sites in real time to improve safety during equipment operation , 2006, SPIE Smart Structures and Materials + Nondestructive Evaluation and Health Monitoring.

[2]  Maris Juberts,et al.  Towards the Ultimate Construction Site Sensor , 2002 .

[3]  Jie Gong,et al.  Data processing for real-time construction site spatial modeling , 2008 .

[4]  Qi Li,et al.  Range image noise suppression in laser imaging system , 2009 .

[5]  Fabio Remondino.  From point cloud to surface , 2003 .

[6]  Roland Siegwart,et al.  Introduction to Autonomous Mobile Robots , 2004 .

[7]  Hans-Gerd Maas,et al.  ANALYSIS OF THE INFORMATION CONTENT OF TERRESTRIAL LASERSCANNER POINT CLOUDS FOR THE AUTOMATIC DETERMINATION OF FOREST INVENTORY PARAMETERS , 2006 .

[8]  Gian Luca Foresti,et al.  Automatic detection and indexing of video-event shots for surveillance applications , 2002, IEEE Trans. Multim..

[9]  M. Uijt de Haag,et al.  Implementation of a Flash-LADAR aided inertial navigator , 2008, 2008 IEEE/ION Position, Location and Navigation Symposium.

[10]  Tomaso Poggio,et al.  A Trainable Object Detection System: Car Detection in Static Images , 1999 .

[11]  Rihua Xiang,et al.  Range image segmentation based on split-merge clustering , 2004, ICPR 2004.

[12]  Katsushi Ikeuchi,et al.  Generating an interpretation tree from a CAD model for 3D-object recognition in bin-picking tasks , 1987, International Journal of Computer Vision.

[13]  Zheng Lin,et al.  Unseeded Region Growing for 3D Image Segmentation , 2000, VIP.

[14]  Carlos H. Caldas,et al.  Real-Time Three-Dimensional Occupancy Grid Modeling for the Detection and Tracking of Construction Resources , 2007 .

[15]  George Vosselman,et al.  Segmentation of point clouds using smoothness constraints , 2006 .

[16]  Marcin Novotni,et al.  Correspondence Generation and Matching of 3D Shape Subparts , 2005 .

[17]  Yali Amit,et al.  2D Object Detection and Recognition: Models, Algorithms, and Networks , 2002 .

[18]  Changwan Kim,et al.  Rapid, on-site spatial information acquisition and its use for infrastructure operation and maintenance , 2005 .

[19]  Katsushi Ikeuchi,et al.  Illumination from Shadows , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Baoxin Li,et al.  Homography-based ground detection for a mobile robot platform using a single camera , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[21]  Robert D. Habbit,et al.  Utilization of flash ladar for cooperative and uncooperative rendezvous and capture , 2003, SPIE Defense + Commercial Sensing.

[22]  Nevin Doss,et al.  3D Modeling of Biological Structures , 2004 .

[23]  David P. Dobkin,et al.  The quickhull algorithm for convex hulls , 1996, TOMS.

[24]  Roger D. Eastman,et al.  Training and optimization of operating parameters for flash LADAR cameras , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[25]  Ioannis Stamos,et al.  Range Image Segmentation for Modeling and Object Detection in Urban Scenes , 2007, Sixth International Conference on 3-D Digital Imaging and Modeling (3DIM 2007).

[26]  Constantin Paleologu,et al.  Efficient Method for Content Extraction Applied in Multimedia Communication , 2008, Seventh International Conference on Networking (icn 2008).

[27]  Brian Curless,et al.  From range scans to 3D models , 1999, COMG.

[28]  Larry S. Davis,et al.  Closely coupled object detection and segmentation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[29]  R. Madhavan,et al.  Applications of a 3D Range Camera Towards Healthcare Mobility Aids , 2006, 2006 IEEE International Conference on Networking, Sensing and Control.

[30]  Jochen Teizer.  3D range imaging camera sensing for active safety in construction , 2008, J. Inf. Technol. Constr..

[31]  Burcu Akinci,et al.  A formalism for utilization of sensor systems and integrated project models for active construction quality control , 2006 .

[32]  Carlos H. Caldas,et al.  Human-Assisted Obstacle Avoidance System using 3D Workspace Modeling for Construction Equipment Operation , 2006 .

[33]  Frédéric Bosché,et al.  Automated retrieval of 3D CAD model objects in construction range images , 2008 .

[34]  Frank Ade,et al.  OBJECT DETECTION AND TRACKING IN RANGE IMAGE SEQUENCES BY SEPARATION OF IMAGE FEATURES , 1998 .

[35]  Jitendra Malik,et al.  Recognizing Objects in Range Data Using Regional Point Descriptors , 2004, ECCV.

[36]  Hui Chen,et al.  3D free-form object recognition in range images using local surface patches , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[37]  Rae-Hong Park,et al.  3D object recognition in range images using hidden markov models and neural networks , 1999, Pattern Recognit..

[38]  Thomas A. Funkhouser,et al.  Min-cut based segmentation of point clouds , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[39]  Sanjiv Singh,et al.  A Robotic Excavator for Autonomous Truck Loading , 1999, Auton. Robots.

[40]  S. V. Sreenivasan,et al.  Rapid Human-Assisted Creation of Bounding Models for Obstacle Avoidance in Construction , 2004 .

[41]  P. Anandan,et al.  A Unified Approach to Moving Object Detection in 2D and 3D Scenes , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Hanseok Ko,et al.  Multiple vehicle tracking based on regional estimation in nighttime CCD images , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[43]  P. Grussenmeyer,et al.  Joint combination of point cloud and DSM for 3D building reconstruction using airborne laser scanner data , 2007, 2007 Urban Remote Sensing Joint Event.

[44]  Jacques Verly,et al.  Some applications of mathematical morphology to range imagery , 1987, ICASSP '87. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[45]  Zhuowen Tu,et al.  Range image segmentation by an effective jump-diffusion method , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  G. Sithole,et al.  Recognising structure in laser scanning point clouds , 2004 .

[47]  S. Pizer,et al.  The Image Processing Handbook , 1994 .

[48]  Sang Uk Lee,et al.  Split-and-merge segmentation employing thresholding technique , 1997, Proceedings of International Conference on Image Processing.

[49]  Deepali Kelkar,et al.  Improved Quadtree Method for Split Merge Image Segmentation , 2008, 2008 First International Conference on Emerging Trends in Engineering and Technology.

[50]  Changwan KimC. Kim,et al.  Applicability of flash laser distance and ranging to three-dimensional spatial information acquisition and modeling on a construction site , 2008 .