A next best view method based on self-occlusion information in depth images for moving object

The determination of next best view of a camera for moving object has wide application in dynamic object scenario, such as unmanned aerial vehicle and automatic recognition. The major challenge of this problem is how to determine the next best view while the visual object is moving. In this work, a novel next best view method based on self-occlusion information in depth images of moving object is proposed. Firstly, a depth image of moving object is acquired and self-occlusion detection is utilized in the acquired image. On this basis, the self-occlusion regions are modeled by utilizing space quadrilateral subdivision. Secondly, according to the modeling result, a method based on the idea of mean shift is proposed to calculate the result of self-occlusion avoidance corresponding to the current object. Thirdly, the second depth image of moving object is acquired, and the feature points in two images are detected and matched, then the 3D motion estimation is accomplished by the 3D coordinates of feature points which are matched. Finally, the next best view is determined by combining the result of self-occlusion avoidance and 3D motion estimation. Experimental results validate that the proposed method is feasible and has relatively high real-time performance.

[1]  Nikolaos Grammalidis,et al.  Capturing the intangible an introduction to the i-Treasures project , 2015, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[2]  Joachim Denzler,et al.  Online Next-Best-View Planning for Accuracy Optimization Using an Extended E-Criterion , 2010, 2010 20th International Conference on Pattern Recognition.

[3]  J. M. M. Montiel,et al.  ORB-SLAM: A Versatile and Accurate Monocular SLAM System , 2015, IEEE Transactions on Robotics.

[4]  Rafael Murrieta-Cid,et al.  Hierarchical Ray Tracing for Fast Volumetric Next-Best-View Planning , 2013, 2013 International Conference on Computer and Robot Vision.

[5]  Anders Heyden,et al.  Covariance Propagation and Next Best View Planning for 3D Reconstruction , 2012, ECCV.

[6]  Anselmo Lastra,et al.  Efficient Constraint Evaluation Algorithms for Hierarchical Next-Best-View Planning , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[7]  Marinos Ioannides,et al.  In the wild image retrieval and clustering for 3D cultural heritage landmarks reconstruction , 2014, Multimedia Tools and Applications.

[8]  Andrea Cavallaro,et al.  Multi-camera Scheduling for Video Production , 2011, 2011 Conference for Visual Media Production.

[9]  Michael M. Zavlanos,et al.  A hybrid control approach to the Next-Best-View problem using stereo vision , 2013, 2013 IEEE International Conference on Robotics and Automation.

[10]  Mateu Sbert,et al.  Automatic View Selection Using Viewpoint Entropy and its Application to Image‐Based Modelling , 2003, Comput. Graph. Forum.

[11]  Ruzena Bajcsy,et al.  Occlusions as a Guide for Planning the Next View , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Luc Van Gool,et al.  A unified framework for content-aware view selection and planning through view importance , 2014, BMVC.

[13]  Marinos Ioannides,et al.  Modelling of Static and Moving Objects: Digitizing Tangible and Intangible Cultural Heritage , 2017, Mixed Reality and Gamification for Cultural Heritage.

[14]  Yanning Zhang,et al.  Kinect based real-time synthetic aperture imaging through occlusion , 2015, Multimedia Tools and Applications.

[15]  Shi-Hui Zhang,et al.  A Self-Occlusion Detection Approach Based on Depth Image Using SVM: , 2012 .

[16]  Richard Pito,et al.  A Solution to the Next Best View Problem for Automated Surface Acquisition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Zhengxing Sun,et al.  Best view selection of 3D models based on unsupervised feature learning and discrimination ability , 2013, VINCI '13.

[18]  Bhaskara Marthi,et al.  Object disappearance for object discovery , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[20]  Georgios Miaoulis,et al.  Active learning of user’s preferences estimation towards a personalized 3D navigation of geo-referenced scenes , 2013, GeoInformatica.

[21]  Youfu Li,et al.  Information entropy-based viewpoint planning for 3-D object reconstruction , 2005, IEEE Transactions on Robotics.

[22]  Christophe De Vleeschouwer,et al.  Multi-sensored Vision for Autonomous Production of Personalized Video Summaries , 2010, UCMedia.

[23]  Gaurav S. Sukhatme,et al.  A probabilistic framework for next best view estimation in a cluttered environment , 2014, J. Vis. Commun. Image Represent..

[24]  Matthias Nießner,et al.  Combining Inertial Navigation and ICP for Real-time 3D Surface Reconstruction , 2014, Eurographics.

[25]  John Soldatos,et al.  Real-time video analysis and personalized media streaming environments for large scale athletic events , 2008, AREA '08.

[26]  Joachim Denzler,et al.  An Information Theoretic Approach for Next Best View Planning in 3-D Reconstruction , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[27]  Joachim Denzler,et al.  INCORPORATION OF A-PRIORI INFORMATION IN PLANNING THE NEXT BEST VIEW , 2006 .

[28]  Noel E. O'Connor,et al.  Evaluating a dancer's performance using kinect-based skeleton tracking , 2011, ACM Multimedia.

[29]  Daniela Giorgi,et al.  3D shape retrieval based on best view selection , 2010, 3DOR '10.

[30]  Subhashis Banerjee,et al.  Recognizing large 3-D objects through next view planning using an uncalibrated camera , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[31]  Christophe De Vleeschouwer,et al.  Detection and recognition of sports(wo)men from multiple views , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[32]  C. Ian Connolly,et al.  The determination of next best views , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[33]  Joachim Denzler,et al.  Combined GKLT Feature Tracking and Reconstruction for Next Best View Planning , 2009, DAGM-Symposium.

[34]  Jan-Michael Frahm,et al.  Next Best View Planning for Active Model Improvement , 2009, BMVC.

[35]  Junhao Xiao,et al.  Finding next best views for autonomous UAV mapping through GPU-accelerated particle simulation , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[36]  Peter K. Allen,et al.  Data acquisition and view planning for 3-D modeling tasks , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[37]  Andrea Cavallaro,et al.  Resource Allocation for Personalized Video Summarization , 2014, IEEE Transactions on Multimedia.

[38]  Gerd Hirzinger,et al.  A surface-based Next-Best-View approach for automated 3D model completion of unknown objects , 2011, 2011 IEEE International Conference on Robotics and Automation.

[39]  Tsuhan Chen,et al.  Active view selection for object and pose recognition , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[40]  Christophe Dumont,et al.  A next-best-view system for autonomous 3-D object reconstruction , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[41]  Menglong Yan,et al.  Building Reconstruction From High-Resolution Multiview Aerial Imagery , 2015, IEEE Geoscience and Remote Sensing Letters.

[42]  A. Maitra,et al.  miR-181c Regulates the Mitochondrial Genome, Bioenergetics, and Propensity for Heart Failure In Vivo , 2014, PloS one.