3D Data Acquisition and Registration Using Two Opposing Kinects

We present an automatic, open source data acquisition and calibration approach using two opposing RGBD sensors (Kinect V2) and demonstrate its efficacy for dynamic object reconstruction in the context of monitoring for remote lung function assessment. First, the relative pose of the two RGBD sensors is estimated through a calibration stage and rigid transformation parameters are computed. These are then used to align and register point clouds obtained from the sensors at frame level. We validated the proposed system by performing experiments on known-size box objects with the results demonstrating accurate measurements. We also report on dynamic object reconstruction by way of human subjects undergoing respiratory functional assessment.

[1]  Duane C. Brown,et al.  Close-Range Camera Calibration , 1971 .

[2]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[3]  William Schroeder,et al.  The Visualization Toolkit: An Object-Oriented Approach to 3-D Graphics , 1997 .

[4]  Maysam Ghovanloo,et al.  A Vision-Based Respiration Monitoring System for Passive Airway Resistance Estimation , 2016, IEEE Transactions on Biomedical Engineering.

[5]  Joan Lasenby,et al.  SLP: A Zero-Contact Non-Invasive Method for Pulmonary Function Testing , 2010, BMVC.

[6]  Matthias Nießner,et al.  VolumeDeform: Real-Time Volumetric Non-rigid Reconstruction , 2016, ECCV.

[7]  Oliver G. Staadt,et al.  Calibration of Depth Camera Arrays , 2014, SIGRAD.

[8]  Zhengyou Zhang,et al.  Iterative point matching for registration of free-form curves and surfaces , 1994, International Journal of Computer Vision.

[9]  J. Challis A procedure for determining rigid body transformation parameters. , 1995, Journal of biomechanics.

[10]  Maysam Ghovanloo,et al.  A passive quantitative measurement of airway resistance using depth data , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[11]  Majid Mirmehdi,et al.  Remote pulmonary function testing using a depth sensor , 2015, 2015 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[12]  Ming-Sui Lee,et al.  Noncontact respiratory measurement of volume change using depth camera , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[13]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[14]  Andreas Geiger,et al.  Automatic camera and range sensor calibration using a single shot , 2012, 2012 IEEE International Conference on Robotics and Automation.

[15]  Tim Weyrich,et al.  A practical structured light acquisition system for point-based geometry and texture , 2005, Proceedings Eurographics/IEEE VGTC Symposium Point-Based Graphics, 2005..

[16]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[17]  Marek Kowalski,et al.  Livescan3D: A Fast and Inexpensive 3D Data Acquisition System for Multiple Kinect v2 Sensors , 2015, 2015 International Conference on 3D Vision.

[18]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[19]  Petros Daras,et al.  Fast and smooth 3D reconstruction using multiple RGB-Depth sensors , 2014, 2014 IEEE Visual Communications and Image Processing Conference.

[20]  Roberto Cipolla,et al.  Multiview Photometric Stereo , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  James Reinders,et al.  Intel® threading building blocks , 2008 .

[22]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[23]  Björn E. Ottersten,et al.  RGB-D Multi-view System Calibration for Full 3D Scene Reconstruction , 2014, 2014 22nd International Conference on Pattern Recognition.

[24]  P. Rokita,et al.  A Survey of Passive 3D Reconstruction Methods on the Basis of More than One Image , 2012, Machine Graphics and Vision.

[25]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[26]  Michael Arens,et al.  Low-cost commodity depth sensor comparison and accuracy analysis , 2014, Security and Defence.

[27]  Oscar Meruvia Pastor,et al.  DeReEs: Real-Time Registration of RGBD Images Using Image-Based Feature Detection And Robust 3D Correspondence Estimation and Refinement , 2014, IVCNZ '14.

[28]  Roland Siegwart,et al.  Comparing ICP variants on real-world data sets , 2013, Auton. Robots.

[29]  Andrew W. Fitzgibbon,et al.  Real-time non-rigid reconstruction using an RGB-D camera , 2014, ACM Trans. Graph..

[30]  Daniel G. Aliaga,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 a Self-calibrating Method for Photogeometric Acquisition of 3d Objects , 2022 .

[31]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[32]  Sebastian Thrun,et al.  Unsupervised extrinsic calibration of depth sensors in dynamic scenes , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Oscar Meruvia Pastor,et al.  Automatic and adaptable registration of live RGBD video streams , 2015, MIG.

[34]  Dieter Fox,et al.  DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Reinhard Koch,et al.  Time-of-Flight sensor calibration for accurate range sensing , 2010, Comput. Vis. Image Underst..

[36]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Bernd Fröhlich,et al.  Immersive Group-to-Group Telepresence , 2013, IEEE Transactions on Visualization and Computer Graphics.

[38]  Christopher Schwartz,et al.  Fusing Structured Light Consistency and Helmholtz Normals for 3D Reconstruction , 2012, BMVC.

[39]  Petros Daras,et al.  Real-Time, Full 3-D Reconstruction of Moving Foreground Objects From Multiple Consumer Depth Cameras , 2013, IEEE Transactions on Multimedia.

[40]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[41]  Henry Fuchs,et al.  Encumbrance-free telepresence system with real-time 3D capture and display using commodity depth cameras , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[42]  Bernd Fröhlich,et al.  Volumetric calibration and registration of multiple RGBD-sensors into a joint coordinate system , 2015, 2015 IEEE Symposium on 3D User Interfaces (3DUI).

[43]  Jianfei Cai,et al.  Registration of multiple RGBD cameras via local rigid transformations , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[44]  Dieter Schmalstieg,et al.  OmniKinect: real-time dense volumetric data acquisition and applications , 2012, VRST '12.

[45]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[46]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[47]  Carlos Hernandez,et al.  Multi-View Stereo: A Tutorial , 2015, Found. Trends Comput. Graph. Vis..