Body Part Modeling on the Phone

In this work we propose a software pipeline which uses the inbuilt RGB camera of an off-theshelf smartphone, to create a 3D model from a set of ordered images. The motion of the camera is tracked visually and a subset of the images is processed to create depthmaps that are then fused into a single point cloud. Subsequently a textured mesh can be extracted. The software runs entirely on the phone in real-time, transforming the mobile platform into a portable handheld 3D scanner. The resulting 3D models are compared to 3D models created by a typical structured light sensor for mobile devices.

[1]  Pushmeet Kohli,et al.  Fusion4D , 2016, ACM Trans. Graph..

[2]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[3]  Jan-Michael Frahm,et al.  Detailed Real-Time Urban 3D Reconstruction from Video , 2007, International Journal of Computer Vision.

[4]  Tom Drummond,et al.  Rapid scene reconstruction on mobile phones from panoramic images , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[5]  Victor S. Lempitsky,et al.  Seamless Mosaicing of Image-Based Texture Maps , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Frank Dellaert,et al.  IMU Preintegration on Manifold for Efficient Visual-Inertial Maximum-a-Posteriori Estimation , 2015, Robotics: Science and Systems.

[7]  Jan-Michael Frahm,et al.  Building Rome on a Cloudless Day , 2010, ECCV.

[8]  Marc Pollefeys,et al.  Turning Mobile Phones into 3D Scanners , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Reinhard Koch,et al.  Realistic surface reconstruction of 3D scenes from uncalibrated image sequences , 2000, Comput. Animat. Virtual Worlds.

[10]  Marc Pollefeys,et al.  Live Metric 3D Reconstruction on Mobile Phones , 2013, 2013 IEEE International Conference on Computer Vision.

[11]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[12]  Dieter Fox,et al.  DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Stergios I. Roumeliotis,et al.  Vision-Aided Inertial Navigation for Spacecraft Entry, Descent, and Landing , 2009, IEEE Transactions on Robotics.

[14]  Davide Scaramuzza,et al.  SVO: Fast semi-direct monocular visual odometry , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[15]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[16]  Matthias Nießner,et al.  Real-time 3D reconstruction at scale using voxel hashing , 2013, ACM Trans. Graph..

[17]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[18]  Pushmeet Kohli,et al.  MobileFusion: Real-Time Volumetric Surface Reconstruction and Dense Tracking on Mobile Phones , 2015, IEEE Transactions on Visualization and Computer Graphics.

[19]  Salah Sukkarieh,et al.  Visual-Inertial-Aided Navigation for High-Dynamic Motion in Built Environments Without Initial Conditions , 2012, IEEE Transactions on Robotics.

[20]  Reinhard Koch,et al.  Metric 3D Surface Reconstruction from Uncalibrated Image Sequences , 1998, SMILE.

[21]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[22]  Dimitrios G. Kottas,et al.  Towards Consistent Vision-Aided Inertial Navigation , 2012, WAFR.

[23]  Andrew J. Davison,et al.  Live dense reconstruction with a single moving camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Torsten Sattler,et al.  3D Modeling on the Go: Interactive 3D Reconstruction of Large-Scale Scenes on Mobile Devices , 2015, 2015 International Conference on 3D Vision.

[25]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[26]  Maarten Vergauwen,et al.  From image sequences to 3D models , 2001 .

[27]  David W. Murray,et al.  Parallel Tracking and Mapping on a camera phone , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.