Depth analysis of kinect v2 sensor in different mediums

From the last few years, RGB-D cameras are widely used by researchers in various fields. Their reasonable cost and the ability to estimate distances at a high frame rate have made these sensors recommendable for applications in gaming accessories, robotics, computer vision, etc. In addition to color, these sensors also provide depth information. Aspects like the stability, accuracy and reliability of depth-sensing cameras like Kinect v2 must also be considered before using the device for applications like that of 3D space modelling. In this paper, an analysis of the error in the depth measurement as well as calculation of Depth Entropy given by Kinect v2 sensor in different mediums viz. air, glass and water has been done. We have validated our findings using the theories of optics. The findings from error analysis are used to make an error compensation model which can correct depth at each pixel of the image. The error analysis and error compensation model proposed herewith will help in improving the accuracy of present and future depth sensing devices.

[1]  BINHUI LIU,et al.  Edge-Guided Depth Image Super-Resolution Based on KSVD , 2020, IEEE Access.

[2]  Matteo Munaro,et al.  Performance evaluation of the 1st and 2nd generation Kinect for multimedia applications , 2015, 2015 IEEE International Conference on Multimedia and Expo (ICME).

[3]  Tomás Pajdla,et al.  3D with Kinect , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[4]  Shang-Hong Lai,et al.  Depth image super-resolution via multi-frame registration and deep learning , 2016, 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).

[5]  K. S. Shifrin.  Physical optics of ocean water , 1988 .

[6]  Lin Yang,et al.  Evaluating and Improving the Depth Accuracy of Kinect for Windows v2 , 2015, IEEE Sensors Journal.

[7]  Marcus A. Magnor,et al.  Markerless Motion Capture using multiple Color-Depth Sensors , 2011, VMV.

[8]  Andreas Kolb,et al.  Kinect range sensing: Structured-light versus Time-of-Flight Kinect , 2015, Comput. Vis. Image Underst..

[9]  Xueying Qin,et al.  Deep Depth Super-Resolution: Learning Depth Super-Resolution Using Deep Convolutional Neural Network , 2016, ACCV.

[10]  Mohamed Rehan,et al.  A Kinect-Based 3D Object Detection and Recognition System with Enhanced Depth Estimation Algorithm , 2018, 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON).

[11]  Rebecca Fahrig,et al.  Toward analyzing mutual interference on infrared-enabled depth cameras , 2019, Comput. Vis. Image Underst..

[12]  Henry Fuchs,et al.  Encumbrance-free telepresence system with real-time 3D capture and display using commodity depth cameras , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[13]  H. Macher,et al.  FIRST EXPERIENCES WITH KINECT V2 SENSOR FOR CLOSE RANGE 3D MODELLING , 2015 .

[14]  Fan Zhong,et al.  Estimation of Kinect depth confidence through self-training , 2014, The Visual Computer.

[15]  A. Gaber,et al.  Automated grading of facial paralysis using the Kinect v2: A proof of concept study , 2015, 2015 International Conference on Virtual Rehabilitation (ICVR).

[16]  I-Jui Lee,et al.  Kinect-for-windows with augmented reality in an interactive roleplay system for children with an autism spectrum disorder , 2020, Interact. Learn. Environ..

[17]  Ruigang Yang,et al.  Channel Attention Based Iterative Residual Learning for Depth Map Super-Resolution , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Dmitriy Vatolin,et al.  Temporal filtering for depth maps generated by Kinect depth camera , 2011, 2011 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[19]  Vincent Lepetit,et al.  SharpNet: Fast and Accurate Recovery of Occluding Contours in Monocular Depth Estimation , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[20]  Rogério Schmidt Feris,et al.  Edge-Guided Single Depth Image Super Resolution , 2016, IEEE Transactions on Image Processing.

[21]  Elise Lachat,et al.  Assessment and Calibration of a RGB-D Camera (Kinect v2 Sensor) Towards a Potential Use for Close-Range 3D Modeling , 2015, Remote. Sens..

[22]  Kourosh Khoshelham,et al.  Accuracy analysis of kinect depth data , 2012 .

[23]  J. Curcio,et al.  Near infrared absorption spectrum of liquid water , 1951 .

[24]  Hongbin Zha,et al.  Similarity-Aware Patchwork Assembly for Depth Image Super-resolution , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Mario Fritz,et al.  Improving the Kinect by Cross-Modal Stereo , 2011, BMVC.

[26]  Michael Riis Andersen,et al.  Kinect Depth Sensor Evaluation for Computer Vision Applications , 2012 .

[27]  Achim J. Lilienthal,et al.  Comparative Evaluation of Range Sensor Accuracy in Indoor Environments , 2011, ECMR.

[28]  Reza Lotfian,et al.  Immersive multiplayer tennis with microsoft kinect and body sensor networks , 2012, ACM Multimedia.

[29]  Livio Pinto,et al.  Calibration of Kinect for Xbox One and Comparison between the Two Generations of Microsoft Sensors , 2015, Sensors.

[30]  Chunyu Li,et al.  Learning skeleton information for human action analysis using Kinect , 2020, Signal Process. Image Commun..