Perception-Lossless Codec of Haptic Data with Low Delay

In multimedia services, the introduction of haptic signals provides a more immersive user experience besides of conventional audio-visual perceptions. To support synchronous streaming and display of these information, it is imperative to efficiently compress and store the haptic signals, which promotes the development and optimization of haptic codecs. In this paper, we propose an end-to-end haptic codec for high-efficiency, low-delay and perception-lossless compression of kinesthetic signal, one of two major components of haptic signals. The proposed encoder consists of amplifier, DCT, quantizer, run-length encoder and entropy encoder, while the decoder includes all counterpart modules of the encoder. In particular, all parameters of these modules are deliberately calibrated aimed at a high compression efficiency of kinesthetic information. We allow a maximal DCT length of 8 samples, in order to guarantee a maximal encoding delay of 7ms for a popular haptic simulator of 1000Hz. Incorporating the model of perception deadband, the proposed codec is capable of realizing perception-lossless kinesthetic bitsteam. Finally, we examine the proposed codec on the standard database of IEEE P1918.1.1 Haptic Codecs Task Group. Comprehensive experiments reveal that our codec outperforms its rivals with 50% bit rate reduction, improved perception quality and a negligible encoder delay.

[1]  Sandra Hirche,et al.  A novel, psychophysically motivated transmission approach for haptic data streams in telepresence and teleaction systems , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[2]  Eckehard G. Steinbach,et al.  HSSIM: An Objective Haptic Quality Assessment Measure for Force-Feedback Signals , 2018, 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX).

[3]  Shahram Payandeh,et al.  Signal processing techniques for haptic data compression in teleoperation systems , 2012, 2012 IEEE Haptics Symposium (HAPTICS).

[4]  Eray A. Baran,et al.  Comparative Analysis of a Selected DCT-Based Compression Scheme for Haptic Data Transmission , 2016, IEEE Transactions on Industrial Informatics.

[5]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Subhasis Chaudhuri,et al.  Deadzone analysis of 2D kinesthetic perception , 2014, 2014 IEEE Haptics Symposium (HAPTICS).

[7]  Katherine J. Kuchenbecker,et al.  Improving contact realism through event-based haptic feedback , 2006, IEEE Transactions on Visualization and Computer Graphics.

[8]  Mark W. Spong,et al.  Bilateral control of teleoperators with time delay , 1988, Proceedings of the 1988 IEEE International Conference on Systems, Man, and Cybernetics.

[9]  Abdulmotaleb El-Saddik,et al.  A candidate hardware and software reference setup for kinesthetic codec standardization , 2017, 2017 IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE).

[10]  Kouhei Ohnishi,et al.  Lossy data compression using FDCT for haptic communication , 2010, 2010 11th IEEE International Workshop on Advanced Motion Control (AMC).

[11]  Gregory K. Wallace,et al.  The JPEG still picture compression standard , 1991, CACM.

[12]  Sandra Hirche,et al.  Perception-Based Data Reduction and Transmission of Haptic Data in Telepresence and Teleaction Systems , 2008, IEEE Transactions on Signal Processing.

[13]  Eckehard G. Steinbach,et al.  Low-delay compression of polygon mesh deformation data for remote haptic interaction with simulated deformable objects , 2016, 2016 IEEE Haptics Symposium (HAPTICS).

[14]  Eckehard G. Steinbach,et al.  Disposal of explosive ordnances by use of a bimanual haptic telepresence system , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[15]  Christoph W. Borst,et al.  Predictive coding for efficient host-device communication in a pneumatic force-feedback display , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[16]  Sandra Hirche,et al.  Haptic Data Compression and Communication , 2011, IEEE Signal Processing Magazine.

[17]  Sandra Hirche,et al.  Human-Oriented Control for Haptic Teleoperation , 2012, Proceedings of the IEEE.

[18]  Siwei Ma,et al.  Framework of AVS2-video coding , 2013, 2013 IEEE International Conference on Image Processing.

[19]  Eckehard G. Steinbach,et al.  Model-Mediated Teleoperation: Toward Stable and Transparent Teleoperation Systems , 2016, IEEE Access.

[20]  S. Gelfan,et al.  Muscle sense in man. , 1967, Experimental neurology.

[21]  Antonio Ortega,et al.  Rate-distortion methods for image and video compression , 1998, IEEE Signal Process. Mag..

[22]  Orestis Georgiou,et al.  Using Ultrasonic Mid-air Haptic Patterns in Multi-Modal User Experiences , 2019, 2019 IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE).

[23]  R. Klatzky,et al.  Haptic perception: A tutorial , 2009, Attention, perception & psychophysics.

[24]  Alexei Sourin,et al.  Towards Asynchronous Video-Haptic Interaction in Cyberspace , 2018, 2018 International Conference on Cyberworlds (CW).

[25]  Hugo Costelha,et al.  Subjective Evaluation of Haptic Feedback Technologies for Interactive Multimedia , 2019, IEEE EUROCON 2019 -18th International Conference on Smart Technologies.

[26]  K. Ohnishi,et al.  Haptic data compression/decompression using DCT for motion copy system , 2009, 2009 IEEE International Conference on Mechatronics.

[27]  Gregory D. Abowd,et al.  NotifiVR: Exploring Interruptions and Notifications in Virtual Reality , 2018, IEEE Transactions on Visualization and Computer Graphics.

[28]  Liu Qian,et al.  Quality-of-Experience in Human-in-the-Loop Haptic Communications , 2019 .

[29]  Saeid Nahavandi,et al.  Applying Inverse Just-Noticeable-Differences of Velocity to Position Data for Haptic Data Reduction , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[30]  Gabriel-Miro Muntean,et al.  Is Multimedia Multisensorial? - A Review of Mulsemedia Systems , 2018, ACM Comput. Surv..

[31]  R. Meena,et al.  Simulation in neurosurgery: Past, present, and future. , 2016, Neurology India.

[32]  Shaohua Zheng,et al.  Visual Attention and Haptic Control: A Cross-Study , 2019, 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM).

[33]  Thomas Sikora,et al.  Trends and Perspectives in Image and Video Coding , 2005, Proceedings of the IEEE.

[34]  Kouhei Ohnishi,et al.  A quantization method for haptic data lossy compression , 2015, 2015 IEEE International Conference on Mechatronics (ICM).

[35]  Sandra Hirche,et al.  Transparency of haptic telepresence systems with constant time delay , 2005 .

[36]  Xun Liu,et al.  Haptic Codecs for the Tactile Internet , 2019, Proceedings of the IEEE.

[37]  Claudio Pacchierotti,et al.  Rendering of Pressure and Textures Using Wearable Haptics in Immersive VR Environments , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[38]  Eckehard G. Steinbach,et al.  Haptic data reduction for time-delayed teleoperation using the time domain passivity approach , 2015, 2015 IEEE World Haptics Conference (WHC).

[39]  Kyoung Shin Park,et al.  Effects of network characteristics on human performance in a collaborative virtual environment , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[40]  Antonio Ortega,et al.  A comparison of different haptic compression techniques , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[41]  Jordi Artigas,et al.  A passive bilateral control scheme for a teleoperator with time-varying communication delay , 2010 .