Haptic Data Reduction and Protection in Multimodal Virtual Environments

The emerging field of haptics, which enables the sensing and manipulation of virtual environments through touch, is recognized as an important element in the area of human computer interaction. Many of its applications, including medical training, rehabilitation, security, entertainment, etc. involve the transmission, storage, protection, and analysis of haptic information. However, the multi-dimensional and time-varying nature of haptic data along with the fact that high sampling frequencies are required when generating haptic signals, motivate the research of haptic data reduction. This thesis discusses two important and related topics: data reduction and protection in multimodal visual-haptic virtual environments. The focus of the first topic is on the design, evaluation, and analysis of haptic data reduction techniques in order to: (1) Improve packet transmission in telehaptic systems, and (2) reduce the dimensionality of the inherently large haptic datasets. In the former case, robust perception-based data reduction methods in 3and 6-degree-of-freedom telehaptic systems are derived. The proposed techniques are evaluated in different experimental settings, including a haptic-enabled telesurgery simulation, and demonstrate a significant reduction in haptic data traffic while maintaining a high-quality telehaptic experience. In the latter case, data reduction in haptic information analysis problems is investigated. Specifically, data reduction methods are presented within the context of haptic data mining and knowledge discovery to help facilitate the analysis of the inherently high-dimensional haptic datasets. Experiments using datasets of haptic-based handwritten signatures demonstrate the accuracy of the techniques in reducing the high dimensionality of haptic feature spaces. The protection of haptic information through digital watermarking is also investigated in this thesis due to its conceptual similarity to haptic data reduction and compression. A foundational study is conducted to examine the role of multisensory feedback in the perception of a watermark embedded in a haptic-enabled 3D virtual surface. In particular, the analysis aims to answer numerous important questions in this brand-new research area, including: Do water-

[1]  Susan J. Lederman,et al.  Roughness and spatial density judgments on visual and haptic textures using virtual reality , 2004 .

[2]  Janusz Zalewski,et al.  Rough sets: Theoretical aspects of reasoning about data , 1996 .

[3]  Mauro Barni,et al.  Towards 3D watermarking technology , 2003, The IEEE Region 8 EUROCON 2003. Computer as a Tool..

[4]  David Swapp,et al.  Interaction with co-located haptic feedback in virtual reality , 2006, Virtual Reality.

[5]  Sandra Hirche,et al.  A novel, psychophysically motivated transmission approach for haptic data streams in telepresence and teleaction systems , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[6]  Mark Paterson,et al.  The Senses of Touch: Haptics, Affects and Technologies , 2007 .

[7]  Kari Karhunen,et al.  Über lineare Methoden in der Wahrscheinlichkeitsrechnung , 1947 .

[8]  Nizar Sakr,et al.  Network traffic reduction in six degree-of-freedom haptic, telementoring systems , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[9]  Domenico Prattichizzo,et al.  Perceptibility of haptic digital watermarking of virtual textures , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[10]  Janet M. Weisenberger,et al.  Multisensory roughness perception of virtual surfaces: effects of correlated cues , 2004, 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2004. HAPTICS '04. Proceedings..

[11]  Thomas Kerwin,et al.  The role of multisensory feedback in haptic surface perception , 2003, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings..

[12]  Huan Liu,et al.  Feature Selection: An Ever Evolving Frontier in Data Mining , 2010, FSDM.

[13]  Cândida Ferreira,et al.  Gene Expression Programming: Mathematical Modeling by an Artificial Intelligence , 2014, Studies in Computational Intelligence.

[14]  Jean-Jacques E. Slotine,et al.  Stable Adaptive Teleoperation , 1990, 1990 American Control Conference.

[15]  Sandra Hirche,et al.  Perception-Based Data Reduction and Transmission of Haptic Data in Telepresence and Teleaction Systems , 2008, IEEE Transactions on Signal Processing.

[16]  Sandra Hirche,et al.  Perception-Based Data Reduction for Haptic Force-Feedback Signals Using Velocity-Adaptive Deadbands , 2010, PRESENCE: Teleoperators and Virtual Environments.

[17]  Andreas Galka,et al.  Topics in Nonlinear Time Series Analysis, with Implications for Eeg Analysis , 2000 .

[18]  Nizar Sakr,et al.  Multimodal Vision–Haptic Perception of Digital Watermarks Embedded in 3-D Meshes , 2010, IEEE Transactions on Instrumentation and Measurement.

[19]  Sandra Hirche,et al.  Lossy data reduction methods for haptic telepresence systems , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[20]  Sandra Hirche,et al.  Transparent Data Reduction in Networked Telepresence and Teleaction Systems. Part II: Time-Delayed Communication , 2007, PRESENCE: Teleoperators and Virtual Environments.

[21]  M. A. Srinivassan.  The impact of visual information on the haptic perception of stiffness in virtual environments , 1996 .

[22]  Aleksander Øhrn,et al.  Discernibility and Rough Sets in Medicine: Tools and Applications , 2000 .

[23]  Mark A. Hall,et al.  Correlation-based Feature Selection for Machine Learning , 2003 .

[24]  A. El Saddik,et al.  Behavioral Features for Different Haptic-based Biometric Tasks , 2007, 2007 IEEE International Workshop on Haptic, Audio and Visual Environments and Games.

[25]  Antonio Ortega,et al.  A comparison of different haptic compression techniques , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[26]  Eckehard G. Steinbach,et al.  A Psychophysically Motivated Compression Approach for 3D Haptic Data , 2006, 2006 14th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[27]  Julio J. Valdés,et al.  Virtual Reality Representation of Information Systems and Decision Rules: An Exploratory Technique for Understanding Data and Knowledge Structure , 2003, RSFDGrC.

[28]  Sandra Hirche,et al.  Transparent Data Reduction in Networked Telepresence and Teleaction Systems. Part I: Communication without Time Delay , 2007, PRESENCE: Teleoperators and Virtual Environments.

[29]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[30]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[31]  J. A. Rod Blais,et al.  Least Squares for Practitioners , 2010 .

[32]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[33]  Nizar Sakr,et al.  Robust perception-based data reduction and transmission in telehaptic systems , 2009, World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[34]  Sandra Hirche,et al.  TOWARDS DEADBAND CONTROL IN NETWORKED TELEOPERATION SYSTEMS , 2005 .

[35]  David J. Hand,et al.  Intelligent Data Analysis: An Introduction , 2005 .

[36]  S. Lederman,et al.  Texture perception: studies of intersensory organization using a discrepancy paradigm, and visual versus tactual psychophysics. , 1981, Journal of experimental psychology. Human perception and performance.

[37]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[38]  Hideko F. Norman,et al.  The visual and haptic perception of natural object shape , 2010 .

[39]  Eckehard G. Steinbach,et al.  Deadband-based offline-coding of haptic media , 2008, ACM Multimedia.

[40]  Bartlomiej Stanczyk,et al.  Advanced Telerobotics: Dual-Handed and Mobile Remote Manipulation , 2007, Advances in Telerobotics.

[41]  Benoit M. Macq,et al.  Shape quality measurement for 3D watermarking schemes , 2006, Electronic Imaging.

[42]  Domenico Prattichizzo,et al.  Is haptic watermarking worth it? , 2006, Electronic Imaging.

[43]  Abdulmotaleb El-Saddik,et al.  Visualizing Human Behavioral Features based on Signature Haptic Data , 2008, 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[44]  Martin Buss,et al.  Passive Haptic Data-Compression Methods With Perceptual Coding for Bilateral Presence Systems , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[45]  James J. Kuffner,et al.  Effective sampling and distance metrics for 3D rigid body path planning , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[46]  Elizae Qaiser,et al.  Quantization of Real-Valued Attributes for Data Mining , 2001 .

[47]  C. Spence,et al.  What role does multisensory integration play in the visuotactile perception of texture? , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[48]  M. C. Cavusoglu,et al.  In Touch with Robotics: Neurosurgery for the Future , 2005, Neurosurgery.

[49]  Jilin Zhou,et al.  Haptic-Enabled Telementoring Surgery Simulation , 2008, IEEE MultiMedia.

[50]  Daniel A. Keim,et al.  Information Visualization and Visual Data Mining , 2002, IEEE Trans. Vis. Comput. Graph..

[51]  Jilin Zhou,et al.  Uniform Hardness Perception in 6-DOF Haptic Rendering , 2011, IEEE Transactions on Instrumentation and Measurement.

[52]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[53]  Heidrun Schumann,et al.  A Flexible Approach for Visual Data Mining , 2002, IEEE Trans. Vis. Comput. Graph..

[54]  Xiaojun Shen,et al.  Design and Implementation of Haptic Tele-mentoring over the Internet , 2007 .

[55]  Nizar Sakr,et al.  Prediction-Based Haptic Data Reduction and Transmission in Telementoring Systems , 2009, IEEE Transactions on Instrumentation and Measurement.

[56]  S. Lederman,et al.  Perception of texture by vision and touch: multidimensionality and intersensory integration. , 1986, Journal of experimental psychology. Human perception and performance.

[57]  Eckehard G. Steinbach,et al.  Disposal of explosive ordnances by use of a bimanual haptic telepresence system , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[58]  Ingemar J. Cox,et al.  Digital Watermarking and Steganography , 2014 .

[59]  James Moyne,et al.  Using deadbands to reduce communication in networked control systems , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[60]  Christoph W. Borst,et al.  Predictive coding for efficient host-device communication in a pneumatic force-feedback display , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[61]  P. Hinterseer,et al.  Model based data compression for 3D virtual haptic teleinteraction , 2006, 2006 Digest of Technical Papers International Conference on Consumer Electronics.

[62]  Eckehard G. Steinbach,et al.  On the Impact of Haptic Data Reduction and Feedback Modality on Quality and Task Performance in a Telepresence and Teleaction System , 2010, EuroHaptics.

[63]  N.D. Georganas,et al.  Prediction-based Haptic Data Reduction and Compression in Tele-Mentoring Systems , 2008, 2008 IEEE Instrumentation and Measurement Technology Conference.

[64]  Abdulmotaleb El-Saddik,et al.  A Novel Biometric System for Identification and Verification of Haptic Users , 2007, IEEE Transactions on Instrumentation and Measurement.

[65]  Staal A. Vinterbo,et al.  Minimal approximate hitting sets and rule templates , 2000, Int. J. Approx. Reason..

[66]  Touradj Ebrahimi,et al.  A Multi-Scale Roughness Metric for 3D Watermarking Quality Assessment , 2005 .

[67]  Ming C. Lin,et al.  High Fidelity Haptic Rendering , 2006, High Fidelity Haptic Rendering.

[68]  Anatole Lécuyer,et al.  The Influence of Spatial Delocation on Perceptual Integration of Vision and Touch , 2006, PRESENCE: Teleoperators and Virtual Environments.

[69]  Domenico Prattichizzo,et al.  Perceptual Issues in Haptic Digital Watermarking , 2007, IEEE MultiMedia.

[70]  Jakub Wroblewski,et al.  Ensembles of Classifiers Based on Approximate Reducts , 2001, Fundam. Informaticae.

[71]  Makoto Sato,et al.  Effect of Coupling Haptics and Stereopsis on Depth Perception in Virtual Environment , 2000 .

[72]  Abdulmotaleb El-Saddik,et al.  Haptic Virtual Rehabilitation Exercises for Poststroke Diagnosis , 2008, IEEE Transactions on Instrumentation and Measurement.

[73]  Grigore C. Burdea,et al.  Force and Touch Feedback for Virtual Reality , 1996 .

[74]  P. Hinterseer,et al.  LIMITATIONS OF HUMAN 3 D-FORCE DISCRIMINATION , 2006 .

[75]  A. El Saddik,et al.  Characterizing biometric behavior through haptics and Virtual Reality , 2008, 2008 42nd Annual IEEE International Carnahan Conference on Security Technology.

[76]  Grigore C. Burdea,et al.  The Rutgers Master II-new design force-feedback glove , 2002 .

[77]  Julio J. Valdés,et al.  Virtual Reality Spaces for Visual Data Mining with Multiobjective Evolutionary Optimization: Implicit and Explicit Function Representations Mixing Unsupervised and Supervised Properties , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[78]  Mehrdad Hosseini Zadeh,et al.  Perception-based lossy haptic compression considerations for velocity-based interactions , 2007, Multimedia Systems.

[79]  Sandra Hirche,et al.  Perceptual coding of haptic data in time-delayed teleoperation , 2009, World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[80]  Piotr Jedrzejowicz,et al.  Data Reduction Algorithm for Machine Learning and Data Mining , 2008, IEA/AIE.

[81]  Mark W. Spong,et al.  Bilateral control of teleoperators with time delay , 1989 .

[82]  Eckehard G. Steinbach,et al.  Perception-Based Compression of Haptic Data Streams Using Kalman Filters , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[83]  Vincent Hayward,et al.  Discrete-time adaptive windowing for velocity estimation , 2000, IEEE Trans. Control. Syst. Technol..

[84]  Mark Wright,et al.  The Effect of Haptic Feedback and Stereo Graphics in a 3D Target Acquisition Task , 2002 .

[85]  David S. Johnson,et al.  Approximation algorithms for combinatorial problems , 1973, STOC.

[86]  Domenico Prattichizzo,et al.  Perceptibility of digital watermarking in haptically enabled 3D meshes , 2006 .

[87]  Nizar Sakr,et al.  Perceptibility of digital watermarks embedded in 3D meshes using unimodal and bimodal hapto-visual interaction , 2009, 2009 IEEE Instrumentation and Measurement Technology Conference.

[88]  Abdulmotaleb El-Saddik,et al.  Experiments in haptic-based authentication of humans , 2008, Multimedia Tools and Applications.

[89]  Akira Ito,et al.  Role of vision on haptic length perception , 2006, Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC'07).

[90]  I. Borg.  Multidimensional similarity structure analysis , 1987 .

[91]  John W. Sammon,et al.  A Nonlinear Mapping for Data Structure Analysis , 1969, IEEE Transactions on Computers.

[92]  Sandra Hirche,et al.  NETWORK TRAFFIC REDUCTION IN HAPTIC TELEPRESENCE SYSTEMS BY DEADBAND CONTROL , 2005 .

[93]  Eibe Frank,et al.  Large-scale attribute selection using wrappers , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[94]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[95]  Sunil R. Das,et al.  An adaptive compressed MPEG-2 video watermarking scheme , 2005, IEEE Transactions on Instrumentation and Measurement.

[96]  M. Heller.  Visual and tactual texture perception: Intersensory cooperation , 1982, Perception & psychophysics.

[97]  Lawrence R. Rabiner,et al.  Applications of a nonlinear smoothing algorithm to speech processing , 1975 .