Image-based haptic display via a novel pen-shaped haptic device on touch screens

Haptic interaction is a new interactive mode in the interaction technology between human and touch screens. In this work, we present an image-based haptic interaction system on touch screens. A novel haptic pen is designed for haptic display of visual image. It generates force feedback and tactile feedback through electromechanical structure and piezoelectric ceramics respectively. Haptic model of image is built by image processing and haptic rendering. In image processing, the image is decomposed into geometry and textures by local total variation for distinguishing contour shape from surface details. And the contour lines are extracted from the geometry based on the adaptive flow-based difference-of-Gaussians algorithm. Then the height information of image is recovered via shape from shading algorithm and expressed by electromechanical structure. Textures and contour lines are displayed by piezoelectric ceramics. Finally, the haptic perception experiment is conducted to investigate effect of perception with different haptic pens and modes of haptic interaction.

[1]  Wang Dangxiao.  Study on Contour Force Rendering Technology Oriented to Image Perception , 2011 .

[2]  Yi Yang,et al.  Design and control of an integrated haptic interface for touch screen applications , 2013 .

[3]  Yves Meyer,et al.  Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures , 2001 .

[4]  Jose Luis Lisani,et al.  Directional Filters for Cartoon + Texture Image Decomposition , 2016, Image Process. Line.

[5]  Jeong-Mook Lim,et al.  A haptic touchscreen interface for mobile devices , 2013, ICMI '13.

[6]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[7]  Nicolas Roussel,et al.  STIMTAC: a tactile input device with programmable friction , 2011, UIST '11 Adjunct.

[8]  Aiguo Song,et al.  A Novel Haptic Stylus for Mobile Terminal , 2016, EuroHaptics.

[9]  Mandayam A. Srinivasan,et al.  Precise manipulation of GUI on a touch screen with haptic cues , 2009, World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[10]  Sergey Ablameyko,et al.  Image Pixel Force Fields and their Application for Color Map Vectorisation , 2007 .

[11]  Wei Liu,et al.  Experimental Study on the Perception Characteristics of Haptic Texture by Multidimensional Scaling , 2015, IEEE Transactions on Haptics.

[12]  Sergey Ablameyko,et al.  Image Pixel Force Fields and their Application for Color Map Vectorisation , 2007, Ninth International Conference on Document Analysis and Recognition (ICDAR 2007).

[13]  Mubarak Shah,et al.  Shape from shading using linear approximation , 1994, Image Vis. Comput..

[14]  Oliver Deussen,et al.  Depth-aware coherent line drawings , 2015, SIGGRAPH Asia Technical Briefs.

[15]  Lorna M. Brown,et al.  Tactons: Structured Tactile Messages for Non-Visual Information Display , 2004, AUIC.

[16]  Darren Leigh,et al.  Haptic pen: a tactile feedback stylus for touch screens , 2004, UIST '04.

[17]  Hisashi Shimodaira,et al.  A shape-from-shading method of polyhedral objects using prior information , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Alexei Sourin,et al.  Image-Driven Haptic Rendering , 2014, Trans. Comput. Sci..

[19]  Aiguo Song,et al.  A novel miniature multi-mode haptic pen for image interaction on mobile terminal , 2015, 2015 IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE).

[20]  Masahiko Inami,et al.  ImpAct: Immersive haptic stylus to enable direct touch and manipulation for surface computing , 2010, CIE.

[21]  Dario Farina,et al.  A Novel Method to Generate Amplitude-Frequency Modulated Vibrotactile Stimulation , 2016, IEEE Transactions on Haptics.

[22]  R. Klatzky,et al.  Feeling textures through a probe: Effects of probe and surface geometry and exploratory factors , 2003, Perception & psychophysics.

[23]  A. Cockburn.  Proceedings of the fifth conference on Australasian user interface - Volume 28 , 2004 .

[24]  Jun-Young Lee,et al.  Ubi-Pen: A Haptic Interface with Texture and Vibrotactile Display , 2009, IEEE Computer Graphics and Applications.

[25]  Ali Israr,et al.  TeslaTouch: electrovibration for touch surfaces , 2010, UIST.

[26]  Peng Deng,et al.  The Roughness Display with Pen-like Tactile Device for Touchscreen Device , 2016, EuroHaptics.

[27]  Miltiadis K. Hatalis,et al.  Tactile displays: Overview and recent advances , 2008, Displays.

[28]  Seungyong Lee,et al.  Coherent line drawing , 2007, NPAR '07.