Spatial sketch to compose internet images with RealSense 3D camera

Digital content creation become an indispensable part of our lives. In addition, massive internet images provide us abundant creation material. To construct a computer-aided graphics creation system, interaction and convenience are the critical factors. With somatosensory interactive technology, we can interact with our devices in a more natural and intuitive way, and gain immersive experience at the same time. In this paper, we present a sketch driven image creation system by retrieving and composing internet images automatically. These images are retrieved according simple freehand sketch annotated with text labels. With the help of RealSense depth camera, user only need to make various hand gestures to draw the sketch in spatial and tag with the text labels by speech. Then our system will complete a series of tasks to generate image automatically, thus it greatly reduce the burden of painting and increase the fun and interactive friendliness.

[1]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[2]  Michael F. Cohen,et al.  Simultaneous Matting and Compositing , 2006, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Abdolah Chalechale,et al.  Sketch-based image matching Using Angular partitioning , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[4]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Mike Y. Chen,et al.  RealSense: directional interaction for proximate mobile sharing using built-in orientation sensors , 2013, MM '13.

[6]  Alberto Del Bimbo,et al.  Three-Dimensional Interfaces for Querying by Example in Content-Based Image Retrieval , 2002, IEEE Trans. Vis. Comput. Graph..

[7]  Shih-Fu Chang,et al.  Image Retrieval: Current Techniques, Promising Directions, and Open Issues , 1999, J. Vis. Commun. Image Represent..

[8]  Alberto Del Bimbo,et al.  Visual Image Retrieval by Elastic Matching of User Sketches , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Edward H. Adelson,et al.  A multiresolution spline with application to image mosaics , 1983, TOGS.

[11]  Patrick Olivier,et al.  Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor , 2012, UIST.

[12]  Dragos Datcu,et al.  Free-hands interaction in augmented reality , 2013, SUI '13.

[13]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[14]  David Zeltzer,et al.  A survey of glove-based input , 1994, IEEE Computer Graphics and Applications.

[15]  Christian Bauckhage,et al.  Combining speech and haptics for intuitive and efficient navigation through image databases , 2003, ICMI '03.

[16]  Georgia Albuquerque,et al.  Tangible 3D: hand gesture interaction for immersive 3D modeling , 2005, EGVE'05.

[17]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[18]  W. Marsden.  I and J , 2012 .

[19]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[20]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[21]  Toshikazu Kato,et al.  Query by Visual Example - Content based Image Retrieval , 1992, EDBT.

[22]  Turgali Bauyrzhan Kuanishuly.  Leap Motion as expressive gestural interface , 2018 .

[23]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[24]  Alberto Del Bimbo,et al.  Image Retrieval by Elastic Matching of User Sketches , 1995, ICIAP.

[25]  Marc Alexa,et al.  Sketch-Based Image Retrieval: Benchmark and Bag-of-Features Descriptors , 2011, IEEE Transactions on Visualization and Computer Graphics.