MindCamera: Interactive Sketch-Based Image Retrieval and Synthesis

Composing a realistic picture according to the mind is tough work for most people. It is not only a complex operation but also a creation process from nonexistence to existence. Therefore, the core of this problem is to provide rich existing materials for stitching. We present an interactive sketch-based image retrieval and synthesis system, MindCamera. Compared with existing methods, it can use images of daily scenes as the dataset and proposes a sketch-based image of a scene retrieval model. Furthermore, MindCamera can blend the target object in the gradient domain to avoid the visible seam, and it introduces alpha matting to realize real-time foreground object extraction and composition. Experiments verify that our retrieval model has higher precision and provides more reasonable and richer materials for users. The practical usage demonstrates that MindCamera allows the interactive creation of complex images, and its final compositing results are natural and realistic.

[1]  Rui Hu,et al.  Gradient field descriptor for sketch based retrieval and localization , 2010, 2010 IEEE International Conference on Image Processing.

[2]  Honggang Zhang,et al.  Variational Bayesian Matrix Factorization for Bounded Support Data , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Manuel Menezes de Oliveira Neto,et al.  Shared Sampling for Real‐Time Alpha Matting , 2010, Comput. Graph. Forum.

[4]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[5]  Jun Guo,et al.  Feature selection for neutral vector in EEG signal classification , 2016, Neurocomputing.

[6]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[9]  Changhu Wang,et al.  Sketch2Cartoon: composing cartoon images by sketching , 2011, MM '11.

[10]  Marc Alexa,et al.  Sketch-Based Image Retrieval: Benchmark and Bag-of-Features Descriptors , 2011, IEEE Transactions on Visualization and Computer Graphics.

[11]  Shi-Min Hu,et al.  Sketch2Photo: internet image montage , 2009, ACM Trans. Graph..

[12]  Jun Guo,et al.  Cross-modal subspace learning for fine-grained sketch-based image retrieval , 2017, Neurocomputing.

[13]  Wojciech Matusik,et al.  CG2Real: Improving the Realism of Computer Generated Images Using a Large Collection of Photographs , 2011, IEEE Transactions on Visualization and Computer Graphics.

[14]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[15]  Liqing Zhang,et al.  Sketch-based Image Retrieval via Shape Words , 2015, ICMR.

[16]  Udo Kelter,et al.  Shape-based object retrieval by contour segment matching , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[17]  Changhu Wang,et al.  Indexing billions of images for sketch-based retrieval , 2013, ACM Multimedia.

[18]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[20]  Marc Alexa,et al.  Photosketcher: Interactive Sketch-Based Image Synthesis , 2011, IEEE Computer Graphics and Applications.

[21]  John P. Collomosse,et al.  Scalable Sketch-Based Image Retrieval Using Color Gradient Features , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[22]  Liqing Zhang,et al.  Edgel index for large-scale sketch-based image search , 2011, CVPR 2011.