Depth-aware image vectorization and editing

Image vectorization is one of the primary means of creating vector graphics. The quality of a vectorized image depends crucially on extracting accurate features from input raster images. However, correct object edges can be difficult to detect when color gradients are weak. We present an image vectorization technique that operates on a color image augmented with a depth map and uses both color and depth edges to define vectorized paths. We output a vectorized result as a diffusion curve image. The information extracted from the depth map allows us more flexibility in the manipulation of the diffusion curves, in particular permitting high-level object segmentation. Our experimental results demonstrate that this method achieves high reconstruction quality and provides greater control in the organization and editing of vectorized images than existing work based on diffusion curves.

[1]  Pascal Barla,et al.  Structure-preserving manipulation of photographs , 2007, NPAR '07.

[2]  Hao Huang,et al.  Computational Multi-View Imaging with Kinect , 2014, IEEE Transactions on Broadcasting.

[3]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[4]  Xuelong Li,et al.  Depth-Aware Image Seam Carving , 2013, IEEE Transactions on Cybernetics.

[5]  Daniel Thalmann,et al.  Robust 3D Hand Pose Estimation in Single Depth Images: From Single-View CNN to Multi-View CNNs , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Stefan Jeschke,et al.  Estimating Color and Texture Parameters for Vector Graphics , 2011, Comput. Graph. Forum.

[7]  Jianxiong Xiao,et al.  Sliding Shapes for 3D Object Detection in Depth Images , 2014, ECCV.

[8]  Ralph R. Martin,et al.  Automatic and topology-preserving gradient mesh generation for image vectorization , 2009, ACM Trans. Graph..

[9]  Pascal Barla,et al.  Diffusion curves: a vector representation for smooth-shaded images , 2008, SIGGRAPH 2008.

[10]  John Snyder,et al.  Freeform vector graphics with controlled thin-plate splines , 2011, ACM Trans. Graph..

[11]  Jie Zhu,et al.  Video Vectorization via Tetrahedral Remeshing , 2017, IEEE Transactions on Image Processing.

[12]  Ting-Chun Wang,et al.  Depth from Semi-Calibrated Stereo and Defocus , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Derek Nowrouzezahrai,et al.  Hierarchical diffusion curves for accurate automatic image vectorization , 2014, ACM Trans. Graph..

[14]  David A. Forsyth,et al.  A Subdivision-Based Representation for Vector Image Editing , 2012, IEEE Transactions on Visualization and Computer Graphics.

[15]  Xi Wang,et al.  High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth , 2014, GCPR.

[16]  Yizhou Yu,et al.  Patch-based image vectorization with automatic curvilinear feature alignment , 2009, ACM Trans. Graph..

[17]  Yizhou Yu,et al.  Patch-based image vectorization with automatic curvilinear feature alignment , 2009, SIGGRAPH 2009.

[18]  Cyril Concolato,et al.  Biharmonic diffusion curve images from boundary elements , 2013, ACM Trans. Graph..

[19]  Stefan Jeschke,et al.  Generalized Diffusion Curves: An Improved Vector Representation for Smooth‐Shaded Images , 2016, Comput. Graph. Forum.

[20]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[21]  Tony Lindeberg,et al.  Edge Detection and Ridge Detection with Automatic Scale Selection , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Yao Wang,et al.  Color-Guided Depth Recovery From RGB-D Data Using an Adaptive Autoregressive Model , 2014, IEEE Transactions on Image Processing.

[23]  Aimin Hao,et al.  Poisson Vector Graphics (PVG) , 2020, IEEE Transactions on Visualization and Computer Graphics.

[24]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[25]  Frédo Durand,et al.  Inverse Diffusion Curves Using Shape Optimization , 2016, IEEE Transactions on Visualization and Computer Graphics.

[26]  Jitendra Malik,et al.  Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Jianxiong Xiao,et al.  Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Pascal Barla,et al.  A vectorial solver for free-form vector gradients , 2012, ACM Trans. Graph..

[29]  Deqing Sun,et al.  Layered RGBD scene flow estimation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  William A. Barrett,et al.  Object-based vectorization for interactive image editing , 2006, The Visual Computer.

[31]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Peter Wonka,et al.  A GPU Laplacian solver for diffusion curves and Poisson image editing , 2009, ACM Transactions on Graphics.

[33]  Timothy Sun,et al.  Fast multipole representation of diffusion curves and points , 2014, ACM Trans. Graph..

[34]  Ralph R. Martin,et al.  Automatic and topology-preserving gradient mesh generation for image vectorization , 2009, SIGGRAPH 2009.

[35]  Philip J. Schneider,et al.  An algorithm for automatically fitting digitized curves , 1990 .

[36]  William Bialek,et al.  Statistics of Natural Images: Scaling in the Woods , 1993, NIPS.