Contour-driven Sumi-e rendering of real photos

We propose a sketch-based system for rendering oriental brush strokes on complex shapes. While previous research has focused on methods for converting user-specified trajectories into oriental ink painting (Sumi-e) strokes, we propose an approach that takes as input the contours of complex shapes, estimates automatically the sizes of the brush footprints, their orientations, and the brush trajectory. Then, it renders them into oriental ink paintings. The approach performs in three steps; first complex shapes are decomposed into elementary shapes that can be rendered with a single brush stroke. Second, the brush trajectory estimation is formulated as the minimization of an energy function that measures the quality of the trajectory. Finally, the estimated trajectories are rendered into brush strokes by mapping on them footprint textures scanned from real images. We demonstrate that the proposed contour-driven approach is particularly suitable for converting real images into Sumi-e paintings with minimum interaction. To this end we develop a system where images are either interactively or automatically segmented into elementary shapes and converted into Sumi-e paintings. For the automatic segmentation, we observe that the strokes in Sumi-e paintings are nearly parallel to the edge tangent flow (ETF). We propose a segmentation procedure that is based on clustering the curves obtained by integrating the ETF of real images. Experiments on complex shapes show that the proposed contour-based approach produces a large variety of complex strokes compared to trajectory-based approaches.