Sketch-based interactive visualization: a survey

Interactive visualization has been an active subfield of scientific visualization for a longtime, in which user interfaces have traditionally followed the WIMP (Window, Icon, Menu, Pointer) paradigm. Though functional and powerful, they can also be cumbersome and daunting to a novice user, and exploring complex data requires considerable expertise and effort. A recent trend is toward more easy-to-use and natural user interfaces, which has led to interfaces like sketch-based one for interactive visualization (SIV). The goal is to allow intuitive interactive visualization by sketching in the visualization process, from data filtering to rendering. In this state-of-the-art report, we give an overview of relevant research works related to sketch-based interface in visualization. We discuss how sketch-based interaction takes effect at specific stages of the visualization pipeline. We present a categorization based on the aim of interaction, of which there are seven primary categories: selection, cutting, segmentation, matching, coloring, augmentation, and illustration. What is more, we present important items related to SIV interface design, including SIV system modes, necessary tools and some fundamental principles. This survey also provides an overview of some specific applications of SIV and a discussion of important challenges and open problems for researchers to tackle in the coming years.Graphical Abstract