Annotations as a Support for Knowledge Generation - Supporting Visual Analytics in the Field of Ophthalmology

While visual analytics (VA) supports the appraisal of large data amounts, annotations support the amendment of additional information to the VA system. Despite the fact that annotations have occasionally been used to facilitate the analysis, a thorough investigation of annotations themselves is challenging. Although they can represent a suitable way to transfer additional information into the visualization system, there is the need to characterize annotations in order to assure an appropriate use. With our paper we provide a characteristic for annotations, revealing and depicting key issues for the use of annotations. By supplementary fitting our characteristic into the knowledge generation model from Sacha et al. (2014), we provide a systematic view on annotations. We show the general applicability of our characteristic of annotations with a visual analytics approach on medical data in the field of ophthalmology. 1 MOTIVATION AND GENERAL APPROACH While the science of visual analytics is well established, the use of annotations in that context is hardly considered. Visual analytics reveals answers hidden in mounts of data and annotations represent the possibility to integrate additional information to that data into the analysis. The use of annotations is upcoming and increasing in the VA community, yet a thorough analysis is challenging. With this paper we show that annotations are beneficial, presupposing a thorough analysis. We unfold different purposes of annotations and different ways, annotations can be gathered, so that they are available for further processing and visualization. As a result we develop a morphological box, portraying the interplay of annotation characteristics. For suitable use in the VA context, we discuss the integration of annotations into the knowledge generation model from ((Sacha et al., 2014)). Additionally we depict obstacles which accompany the use of annotations. This particularly concerns the visualization of annotations with different certainty levels. For evaluation we project our characterization on a VA approach, annotating optical coherence tomography (OCT) image data with the patients supplementary data, giving users the option to enrich, judge, and comment. We experience that the need to (i) annotate the data, (ii) comment findings and insights or (iii) annotate the work of collaborators is generally present during a visual analysis. Surveying literature emphasizes that postulation, as the use of annotations is seen as critical for the visual analytic process (Heer and Shneiderman, 2012), (Zhao et al., 2017). However, there are problems to be solved, both regarding the data, as well as the purpose of annotations. Concerning the former, we observed that the collected data is unstructured, often incomplete, and sometimes vague and dependent on the interpretation of domain experts. Concerning the latter, an annotation may well support the knowledge generation. Yet, used carelessly, annotations may distort the user’s perception or even amend the data with incorrect information leading to insecure visual analytic outcome. In Chapter 2 we provide an annotation characteristics, which we integrate into the knowledge generation model in Chapter 3. The theoretical basis is evaluated in Chapter 4 with a use case in the field of ophthalmology. Chapter 5 rounds out the paper with a conclusion and a view on the future work. 2 CHARACTERIZING ANNOTATIONS The term ”annotation” is frequently used in literature, yet it is challenging to find a definition or explanatory introduction. There is one definition by (Alm et al., 2015), who declare them as objects (e.g. text snippets,