Camera-Based Whiteboard Reading for Understanding Mind Maps

Mind maps, i.e. the spatial organization of ideas and concepts around a central topic and the visualization of their relations, represent a very powerful and thus popular means to support creative thinking and problem solving processes. Typically created on traditional whiteboards, they represent an important technique for collaborative brainstorming sessions. We describe a camera-based system to analyze hand-drawn mind maps written on a whiteboard. The goal of the presented system is to produce digital representations of such mind maps, which would enable digital asset management, i.e. storage and retrieval of manually created documents. Our system is based on image acquisition by means of a camera, followed by the segmentation of the particular whiteboard image focusing on the extraction of written context, i.e. the ideas captured by the mind map. The spatial arrangement of these ideas is recovered using layout analysis based on unsupervised clustering, which results in graph representations of mind maps. Finally, handwriting recognition derives textual transcripts of the ideas captured by the mind map. We demonstrate the capabilities of our mind map reading system by means of an experimental evaluation, where we analyze images of mind maps that have been drawn on whiteboards, without any further constraints other than the underlying topic. In addition to the promising recognition results, we also discuss training strategies, which effectively allow for system bootstrapping using out-of-domain sample data. The latter is important when addressing creative thinking processes where domain-related training data are difficult to obtain as they focus on novelty by definition.