Structuralizing Freeform Notes by Implicit Sketch Understanding

People are accustomed to capture important events and ideas by sketching notes on papers. Pen and paper offer people great freedom and naturalness to perform these activities. During note sketching, people generally don’t pay much attention to the organization of notes, instead they use implicit spatial relationships and concise informal organizing symbols to structuralize notes. We conducted a study to discover how people sketch and organize their notes. The structuralized notes are easy to be maintained, manipulated and reused. Based on our observations, we designed algorithms to structuralize freeform notes into a consistent hierarchical structure. Both implicit spatial parsing and gesture-based structuralizing are enabled. With the supports of these algorithms, we built a system SketchPoint notebook as a tool that allows a user to sketch various notes in a freeform manner. It conducts lazy note structuralizing implicitly. The structuralized notes of SketchPoint enable structure-semantics based ink manipulations that bring much efficiency to users.