Sketching for military courses of action diagrams

A serious barrier to the digitalization of the US military is that commanders find traditional mouse/menu, CAD-style interfaces unnatural. Military commanders develop and communicate battle plans by sketching courses of action (COAs). This paper describes nuSketch Battlespace, the latest version in an evolving line of sketching interfaces that commanders find natural, yet supports significant increased automation. We describe techniques that should be applicable to any specialized sketching domain: glyph bars and compositional symbols to tractably handle the large number of entities that military domains use, specialized glyph types and gestures to keep drawing tractable and natural, qualitative spatial reasoning to provide sketch-based visual reasoning, and comic graphs to describe multiple states and plans. Experiments, both completed and in progress, are described to provide evidence as to the utility of the system.

[1]  James A. Landay,et al.  Sketching storyboards to illustrate interface behaviors , 1996, CHI Conference Companion.

[2]  Kenneth D. Forbus,et al.  How qualitative spatial reasoning can improve strategy game AIs , 2002, IEEE Intelligent Systems.

[3]  Mark D. Gross,et al.  DRAWING ANALOGIES Supporting Creative Architectural Design with Visual References , 1996 .

[4]  Anthony G. Cohn,et al.  Calculi for Qualitative Spatial Reasoning , 1996, AISMC.

[5]  Philip R. Cohen,et al.  QuickSet: multimodal interaction for distributed applications , 1997, MULTIMEDIA '97.

[6]  David L. Westbrook,et al.  Models of defeat , 2002, Proceedings of the Winter Simulation Conference.

[7]  Kenneth D. Forbus.  Introducing Actions into Qualitative Simulation , 1989, IJCAI.

[8]  Kenneth D. Forbus,et al.  Qualitative Spatial Interpretation of Course-of-Action Diagrams , 2000, AAAI/IAAI.

[9]  Philip R. Cohen,et al.  Comparing paper and tangible, multimodal tools , 2002, CHI.

[10]  Kenny R. Coventry.  Spatial prepositions, functional relations, and lexical specification , 1998 .

[11]  Philip R. Cohen,et al.  Creating tangible interfaces by augmenting physical objects with multimodal language , 2001, IUI '01.

[12]  N. Hari Narayanan,et al.  Diagrammatic Reasoning: Cognitive and Computational Perspectives , 1995 .

[13]  Brian Falkenhainer,et al.  The Structure-Mapping Engine: Algorithm and Examples , 1989, Artif. Intell..

[14]  Kenneth D. Forbus,et al.  AI on the battlefield: an experimental exploration , 2002, AAAI/IAAI.

[15]  Ian Lane Davis.  Warp Speed: Path Planning for Star Trek®: Armada , 2000 .

[16]  Mark D. Gross,et al.  The electronic cocktail napkin--computer support for working with diagrams , 1996 .

[17]  Alex Waibel,et al.  Multimodal interfaces for multimedia information agents , 1997 .

[18]  Kenneth D. Forbus,et al.  Qualitative Spatial Reasoning: The Clock Project , 1991, Artif. Intell..

[19]  Kenneth D. Forbus,et al.  Incremental Structure-Mapping , 2019, Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society.

[20]  Kenneth D. Forbus.  Qualitative Spatial Reasoning: Framework and Frontiers , 1992 .

[21]  Thomas F. Stahovich,et al.  Generating Multiple New Designs from a Sketch , 1996, Artif. Intell..

[22]  Kenneth D. Forbus,et al.  Sketching for knowledge capture: a progress report , 2002, IUI '02.

[23]  Kenneth D. Forbus,et al.  MAC/FAC: A Model of Similarity-Based Retrieval , 1995, Cogn. Sci..

[24]  Kenneth D. Forbus,et al.  Towards a computational model of sketching , 2001, IUI '01.

[25]  Geoffrey Edwards,et al.  Toward the simulation of spatial mental images using the Voronoi¨ model , 1998 .

[26]  Patrick Oliver,et al.  Representation and Processing of Spatial Expressions , 1998 .

[27]  Kenneth D. Forbus,et al.  Building Problem Solvers , 1993 .

[28]  Mark D. Gross,et al.  The Electronic Cocktail Napkin—a computational environment for working with design diagrams , 1996 .