Visual representation of information as communicative practice

Anyone who has clarified a thought or prompted a response during a conversation by drawing a picture has exploited the potential of image making to convey information. Images are increasingly ubiquitous in daily communication due to advances in visually enabled information and communication technologies (ICT), such as information visualization applications, image retrieval systems, and virtual collaborative work tools. Although images are often used in social contexts, information science research concerned with the visual representation of information typically focuses on the image artifact and system building. To learn more about image making as a form of social interaction and as a form of information practice, a qualitative study examined face‐to‐face conversations involving the creation of ad hoc visualizations (i.e., “napkin drawings”). Interactional sociolinguistic concepts of conversational involvement and coordination guided multimodal analysis of video‐recorded interactions that included spontaneous drawing. Findings show patterns in communicative activities associated with the visual representation of information. Furthermore, the activity of mark making contributes to the maintenance of conversational involvement in ways that are not always evident in the drawn artifact. This research has implications for the design and evaluation of visually enabled virtual collaboration environments, visual information extraction and retrieval systems, and data visualization tools.

[1]  Hans Hagen,et al.  Collaborative visualization: Definition, challenges, and research agenda , 2011, Inf. Vis..

[2]  Gillian Rose,et al.  Visuality/Materiality: Images, Objects and Practices , 2012 .

[3]  W. Hanks Language & communicative practices , 1995 .

[4]  James Elkins,et al.  The domain of images , 1999 .

[5]  Joan E. Beaudoin,et al.  Finding Visual Information: A Study of Image Resources Used by Archaeologists, Architects, Art Historians, and Artists , 2011, Art Documentation: Journal of the Art Libraries Society of North America.

[6]  Charles Goodwin,et al.  Pointing as Situated Practice , 1998 .

[7]  Jürgen Streeck,et al.  Interaction by inscription , 2001 .

[8]  M. Patterson,et al.  The SAGE Handbook of Nonverbal Communication , 2006 .

[9]  K. Scherer,et al.  The New Handbook of Methods in Nonverbal Behavior Research , 2008 .

[10]  Rodney H. Jones,et al.  Discourse in Action: Introducing Mediated Discourse Analysis , 2005 .

[11]  EunKyung Chung,et al.  Understanding image needs in daily life by analyzing questions in a social Q&A site , 2011, J. Assoc. Inf. Sci. Technol..

[12]  J. Gumperz Discourse strategies: Introduction , 1982 .

[13]  E. Winters,et al.  Visual Culture: The Study of the Visual after the Cultural Turn , 2006 .

[14]  B. Marx The Visual Display of Quantitative Information , 1985 .

[15]  J. Harrigan,et al.  Proxemics, kinesics, and gaze. , 2005 .

[16]  Julia Hirschberg,et al.  Instructions for annotating discourse , 1995 .

[17]  Elinor Ochs,et al.  Interaction and grammar: “When I come down I'm in the domain state”: grammar and graphic representation in the interpretive activity of physicists , 1996 .

[18]  Marja-Liisa Helasvuo,et al.  DISCOURSE AND TECHNOLOGY: MULTIMODAL DISCOURSE ANALYSIS , 2005, Studies in Second Language Acquisition.

[19]  Norah E. Dunbar,et al.  Testing the Interactivity Principle: Effects of Mediation, Propinquity, and Verbal and Nonverbal Modalities in Interpersonal Interaction , 2002 .

[20]  Deborah Tannen,et al.  Interactive Frames and Knowledge Schemas in Interaction: Examples from a Medical Examination/Interview , 1987 .

[21]  Weidong Huang,et al.  Beyond time and error: a cognitive approach to the evaluation of graph drawings , 2008, BELIV '08.

[22]  Ron Scollon,et al.  Mediated Discourse As Social Interaction: A Study of News Discourse , 1998 .

[23]  Paul Tibbetts,et al.  Representation and the realist-constructivist controversy , 1988 .

[24]  T. V. Dijk,et al.  EPISODES AS UNITS OF DISCOURSE ANALYSIS , 2006 .

[25]  Nelson Goodman,et al.  Languages of Art, an Approach to a Theory of Symbols , 1970 .

[26]  B. Asher The Professional Vision , 1994 .

[27]  Jaime Snyder,et al.  Drawing practices in image-enabled collaboration , 2013, CSCW.

[28]  付伶俐 打磨Using Language,倡导新理念 , 2014 .

[29]  J. Gumperz Discourse strategies: Subject index , 1982 .

[30]  Charles Goodwin,et al.  Participation, stance and affect in the organization of activities , 2007 .

[31]  Charles Goodwin Language and Gesture: Gesture, aphasia, and interaction , 2000 .

[32]  Elaine Toms,et al.  Image use within the work task model: Images as information and illustration , 2009, J. Assoc. Inf. Sci. Technol..

[33]  Martin Wattenberg,et al.  Voyagers and voyeurs: supporting asynchronous collaborative information visualization , 2007, CHI.

[34]  Sandra Mathison,et al.  Seeing Is Believing: The Credibility of Image-Based Research and Evaluation , 2009 .

[35]  M. Sheelagh T. Carpendale,et al.  Visual Thinking In Action: Visualizations As Used On Whiteboards , 2011, IEEE Transactions on Visualization and Computer Graphics.

[36]  Martin Wattenberg,et al.  Voyagers and voyeurs: Supporting asynchronous collaborative visualization , 2009, CACM.

[37]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[38]  John J. Bertin,et al.  The semiology of graphics , 1983 .

[39]  Katy Bennett,et al.  Visual methodologies. An Introduction to the Interpretation of visual materials. Doi: 10.5212/Rlagg.v.3.i2.178180 , 2012 .

[40]  Peter Fox,et al.  Changing the Equation on Scientific Data Visualization , 2011, Science.

[41]  Ben Shneiderman,et al.  Strategies for evaluating information visualization tools: multi-dimensional in-depth long-term case studies , 2006, BELIV '06.

[42]  Sandra A. Thompson,et al.  Interaction and grammar: Frontmatter , 1996 .

[43]  Edward R. Tufte,et al.  Envisioning Information , 1990 .

[44]  R. D. Guthrie The nature of Paleolithic art , 2005 .

[45]  Judee K. Burgoon,et al.  Does Participation Affect Deception Success? A Test of the Interactivity Principle , 2001 .

[46]  喜多 壮太郎 Pointing : where language, culture, and cognition meet , 2013 .

[47]  Theo van Leeuwen,et al.  Reading Images: The Grammar of Visual Design , 1996 .

[48]  Jaime Snyder,et al.  Activities & artifacts: the dual nature of image-making in communicative practice , 2012, iConference '12.

[49]  Christin Wirth,et al.  Analyzing Multimodal Interaction A Methodological Framework , 2016 .

[50]  B. Latour Drawing Things Together , 2011 .

[51]  A. Jaffe Stance: Sociolinguistic Perspectives , 2009 .

[52]  Penelope Pitt Visuality/materiality: Images, objects and practices , 2013 .

[53]  Alan J. Dix,et al.  An explorative analysis of user evaluation studies in information visualisation , 2006, BELIV '06.

[54]  Judee K. Burgoon,et al.  Does Participation Affect Deception Success , 2001 .

[55]  P. Ekman,et al.  Handbook of methods in nonverbal behavior research , 1982 .

[56]  Edward Tufte,et al.  Visual Explanations , 1997 .

[57]  Colin Ware,et al.  Information Visualization: Perception for Design , 2000 .

[58]  N. Goodman,et al.  Languages of art : an approach to a theory of symbols , 1979 .

[59]  Christina A. Christie,et al.  What Counts as Credible Evidence in Applied Research and Evaluation Practice , 2008 .

[60]  Barbara Johnstone Qualitative methods in sociolinguistics , 1999 .

[61]  Claudia Mitchell,et al.  Doing Visual Research , 2011 .

[62]  Mary Elizabeth Lynch,et al.  The externalized retina: Selection and mathematization in the visual documentation of objects in the life sciences , 1988 .

[63]  J. Hindmarch Video in Qualitative Research: Analysing Social Interaction in Everyday Life , 2012, QMiP Bulletin.

[64]  K. K. Cetina,et al.  The fixation of (visual) evidence , 1988 .

[65]  Deborah Tannen,et al.  Framing in Discourse , 1993 .

[66]  Wolfgang Klein,et al.  Frame of analysis , 1995 .

[67]  Stephann Makri,et al.  Information for inspiration: Understanding architects' information seeking and use behaviors to inform design , 2010 .