Analysing videos in educational research: an “Inquiry Graphics” approach for multimodal, Peircean semiotic coding of video data

This article introduces an “Inquiry Graphics” (IG) approach for multimodal, Peircean semiotic video analysis and coding. It builds on Charles Sanders Peirce’s core triadic interpretation of sign meaning-making. Multimodal methods offer analytical frameworks, templates and software to analyse video data. However, multimodal video analysis has been scarcely linked to semiotics in/of education (edusemiotics), for the purpose of exploring higher education teaching-learning and settings. This article addresses the mentioned gap by introducing the IG approach, which links multimodality and edusemiotics primarily via Peirce’s triadic sign. The article offers a step-by-step IG coding guide, examples and explanations. IG application can be expanded to video analysis across many fields, levels and subjects, within and beyond higher education research, nationally and internationally.

[1]  B. Cope,et al.  Multiliteracies: Literacy Learning and the Design of Social Futures , 2001 .

[2]  Mie Buhl.  Multimodality: on video mediated counselling for educational purposes , 2010 .

[3]  Daniel Chandler,et al.  Semiotics: The Basics , 2001 .

[4]  Alexey Podlasov,et al.  Multimodal analysis within an interactive software environment: critical discourse perspectives , 2011 .

[5]  Kathrin Otrel-Cass,et al.  Coding the Complexity of Activity in Video Recordings: A proposal for constructing codes for video analysis using activity theory , 2017 .

[6]  C. Jewitt.  The Routledge Handbook of Multimodal Analysis (2nd Edition) , 2014 .

[8]  Jeff Bezemer.  14. Multimodal transcription: A case study , 2014 .

[9]  M. Anstey,et al.  Teaching and Learning Multiliteracies: Changing Times, Changing Literacies , 2006 .

[10]  Arlene Archer,et al.  Multimodality in Higher Education , 2016 .

[11]  Review of Edusemiotics , 2016 .

[12]  Eric Laurier,et al.  The Graphic Transcript: Poaching Comic Book Grammar for Inscribing the Visual, Spatial and Temporal Aspects of Action , 2014 .

[13]  C. Jewitt.  The Routledge handbook of multimodal analysis , 2014 .

[14]  W. Orlikowski.  Sociomaterial Practices: Exploring Technology at Work , 2007 .

[15]  Cary Campbell,et al.  A Short Introduction to Edusemiotics , 2018 .

[16]  Lucia Thesen.  The Past in the Present: Modes, Gaze and Changing Communicative Practices in Lectures , 2016 .

[17]  P. Freebody,et al.  Literacies programs: Debates and demands in cultural context , 1990 .

[18]  Theo van Leeuwen,et al.  Semiotic technology and practice: a multimodal social semiotic approach to PowerPoint , 2014 .

[19]  Dario Martinelli,et al.  A Critical Companion to Zoosemiotics , 2010 .

[20]  S. Derry,et al.  Video Research in the Learning Sciences , 2007 .

[21]  G. Kress,et al.  Exploring Learning Through Visual, Actional and Linguistic Communication: The multimodal environment of a science classroom , 2001 .

[22]  Arlene Archer.  Multimodal texts in higher education and the implications for writing pedagogy , 2010 .

[23]  S. Michaels,et al.  A pedagogy of Multiliteracies Designing Social Futures , 1996 .

[24]  G. Kress,et al.  Multimodal Discourse: The Modes and Media of Contemporary Communication , 2002 .

[25]  M. Peters,et al.  Post-Truth, Fake News: Viral Modernity & Higher Education , 2018 .

[26]  Gunther Kress,et al.  Multimodality, Learning and Communication: A Social Semiotic Frame , 2015 .

[27]  A. Olteanu.  Philosophy of Education in the Semiotics of Charles Peirce: A Cosmology of Learning and Loving , 2015 .

[28]  Amy Vetter,et al.  A Discourse Analytic Approach to Video Analysis of Teaching , 2015 .

[29]  K. O'Halloran,et al.  Book Review: Multimodality and Multimediality in the Distance Learning Age , 2002 .

[30]  A. Stables,et al.  Edusemiotics: Semiotic philosophy as educational foundation , 2014 .

[31]  Kathrin Otrel-Cass.  Reflections on algorithmic thinking for video analysis: sorting out complex human activities , 2018 .

[33]  Sara Price,et al.  Mobile Experiences of Historical Place: A Multimodal Analysis of Emotional Engagement , 2016 .

[34]  Frederik Stjernfelt.  Signs Conveying Information , 2011 .

[35]  G. Kress.  Multimodality: A Social Semiotic Approach to Contemporary Communication , 2009 .

[36]  Peirce's Semiotics, Subdoxastic Aboutness, and the Paradox of Inquiry , 2005 .

[37]  D. Mavers,et al.  Multimodal transcription as academic practice: a social semiotic perspective , 2011 .

[38]  R. Barthes,et al.  Image-Music-Text , 1977 .

[39]  Lesley Gourlay.  Multimodality, argument and the persistence of written text , 2016 .

[40]  Sigrid Norris,et al.  Analyzing Multimodal Interaction: A Methodological Framework , 2004 .

[41]  Colin Lankshear,et al.  Sampling "the new" in New Literacies , 2007 .

[42]  M. Hallewell,et al.  Do pictures ‘tell’ a thousand words in lectures? How lecturers vocalise photographs in their presentations , 2017 .

[43]  Eetu Pikkarainen,et al.  Competence as a Key Concept of Educational Theory: A Semiotic Point of View. , 2014 .

[44]  R. Iedema.  Multimodality, resemiotization: extending the analysis of discourse as multi-semiotic practice , 2003 .

[45]  Sara Price,et al.  The semiotic work of the hands in scientific enquiry , 2014 .

[46]  P. Leonardi,et al.  Materiality and Organizing: Social Interaction in a Technological World , 2013 .

[47]  A. Olteanu.  The Semiosic Evolution of Education. , 2014 .

[48]  Sarah A. Nagro,et al.  Evaluating the Evidence Base of Video Analysis , 2013 .