Multimodal analysis within an interactive software environment: critical discourse perspectives

Critical discourse analysts are increasingly required to account for multimodal phenomena constructed through language and other resources (e.g. images, sound and music) and to relate high-level critical insights on the social motivations of these texts to their realizations in low-level expressive phenomena, and vice versa. In this paper, we use interactive software resources for critical multimodal discourse analysis (O'Halloran, K.L. (2011b). Multimodal analysis and digital technology. In A. Baldry & E. Montagna (Eds.), Interdisciplinary approaches to multimodality: Theory and practice. Readings in intersemiosis and multimedia (pp. 21–34). Campobasso: Palladino; O'Halloran, K. L., Tan, S., Smith, B. A., & Podlasov, A. (2010). Challenges in designing digital interfaces for the study of multimodal phenomena. Information Design Journal, 18(1), 2–12; Smith, B. A., Tan, S., Podlasov, A., & O'Halloran, K.L. (in press). Analyzing multimodality in an interactive digital environment: Software as metasemiotic tool. Social Semiotics) to help in achieving these aims. The field of critical discourse studies, itself being interdisciplinary, along with its holistic approach, is well placed to take advantage of interactive software, as these resources encourage the application, exploration and correlation of various analytical perspectives at different levels of description. Drawing on the analysis of a short video advertisement, we demonstrate how the interactive (multimodal) digital environment is one in which the discourse analyst can effectively draw upon different traditions of analysis, including ‘mainstream’ and social semiotic traditions, as well as other traditions such as media studies, to interpret dynamic audiovisual media texts in a critically self-reflexive manner.

[1]  Rodney H. Jones.  Multimodal Discourse Analysis , 2012, The Encyclopedia of Applied Linguistics.

[2]  D. Chandler.  Semiotics For Beginners , 2011 .

[3]  D. Graddol,et al.  Redesigning English : New Texts, New Identities , 1996 .

[4]  Michael Halliday,et al.  An Introduction to Functional Grammar , 1985 .

[5]  O. Halloran.  Challenges in Designing Digital Interfaces for the Study of Multimodal Phenomena , 2009 .

[6]  D. Rader.  Social Semiotics , 2013 .

[7]  David Bordwell,et al.  Film Art: An Introduction , 1979 .

[8]  N. Fairclough.  Discourse and social change , 1992 .

[9]  Donald Matheson.  Multimodal Discourse: The Modes and Media of Contemporary Communication , 2002 .

[10]  Ferdinand de Saussure.  Course in General Linguistics , 1916 .

[11]  T. V. Leeuwen.  Speech, Music, Sound , 1999 .

[12]  Theo van Leeuwen,et al.  Reading Images: The Grammar of Visual Design , 1996 .

[13]  Douglas Brown,et al.  Film Theory: An Introduction , 2013 .

[14]  K. O'Halloran,et al.  Multimodal analysis and digital technology , 2013 .

[15]  E. Hall,et al.  The Hidden Dimension , 1970 .

[16]  S. Cohan,et al.  Screening the Male: Exploring Masculinities in the Hollywood Cinema , 1993 .

[17]  R. Birdwhistell.  Kinesics and Context: Essays on Body Motion Communication , 1971 .

[18]  Jan-Torsten Milde,et al.  Comparison of multimodal annotation tools: Workshop report , 2006 .

[19]  L. M. O'Toole.  The language of displayed art , 1994 .

[20]  Tony McEnery,et al.  A useful methodological synergy? Combining critical discourse analysis and corpus linguistics to examine discourses of refugees and asylum seekers in the UK press , 2008 .

[21]  S. Cohan,et al.  PROLOGUE MASCULINITY AS SPECTACLE : Reflections on men and mainstream cinema , 2012 .

[22]  Tomlinson Holman,et al.  Sound for film and television , 1997 .

[23]  E. Ventola,et al.  The World Told and the World Shown: Multisemiotic Issues , 2009 .

[24]  R. Wodak,et al.  Methods of critical discourse analysis , 2001 .

[25]  Christian Heath,et al.  Handbook of Multimodal Analysis , 2009 .

[26]  Alexey Podlasov,et al.  Analysing multimodality in an interactive digital environment: software as a meta-semiotic tool , 2011 .

[27]  Sabine Tan.  A Systemic Functional Framework for the Analysis of Corporate Television Advertisements , 2009 .

[28]  R. Barthes,et al.  Image-Music-Text , 1977 .

[29]  Edward T. Hall,et al.  A System for the Notation of Proxemic Behavior1 , 1963 .

[30]  Gerlinde Mautner.  Mining large corpora for social information: The case of elderly , 2007, Language in Society.

[31]  T. V. Leeuwen.  Semiotics and Iconography , 2004 .

[32]  Michael Halliday.  Language As Social Semiotic , 1978 .

[33]  Mike Budd,et al.  Consuming Environments: Television and Commercial Culture , 1999 .

[34]  K. O'Halloran,et al.  Title : Analyzing multimodality in an interactive digital environment : Software as metasemiotic tool , 2010 .

[35]  R. Wodak,et al.  Critical Discourse Analysis. Theory and Interdisciplinarity. , 2003 .

[36]  Veronika Koller.  “The World's Local Bank”: Glocalisation as a Strategy in Corporate Branding Discourse , 2007 .

[37]  P. Ekman,et al.  What the face reveals : basic and applied studies of spontaneous expression using the facial action coding system (FACS) , 2005 .

[38]  C. Jewitt,et al.  The Handbook of Visual Analysis , 2000 .

[39]  R. Wodak.  Critical discourse analysis: an example , 2009 .

[40]  Daniel Chandler,et al.  Semiotics: The Basics , 2001 .