Mixed methods research: Toward new research designs in applied linguistics

Despite criticisms questioning its raison d'etre (e.g., Giddings & Grant, 2007), mixed methods research has been welcomed in social research as a methodology in its own right (Greene, 2008). Recently, it has also been acknowledged and advocated in applied linguistics (Dornyei, 2007; Hashemi, 2012). In an attempt to investigate the status of this relatively new trend in applied linguistics, the current study examines the nature of the integration of qualitative and quantitative methods in terms of research designs, sampling designs, and quality of interpretations. Content analysis of 205 research articles published in seven comprehensive international peer reviewed applied linguistics journals between 1995 and 2008 reveals that concurrent designs are more prevalent than sequential designs and that studies make limited use of mixed designs that are detailed in the mixed methods literature. Moreover, although a considerable number of articles used both qualitative and quantitative methods, only a small number achieved high degrees of integration at various stages of the study as a quality standard for mixed research. The study concludes with several implications for making more effective use of mixed methods research in applied linguistics and calls for a more systematic treatment of this trend as a versatile research methodology. [ABSTRACT FROM AUTHOR]

[1]  Souraya Sidani,et al.  Quantitative and qualitative methods:: Is There an Alternative? , 1995 .

[2]  Angela Calabrese Barton,et al.  Ecologies of Parental Engagement in Urban Education , 2004 .

[3]  Anne Lazaraton,et al.  Evaluative Criteria for Qualitative Research in Applied Linguistics: Whose Criteria and Whose Research? , 2003 .

[4]  Youn-hee Kim.  An investigation into native and non-native teachers' judgments of oral English performance: A mixed methods approach , 2009 .

[5]  Martyn Denscombe,et al.  Communities of Practice , 2008 .

[6]  Miyuki Sasaki,et al.  A Multiple‐Data Analysis of the 3.5‐Year Development of EFL Student Writers , 2004 .

[7]  Kathryn A. Davis,et al.  Qualitative Theory and Methods in Applied Linguistics Research , 1995 .

[8]  S. Gass,et al.  Quantitative Research Methods, Study Quality, and Outcomes: The Case of Interaction Research , 2011 .

[9]  Ming-chung Yu.  Teaching and Learning Sociolinguistic Skills in University EFL Classes in Taiwan , 2008 .

[10]  May Tan.  Mathematics and science teachers’ beliefs and practices regarding the teaching of language in content learning , 2011 .

[11]  D. Mertens.  Philosophy in mixed methods teaching: The transformative paradigm as illustration , 2010 .

[12]  Anne Lazaraton,et al.  Qualitative Research in Applied Linguistics: A Progress Report , 1995 .

[13]  Shahrzad Saif,et al.  Aiming for positive washback: a case study of international teaching assistants , 2006 .

[14]  Pat Bazeley,et al.  Editorial: Integrating Data Analyses in Mixed Methods Research , 2009 .

[15]  F. Vavrus.  Postcoloniality and English: Exploring language policy and the politics of development in Tanzania , 2002 .

[16]  Anne Lazaraton.  2. QUANTITATIVE AND QUALITATIVE APPROACHES TO DISCOURSE ANALYSIS , 2002, Annual Review of Applied Linguistics.

[17]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[18]  Alister Cumming,et al.  Alternatives in TESOL Research: Descriptive, Interpretive, and Ideological Orientations , 1994 .

[19]  C. Teddlie,et al.  Putting the Human Back in ‘‘Human Research Methodology’’: The Researcher in Mixed Methods Research , 2010 .

[20]  Evan S. Lieberman,et al.  Nested Analysis as a Mixed-Method Strategy for Comparative Research , 2005, American Political Science Review.

[21]  J. Creswell,et al.  Editorial: The New Era of Mixed Methods , 2007 .

[22]  Anthony J. Onwuegbuzie,et al.  Mixed Methods Analysis and Information Visualization: Graphical Display for Effective Communication of Research Results , 2008 .

[23]  Dawn Freshwater,et al.  Reading Mixed Methods Research: Contexts for Criticism , 2007 .

[24]  C. J. Fries,et al.  Bourdieu’s Reflexive Sociology as a Theoretical Basis for Mixed Methods Research , 2009 .

[25]  J. Greene.  Is Mixed Methods Social Inquiry a Distinctive Methodology? , 2008 .

[26]  Martin Lamb,et al.  The impact of school on EFL learning motivation: an Indonesian case study , 2007 .

[27]  Bonny Norton Peirce.  The Theory of Methodology in Qualitative Research , 1995 .

[28]  Greg Guest,et al.  Describing Mixed Methods Research , 2013 .

[29]  Anne Lazaraton.  Current Trends in Research Methodology and Statistics in Applied Linguistics , 2000 .

[30]  Catherine Elder,et al.  Judgments of oral proficiency by non-native and native English speaking teacher raters: Competing or complementary constructs? , 2011 .

[31]  Kirsten Howard,et al.  A Synergistic Approach , 2008 .

[32]  Nancy L. Leech,et al.  Linking Research Questions to Mixed Methods Data Analysis Procedures 1 , 2006 .

[33]  Anthony J. Onwuegbuzie,et al.  Mixed Research as a Tool for Developing Quantitative Instruments , 2010 .

[34]  K. Brazil,et al.  Revisiting the Quantitative-Qualitative Debate: Implications for Mixed-Methods Research , 2002, Quality & quantity.

[35]  A. Rudd,et al.  Getting to the Bottom Line: A Method for Synthesizing Findings Within Mixed-method Program Evaluations , 2002 .

[36]  Kathleen M. T. Collins,et al.  Prevalence of Mixed-methods Sampling Designs in Social Science Research , 2006 .

[37]  Norman K. Denzin,et al.  Triangulation 2.0 * , 2012 .

[38]  Thomas W. Christ,et al.  A Recursive Approach to Mixed Methods Research in a Longitudinal Study of Postsecondary Education Disability Support Services , 2007 .

[39]  Charles Teddlie,et al.  Mixed Methods Sampling A Typology With Examples , 2016 .

[40]  Madhabi Chatterji,et al.  Evidence on “What Works”: An Argument for Extended-Term Mixed-Method (ETMM) Evaluation Designs , 2004 .

[41]  Claire M. Woolley,et al.  Meeting the Mixed Methods Challenge of Integration in a Sociological Study of Structure and Agency , 2009 .

[42]  Jennifer Caroline Greene,et al.  Data Analysis Strategies for Mixed-Method Evaluation Designs , 1993 .

[43]  Gitte Sommer Harrits.  More Than Method?: A Discussion of Paradigm Differences Within Mixed Methods Research , 2011 .

[44]  Manfred Max Bergman,et al.  Advances in mixed methods research , 2008 .

[45]  M. Feilzer,et al.  Doing Mixed Methods Research Pragmatically: Implications for the Rediscovery of Pragmatism as a Research Paradigm: , 2010 .

[46]  Michael H. Long.  Second Language Needs Analysis: Methodological issues in learner needs analysis , 2005 .

[47]  Phil Benson,et al.  Qualitative Research in Language Teaching and Learning Journals, 1997-2006. , 2009 .

[48]  Burke Johnson,et al.  Editorial: Living With Tensions , 2008 .

[49]  Khaled Barkaoui,et al.  Do ESL Essay Raters' Evaluation Criteria Change With Experience? A Mixed‐Methods, Cross‐Sectional Study , 2010 .

[50]  J. Egbert.  Quality Analysis of Journals in TESOL and Applied Linguistics , 2007 .

[51]  Charles Teddlie,et al.  A Continuation of the Paradigm Wars? Prevalence Rates of Methodological Approaches Across the Social/Behavioral Sciences , 2010 .

[52]  Scott Kissau,et al.  Gender Identity and Homophobia: The Impact on Adolescent Males Studying French THIS ARTICLE HAS BEEN RETRACTED , 2008 .

[53]  Valerie J. Caracelli,et al.  Toward a Conceptual Framework for Mixed-Method Evaluation Designs , 1989 .

[54]  L. Giddings,et al.  A Trojan Horse for Positivism?: A Critique of Mixed Methods Research , 2007, ANS. Advances in nursing science.

[55]  D. Morgan.  Paradigms Lost and Pragmatism Regained , 2007 .

[56]  Pia Russell,et al.  Integrative Mixed Methods Data Analytic Strategies in Research on School Success in Challenging Circumstances , 2008 .

[57]  A. Onwuegbuzie,et al.  Mixed Methods Research: A Research Paradigm Whose Time Has Come , 2004 .

[58]  Mohammad R. Hashemi,et al.  Reflections on Mixing Methods in Applied Linguistics Research , 2012 .

[59]  Udo O.H. Jung.  Paris in London revisited or the foreign language teacher's top-most journals , 2004 .

[60]  Kim Hughes Wilhelm,et al.  Sometimes Kicking and Screaming: Language Teachers‐in‐Training React to a Collaborative Learning Model , 1997 .

[61]  Kathleen M. T. Collins,et al.  A Mixed Methods Investigation of Mixed Methods Sampling Designs in Social and Health Science Research , 2007 .

[62]  Laura L. Ellingson,et al.  Engaging Crystallization in Qualitative Research: An Introduction , 2008 .

[63]  John W. Creswell,et al.  Editorial: Exploring the Nature of Research Questions in Mixed Methods Research , 2007 .

[64]  A. Bryman.  Barriers to Integrating Quantitative and Qualitative Research , 2007 .

[65]  J. Creswell,et al.  Mixed Methods Research Designs in Counseling Psychology. , 2005 .

[66]  John H. Hitchcock,et al.  Mixed Methods in Intervention Research: Theory to Adaptation , 2007 .

[67]  Jo Durham,et al.  Utilizing Mixed Research Methods to Develop a Quantitative Assessment Tool , 2011 .

[68]  Rosalind Pritchard,et al.  Improving reading performance among Egyptian engineering students: principles and practice ☆ , 2004 .

[69]  Young-Ju Lee,et al.  The Predictive Validity of an ESL Placement Test , 2007 .

[70]  Jon Nicholl,et al.  Integration and Publications as Indicators of "Yield" From Mixed Methods Studies , 2007 .

[71]  N. Leech,et al.  Toward a Unified Validation Framework in Mixed Methods Research , 2007 .

[72]  D. Mertens.  Transformative Paradigm , 2007 .

[73]  Alison Mackey.  Feedback, noticing and instructed second language learning , 2006 .