Assessing and Evaluating Multidisciplinary Translational Teams

A case report illustrates how multidisciplinary translational teams can be assessed using outcome, process, and developmental types of evaluation using a mixed-methods approach. Types of evaluation appropriate for teams are considered in relation to relevant research questions and assessment methods. Logic models are applied to scientific projects and team development to inform choices between methods within a mixed-methods design. Use of an expert panel is reviewed, culminating in consensus ratings of 11 multidisciplinary teams and a final evaluation within a team-type taxonomy. Based on team maturation and scientific progress, teams were designated as (a) early in development, (b) traditional, (c) process focused, or (d) exemplary. Lessons learned from data reduction, use of mixed methods, and use of expert panels are explored.

[1]  H. S. Booker,et al.  Research Methods for the Behavioral Sciences , 2020 .

[2]  T. McLeish,et al.  Evaluating interdisciplinary research: the elephant in the peer-reviewers’ room , 2016, Palgrave Communications.

[3]  C. Blank.  SAGE handbook of mixed methods in social & behavioral research , 2013 .

[4]  Howard Gadlin,et al.  Collaboration & team science: a field guide , 2013 .

[5]  Kevin C. Wooten,et al.  The Culture of Translational Science Research , 2013, International review of qualitative research : IRQR.

[6]  Kevin Wooten,et al.  The CTSA as an Exemplar Framework for Developing Multidisciplinary Translational Teams , 2013, Clinical and translational science.

[7]  Daniel Stokols,et al.  A four-phase model of transdisciplinary team-based research: goals, team processes, and strategies , 2012, Translational behavioral medicine.

[8]  Frances Lawrenz,et al.  Expert panel reviews of research centers: the site visit process. , 2012, Evaluation and program planning.

[9]  Marshall Scott Poole,et al.  Research methods for studying groups and teams : a guide to approaches, tools, and technologies , 2012 .

[10]  Carl Mitcham,et al.  The Oxford Handbook of Interdisciplinarity , 2012 .

[11]  Holly J. Falk-Krzesinski,et al.  Mapping a research agenda for the science of team science. , 2011, Research Evaluation.

[12]  Allan Best,et al.  Evaluating the Science of Discovery in Complex Health Systems , 2011 .

[13]  Joon-Oh Park,et al.  The Increasing Dominance of Teams in Production of Knowledge , 2011 .

[14]  Katy Börner,et al.  Advancing the Science of Team Science , 2010, Clinical and translational science.

[15]  Lutz Bornmann,et al.  Panel peer review of grant applications: what do we know from research in social psychology on judgment and decision-making in groups? , 2010 .

[16]  Katy Börner,et al.  A Multi-Level Systems Perspective for the Science of Team Science , 2010, Science Translational Medicine.

[17]  Michael Quinn Patton,et al.  Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use , 2010 .

[18]  Liz Hoggarth,et al.  A Practical Guide to Outcome Evaluation , 2010 .

[19]  John T. Slattery,et al.  The Road We Must Take: Multidisciplinary Team Science , 2010, Science Translational Medicine.

[20]  D. Rubio,et al.  Defining Translational Research: Implications for Training , 2010, Academic medicine : journal of the Association of American Medical Colleges.

[21]  Benjamin F. Jones,et al.  Multi-University Research Teams: Shifting Impact, Geography, and Stratification in Science , 2008, Science.

[22]  Abbas Tashakkori,et al.  Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences , 2008 .

[23]  Kara L Hall,et al.  Moving the science of team science forward: collaboration and creativity. , 2008, American journal of preventive medicine.

[24]  Kara L Hall,et al.  Measuring collaboration and transdisciplinary integration in team science. , 2008, American journal of preventive medicine.

[25]  Kara L. Hall,et al.  The science of team science: overview of the field and introduction to the supplement. , 2008, American journal of preventive medicine.

[26]  S. Fiore.  Interdisciplinarity as Teamwork , 2008 .

[27]  Denis O. Gray,et al.  Making team science better: Applying improvement‐oriented evaluation principles to evaluation of cooperative research centers , 2008 .

[28]  Tammy L. Rapp,et al.  Team Effectiveness 1997-2007: A Review of Recent Advancements and a Glimpse Into the Future , 2008 .

[29]  Stephen E. Marcus,et al.  The Evaluation of Large Research Initiatives , 2008 .

[30]  Chris L. S. Coryn,et al.  Models and Mechanisms for Evaluating Government-Funded Research , 2007 .

[31]  K. Wade,et al.  Practice-Based Research , 2007, Social work in health care.

[32]  A. Onwuegbuzie,et al.  Toward a Definition of Mixed Methods Research , 2007 .

[33]  Kathleen M. Carley,et al.  Social Network Analysis to Evaluate an Interdisciplinary Research Center , 2007 .

[34]  Joy A. Frechtling.  Logic Modeling Methods in Program Evaluation , 2007 .

[35]  J. Westfall,et al.  Practice-based research--"Blue Highways" on the NIH roadmap. , 2007, JAMA.

[36]  John W. Creswell,et al.  Designing and Conducting Mixed Methods Research , 2006 .

[37]  Daniel Stokols,et al.  Toward a Science of Transdisciplinary Action Research , 2006, American journal of community psychology.

[38]  E. Zerhouni.  Clinical Research at a Crossroads , 2006, Journal of Investigative Medicine.

[39]  Julie Thompson Klein,et al.  Afterword: the emergent literature on interdisciplinary and transdisciplinary research evaluation , 2006 .

[40]  Hanne Foss Hansen,et al.  Choosing Evaluation Models , 2005 .

[41]  Ruth P Saunders,et al.  Developing a Process-Evaluation Plan for Assessing Health Promotion Program Implementation: A How-To Guide , 2005, Health promotion practice.

[42]  Dustin K. Jundt,et al.  Teams in organizations: from input-process-output models to IMOI models. , 2005, Annual review of psychology.

[43]  William M. K. Trochim.  Research Methods: The Concise Knowledge Base , 2004 .

[44]  M. Clark,et al.  Evaluating transdisciplinary science. , 2003, Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco.

[45]  G. Vandervegt.  Joint Impact of Interdependence and Group Diversity on Innovation , 2003 .

[46]  Stephen B. Johnson,et al.  Central challenges facing the national clinical research enterprise. , 2003, JAMA.

[47]  Dennis J. Devine,et al.  Teams in Organizations , 1999 .

[48]  S. G. Cohen,et al.  What Makes Teams Work: Group Effectiveness Research from the Shop Floor to the Executive Suite , 1997 .

[49]  Richard A. Guzzo,et al.  Teams in organizations: recent research on performance and effectiveness. , 1996, Annual review of psychology.

[50]  E. Sundstrom,et al.  Work teams: Applications and effectiveness. , 1990 .

[51]  Valerie J. Caracelli,et al.  Toward a Conceptual Framework for Mixed-Method Evaluation Designs , 1989 .

[52]  References , 1971 .