The meaning and measurement of implementation climate

BackgroundClimate has a long history in organizational studies, but few theoretical models integrate the complex effects of climate during innovation implementation. In 1996, a theoretical model was proposed that organizations could develop a positive climate for implementation by making use of various policies and practices that promote organizational members' means, motives, and opportunities for innovation use. The model proposes that implementation climate--or the extent to which organizational members perceive that innovation use is expected, supported, and rewarded--is positively associated with implementation effectiveness. The implementation climate construct holds significant promise for advancing scientific knowledge about the organizational determinants of innovation implementation. However, the construct has not received sufficient scholarly attention, despite numerous citations in the scientific literature. In this article, we clarify the meaning of implementation climate, discuss several measurement issues, and propose guidelines for empirical study.DiscussionImplementation climate differs from constructs such as organizational climate, culture, or context in two important respects: first, it has a strategic focus (implementation), and second, it is innovation-specific. Measuring implementation climate is challenging because the construct operates at the organizational level, but requires the collection of multi-dimensional perceptual data from many expected innovation users within an organization. In order to avoid problems with construct validity, assessments of within-group agreement of implementation climate measures must be carefully considered. Implementation climate implies a high degree of within-group agreement in climate perceptions. However, researchers might find it useful to distinguish implementation climate level (the average of implementation climate perceptions) from implementation climate strength (the variability of implementation climate perceptions). It is important to recognize that the implementation climate construct applies most readily to innovations that require collective, coordinated behavior change by many organizational members both for successful implementation and for realization of anticipated benefits. For innovations that do not possess these attributes, individual-level theories of behavior change could be more useful in explaining implementation effectiveness.SummaryThis construct has considerable value in implementation science, however, further debate and development is necessary to refine and distinguish the construct for empirical use.

[1]  Steve W. J. Kozlowski,et al.  Integration of climate and leadership: Examination of a neglected issue. , 1989 .

[2]  B. Schneider,et al.  Climate strength: a new direction for climate research. , 2002, The Journal of applied psychology.

[3]  Organizations are not central tendencies: Shadowboxing in the dark, Round 2. , 1988 .

[4]  B. Schneider,et al.  Organizational climate and culture. , 2009, Annual review of psychology.

[5]  Teresa M. Amabile,et al.  Assessing the Work Environment for Creativity , 1996 .

[6]  Nicole E. Allen,et al.  Promoting systems change in the health care response to domestic violence , 2007 .

[7]  B. Mark,et al.  Quality and strength of patient safety climate on medical—surgical units , 2012, The Journal of nursing administration.

[8]  J. Slocum,et al.  Organizational Climate: Measures, Research and Contingencies , 1974 .

[9]  D. Rousseau Issues of level in organizational research: Multi-level and cross-level perspectives. , 1985 .

[10]  Susan S. White,et al.  Linking service climate and customer perceptions of service quality: test of a causal model. , 1998, The Journal of applied psychology.

[11]  G. H. Litwin,et al.  Motivation and Organizational Climate , 1968 .

[12]  Chien‐Ming Wu,et al.  Mediating Effect of Cooperative Norm in Predicting Organizational Citizenship Behaviors from Procedural Justice Climate , 2007, Psychological reports.

[13]  Gil Luria,et al.  Climate strength – How leaders form consensus , 2008 .

[14]  Brenda D. Smith,et al.  Inter-Agency Collaboration , 2008 .

[15]  A. Kaluzny,et al.  The Effects of Managed Care and Competition on Community-Based Clinical Research , 2006, Medical care.

[16]  G. Aarons Measuring provider attitudes toward evidence-based practice: consideration of organizational context and individual differences. , 2005, Child and adolescent psychiatric clinics of North America.

[17]  Chris Pullig,et al.  Salesforce automation systems: an exploratory examination of organizational factors associated with effective implementation and salesforce productivity , 2002 .

[18]  João Bosco Jardim,et al.  Diffusion of innovations in health service organisations: a systematic literature review , 2008 .

[19]  Samuel T. Hunter,et al.  Climate for Creativity: A Quantitative Review , 2007 .

[20]  E Doveh,et al.  Statistical properties of the rWG(J) index of agreement. , 2001, Psychological methods.

[21]  J. Dawson,et al.  Validating the organizational climate measure: links to managerial practices, productivity and innovation , 2005 .

[22]  W. Mischel Toward a cognitive social learning reconceptualization of personality. , 1973, Psychological review.

[23]  S. Kozlowski,et al.  From Micro to Meso: Critical Steps in Conceptualizing and Conducting Multilevel Research , 2000 .

[24]  Lawrence R. James,et al.  Organizational climate: A review of theory and research. , 1974 .

[25]  K. Klein,et al.  Levels Issues in Theory Development, Data Collection, and Analysis , 1994 .

[26]  D. Chan Functional Relations among Constructs in the Same Content Domain at Different Levels of Analysis: A Typology of Composition Models , 1998 .

[27]  Michael K. Lindell,et al.  Recent Issues Regarding rWG, rWG, rWG(J), and rWG(J) , 2005 .

[28]  K. Kelleher,et al.  Therapist Turnover and New Program Sustainability in Mental Health Clinics as a Function of Organizational Culture, Climate, and Service Structure , 2008, Administration and Policy in Mental Health and Mental Health Services Research.

[29]  B. Schneider Organizational Climates: An Essay. , 1975 .

[30]  Christopher A. Higgins,et al.  Testing Klein and Sorra's innovation implementation model: An empirical examination , 2008 .

[31]  K. Kelleher,et al.  Assessing the Organizational Social Context (OSC) of Mental Health Services: Implications for Research and Practice , 2008, Administration and Policy in Mental Health and Mental Health Services Research.

[32]  T. Perneger,et al.  Development and validation of a scale measuring self-efficacy of current and former smokers. , 2000, Addiction.

[33]  J. Lowery,et al.  Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science , 2009, Implementation science : IS.

[34]  Arnon E. Reichers,et al.  On the Etiology of Climates. , 1983 .

[35]  William H. Glick,et al.  Response: Organizations Are Not Central Tendencies: Shadowboxing in the Dark, Round 2 , 1988 .

[36]  L. James Aggregation Bias in Estimates of Perceptual Agreement. , 1982 .

[37]  D. Zohar A group-level model of safety climate: testing the effect of group climate on microaccidents in manufacturing jobs. , 2000, The Journal of applied psychology.

[38]  L. Palinkas,et al.  Implementation of Evidence-based Practice in Child Welfare: Service Provider Perspectives , 2007, Administration and Policy in Mental Health and Mental Health Services Research.

[39]  M. Johnston,et al.  Self-care self-efficacy, quality of life, and depression after stroke. , 2000, Archives of physical medicine and rehabilitation.

[40]  B. Weiner,et al.  Adapting clinical trials networks to promote cancer prevention and control research , 2006, Cancer.

[41]  R. Motl,et al.  Factorial invariance and latent mean structure of questionnaires measuring social-cognitive determinants of physical activity among black and white adolescent girls. , 2002, Preventive medicine.

[42]  Zvi H. Aronson,et al.  Implementing computer technology: a multiorganizational test of Klein and Sorra’s model , 2004 .

[43]  Neil M. A. Hauenstein,et al.  Interrater Agreement Reconsidered: An Alternative to the rwg Indices , 2005 .

[44]  F. Pouwer,et al.  The confidence in diabetes self-care scale: psychometric properties of a new measure of diabetes-specific self-efficacy in Dutch and US patients with type 1 diabetes. , 2003, Diabetes care.

[45]  L. Ryhammar,et al.  The Creative Climate: Its Determinants and Effects at a Swedish University , 1999 .

[46]  S. Owen,et al.  A measure of self-care self-efficacy. , 1996, Research in nursing & health.

[47]  Gro Ellen Mathisen,et al.  A Review of Instruments Assessing Creative and Innovative Environments Within Organizations , 2004 .

[48]  H. Knudsen,et al.  The adoption of wraparound services among substance abuse treatment organizations serving criminal offenders: The role of a women-specific program. , 2009, Drug and alcohol dependence.

[49]  B. Schneider,et al.  Employee and customer perceptions of service in banks: Replication and extension. , 1985 .

[50]  D. Zohar,et al.  Climate as a social-cognitive construction of supervisory safety practices: scripts as proxy of behavior patterns. , 2004, The Journal of applied psychology.

[51]  Jeroen K. Vermunt,et al.  Composing Group-Level Constructs From Individual-Level Survey Data , 2009 .

[52]  Danielle L. Fettes,et al.  Evidence-based practice implementation and staff emotional exhaustion in children's services. , 2009, Behaviour research and therapy.

[53]  H. Knudsen,et al.  Organizational-level correlates of the provision of detoxification services and medication-based treatments for substance abuse in correctional institutions. , 2009, Drug and alcohol dependence.

[54]  D. Zohar Safety climate in industrial organizations: theoretical and applied implications. , 1980, The Journal of applied psychology.

[55]  Vicente González-Romá,et al.  Team climate, climate strength and team performance. A longitudinal study , 2009 .

[56]  B. Schneider Employee and Customer Perceptions of Service in Banks. , 1980 .

[57]  B. Mark,et al.  Quality and strength of patient safety climate on medical-surgical units. , 2009, Health care management review.

[58]  Dov Zohar,et al.  Thirty years of safety climate research: reflections and future directions. , 2010, Accident; analysis and prevention.

[59]  William H. Glick,et al.  Conceptualizing and Measuring Organizational and Psychological Climate: Pitfalls in Multilevel Research , 1985 .

[60]  Aileen B. Sedman,et al.  A longitudinal study , 1987 .

[61]  Francis J. Yammarino,et al.  Avoiding the “Fallacy of the Wrong Level” , 2006 .

[62]  K. Hofman,et al.  Implementation Science , 2007, Science.

[63]  Brenda D. Smith,et al.  Multi-level influences on the practice of inter-agency collaboration in child welfare and substance abuse treatment , 2007 .

[64]  O. Claber,et al.  The Teesside Cancer Family History Service: change management and innovation at cancer network level , 2007, Familial Cancer.

[65]  C. Lau,et al.  An evaluation of the psychometric properties of the Smoking Self-Efficacy Questionnaire (SEQ-12) among Chinese cardiac patients who smoke. , 2008, Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco.

[66]  Harold W. Goldstein,et al.  The precursors and products of justice climates: Group leader antecedents and employee attitudinal consequences. , 2007 .

[67]  A. Kaluzny,et al.  Translating research into evidence‐based practice , 2010, Cancer.

[68]  L. James,et al.  Estimating within-group interrater reliability with and without response bias. , 1984 .

[69]  Trisha Greenhalgh,et al.  Diffusion of innovations in health service organisations , 2005 .

[70]  Raymond A. Noe,et al.  JUSTICE IN TEAMS: ANTECEDENTS AND CONSEQUENCES OF PROCEDURAL JUSTICE CLIMATE , 2002 .

[71]  Ayala Cohen,et al.  Statistical properties of the r-sub(WG(J) ) index of agreement. , 2001 .

[72]  Scott G. Isaksen,et al.  Situational Outlook Questionnaire: A Measure of the Climate for Creativity and Change , 1999 .

[73]  Joseph Finkelstein,et al.  Feasibility of Promoting Smoking Cessation Among Methadone Users Using Multimedia Computer-Assisted Education , 2008, Journal of medical Internet research.

[74]  K. Klein,et al.  The Challenge of Innovation Implementation , 1996 .

[75]  Inbal Nahum-Shani,et al.  Testing Agreement for Multi-Item Scales With the IndicesrWG(J) and AD M(J) , 2009 .

[76]  Nathan Bennett,et al.  A Case for Procedural Justice Climate: Development and Test of a Multilevel Model. , 2000 .

[77]  D. Zohar The effects of leadership dimensions, safety climate, and assigned priorities on minor injuries in work groups , 2002 .

[78]  Amy Buhl Conn,et al.  Implementing computerized technology: an organizational analysis. , 2001, The Journal of applied psychology.

[79]  Karlene H. Roberts,et al.  Hypothesized Interdependence, Assumed Independence , 1984 .

[80]  Allan P. Jones,et al.  Psychological climate: Dimensions and relationships of individual and aggregated work environment perceptions☆ , 1979 .

[81]  Christopher P. Parker,et al.  Psychological climate: A comparison of organizational and individual level referents , 2009 .

[82]  Daniel A. Newman,et al.  How Do Missing Data Bias Estimates of Within-Group Agreement? Sensitivity of SD WG, CVWG, rWG(J), rWG(J) * , and ICC to Systematic Nonresponse , 2009 .

[83]  Kweku-Muata Osei-Bryson,et al.  Exploring managerial factors affecting ERP implementation: an investigation of the Klein‐Sorra model using regression splines , 2008, Inf. Syst. J..

[84]  R. Guion A note on organizational climate , 1973 .

[85]  Marsha Dowda,et al.  Self-efficacy partially mediates the effect of a school-based physical-activity intervention among adolescent girls. , 2004, Preventive medicine.

[86]  C. Ruppel,et al.  Sharing knowledge through intranets: a study of organizational culture and intranet implementation , 2001 .

[87]  J. Peiró,et al.  An examination of the antecedents and moderator influences of climate strength. , 2002, The Journal of applied psychology.

[88]  Y. Donchin,et al.  Healthcare climate: A framework for measuring and improving patient safety* , 2007, Critical care medicine.

[89]  P. Bliese,et al.  Group Size and Measures of Group-Level Properties: An Examination of Eta-Squared and ICC Values , 1998 .

[90]  Amy Buhl Conn,et al.  Is everyone in agreement? An exploration of within-group agreement in employee perceptions of the work environment. , 2001, The Journal of applied psychology.

[91]  Ali E. Akgün,et al.  Procedural Justice Climate in New Product Development Teams: Antecedents and Consequences , 2010 .

[92]  B. Chorpita,et al.  Dissemination of effective mental health treatment procedures: Maximizing the return on a significant investment. , 2009, Behaviour research and therapy.