Intervention Selection: An Examination of Evidence and Changes in Belief During the Decision-Making Process

This study examines what sources of evidence are used in intervention selection and what changes in belief occur when performance improvement professionals make these decisions. Sixty-one certified performance technologists completed a dynamic, web-delivered questionnaire in which they provided a general assessment of intervention success (Pr1), then responded to 12 performance improvement scenarios by selecting an intervention, providing a prior probability, receiving additional evidence, giving a posterior probability (Pr3), indicating whether the initial intervention was still preferred, and making a subsequent choice if not. Findings bolster the long-standing concern about the technical nature of performance improvement, and practitioners are strongly encouraged to approach intervention selection as a decision, where their intervention preferences and beliefs of likely success are carefully adjudicated on the basis of the evidence they obtain. Future research with other types of performance improvement practitioners, replication studies, longitudinal, structural equation modeling, externally verifiable probabilities, and natural environments are recommended.

[1]  Joe H. Harless.  Performance Technology Skills in Business: Implications for Preparation , 2008 .

[2]  Peter R. Hybert.  Give customers what they meant to ask for designing training systems at three levels , 2001 .

[3]  Linda Huglin,et al.  Research Priorities in Performance Technology A Delphi Study , 2008 .

[4]  Richard E. Clark,et al.  Fostering the work motivation of individuals and teams , 2003 .

[6]  Jieun Lee,et al.  Current status, future trends, and issues in human performance technology, part 2: Models, influential disciplines, and research and development , 2008 .

[7]  C. Chapman.  Prior Probability Bias in Information Seeking and Opinion Revision. , 1973 .

[8]  Richard E. Clark,et al.  Turning research and evaluation into results for ISPI , 2003 .

[9]  B. McCormack,et al.  What counts as evidence in evidence-based practice? , 2004, Journal of advanced nursing.

[10]  Richard E. Clark,et al.  Snake Oil, Science, and Performance Products. , 2000 .

[11]  Gordon Rowland,et al.  What Do Instructional Designers Actually Do? An Initial Investigation of Expert Practice. , 2008 .

[12]  Jeff A. Weekley,et al.  FURTHER STUDIES OF SITUATIONAL TESTS , 1999 .

[13]  Cathleen Smith Hutchison,et al.  A Whole New World of Interventions: The Performance Technologist as Integrating Generalist , 1997 .

[14]  Dale M. Brethower,et al.  Historical background for HPT Certification Standard 2, take a systems view, part 2 , 2008 .

[15]  Sharon J. Korth,et al.  Creativity and the Design Process , 2008 .

[16]  Ingrid J. Guerra.  Performance Improvement Based on Results: Is Our Field Interested in Adding Value?. , 2001 .

[17]  Michael Z. Sincoff,et al.  Content Guidelines for an Undergraduate Human Resources Curriculum: Recommendations from Human Resources Professionals. , 2004 .

[18]  Cathleen Hutchison,et al.  Potential strategies and tactics for organizational performance improvement , 1996 .

[19]  James A. Pershing,et al.  International Society for Performance Improvement Professional Practices Survey: A report , 2006 .

[20]  Richard E. Clark,et al.  A Proposal for the Collaborative Development of Authentic Performance Technology. , 2000 .

[21]  John Fox,et al.  Training Situation Analysis: Conducting a Needs Analysis for Teams and New Systems. , 1998 .

[22]  Barry M. Staw,et al.  Knee-deep in the Big Muddy: A study of escalating commitment to a chosen course of action. , 1976 .

[23]  Russell T. Osguthorpe,et al.  How Do Instructional-Design Practitioners Make Instructional-Strategy Decisions?. , 2008 .

[24]  D. Brethower.  Integrating theory, research, and practice in human performance technology: Examples from behavioral, cognitive, and constructivist theories , 2000 .

[25]  Danny G. Langdon.  Are We Doing What We Say , 1997 .

[26]  Tony O'Driscoll.  Learning from History: Chronicling the Emergence of Human Performance Technology. , 2003 .

[27]  James O. Berger,et al.  Bayesian Analysis: A Look at Today and Thoughts of Tomorrow , 2000 .

[28]  M. D. Dunnette,et al.  An alternative selection procedure: The low-fidelity simulation. , 1990 .

[29]  Mary Norris Thomas,et al.  Evidence-practice partnership , 2006 .

[30]  Gordon Rowland,et al.  HPT in Academic Curricula: Survey Results. , 2008 .

[31]  WHO IS THE “REAL” FATHER OF HPT? , 2007 .

[32]  N. Nguyen,et al.  Situational Judgment Tests: A Review of Practice and Constructs Assessed , 2001 .

[33]  Robert A. Reiser.  A history of instructional design and technology: Part II: A history of instructional design , 2001 .

[34]  Dustin J. Sleesman,et al.  CLEANING UP THE BIG MUDDY: A META-ANALYTIC REVIEW OF THE DETERMINANTS OF ESCALATION OF COMMITMENT , 2012 .

[35]  Harold D. Stolovitch,et al.  Report of 1999 ISPI symposium appropriate inquiry in human performance technology , 2000 .

[36]  Edgar Necochea,et al.  Measurement and evaluation in performance improvement in international reproductive health , 2003 .

[37]  Sharon J. Korth.  Planning HRD Interventions; What, Why, and How , 2008 .

[38]  R. Clark,et al.  Technology or Craft: What Are We Doing?. , 1998 .

[39]  Richard E. Clark,et al.  Re‐establishing performance improvement as a legitimate area of inquiry, activity, and contribution: Rules of the road , 1999 .

[40]  Ingrid Guerra.  Key Competencies Required of Performance Improvement Professionals , 2008 .

[41]  J. C. Flanagan.  Psychological Bulletin THE CRITICAL INCIDENT TECHNIQUE , 2022 .

[42]  Nancy Tippins,et al.  Further studies of the low‐fidelity simulation in the form of a situational inventory , 1993 .

[43]  Abbas Darabi.  Systems thinking and systematic methodology a semi-empirical experience in support of the ISPI value proposition , 2003 .

[44]  A. Greenwald.  Within-subjects designs: To use or not to use? , 1976 .

[45]  W. Hannum.  Training myths: False beliefs that limit the efficiency and effectiveness of training solutions, part 1 , 2009 .

[46]  Rob Foshay.  Science and Research Applications in Operational Environments. , 2000 .

[47]  Kent L. Gustafson,et al.  Paradigms in the theory and practice of education and training design , 2004 .

[48]  Roger Kaufman,et al.  A scientific dialogue: A performance accomplishment code of professional conduct , 2000 .