Prediction in HRM research–A gap between rhetoric and reality

Funding information WOA Institution: OTTO VON GUERICKE UNIVERSITAET MAGDEBURG Blended DEAL: Projekt DEAL. Abstract There are broadly two dimensions on which researchers can evaluate their statistical models: explanatory power and predictive power. Using data on job satisfaction in ageing workforces, we empirically highlight the importance of distinguishing between these two dimensions clearly by showing that a model with a certain degree of explanatory power can produce vastly different levels of predictive power and vice versa—in the same and different contexts. In a further step, we review all the papers published in three top‐tier human resource management journals between 2014 and 2018 to show that researchers generally confuse explanation and prediction. Specifically, while almost all authors rely solely on explanatory power assessments (i.e., assessing whether the coefficients are significant and in the hypothesised direction), they also derive practical recommendations, which inherently result from a predictive scenario. Based on our results, we provide HRM researchers recommendations on how to improve the rigour of their explanatory studies.

[1]  M. Edwards,et al.  Multi‐foci CSR perceptions, procedural justice and in‐role employee performance: the mediating role of commitment and pride , 2017 .

[2]  M. Triana,et al.  Responses to Discrimination: Relationships Between Social Support Seeking, Core Self-Evaluations, and Withdrawal Behaviors , 2015 .

[3]  Malcolm R. Forster,et al.  Predictive Accuracy as an Achievable Goal of Science , 2002, Philosophy of Science.

[4]  N. Sugiura Further analysts of the data by akaike' s information criterion and the finite corrections , 1978 .

[5]  G. Kitagawa,et al.  Information Criteria and Statistical Modeling , 2007 .

[6]  Mohak Shah,et al.  Evaluating Learning Algorithms: A Classification Perspective , 2011 .

[7]  B. Heijden,et al.  Multisource Ratings Of Managerial Competencies And Their Predictive Value For Managerial And Organizational Effectiveness , 2014 .

[8]  Hong Deng,et al.  Spillover Effects of Emotional Labor in Customer Service Encounters Toward Coworker Harming: A Resource Depletion Perspective , 2017 .

[9]  Sylvain Arlot,et al.  A survey of cross-validation procedures for model selection , 2009, 0907.4728.

[10]  Heungsun Hwang,et al.  A concept analysis of methodological research on composite-based structural equation modeling: bridging PLSPM and GSCA , 2019, Behaviormetrika.

[11]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[12]  Aki Vehtari,et al.  Comparison of Bayesian predictive methods for model selection , 2015, Stat. Comput..

[13]  Max Kuhn,et al.  caret: Classification and Regression Training , 2015 .

[14]  Rüdiger Kabst,et al.  The Effectiveness of Recruitment Advertisements and Recruitment Websites: Indirect and Interactive Effects on Applicant Attraction , 2014 .

[15]  Alejandro Bello-Pintado,et al.  Bundles of HRM practices and performance: empirical evidence from a Latin American context , 2015 .

[16]  J. Briscoe,et al.  Are freelancers a breed apart? The role of protean and boundaryless career attitudes in employability and career success , 2018 .

[17]  John Geweke,et al.  Estimating regression models of finite but unknown order , 1981 .

[18]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[19]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[20]  Robert M. Scheller,et al.  Models in Ecosystem Science , 2004, Landscape Ecology.

[21]  Nathan P. Podsakoff,et al.  On the Limitations of Using Situational Judgment Tests to Measure Interpersonal Skills: The Moderating Influence of Employee Anger , 2014 .

[22]  J. Swart,et al.  Why should I share my knowledge? A multiple foci of commitment perspective , 2014 .

[23]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[24]  Bruce A. Rayton,et al.  The Costs of Exhibiting Organizational Citizenship Behavior , 2017 .

[25]  Olli-Pekka Kauppila,et al.  When and How Does LMX Differentiation Influence Followers’ Work Outcomes? The Interactive Roles of One's own LMX Status and Organizational Context , 2016 .

[26]  D. Ruppert The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[27]  E. Ng,et al.  The glass ceiling in context: the influence of CEO gender, recruitment practices and firm internationalisation on the representation of women in management , 2017 .

[28]  J. Farh,et al.  Enhancing Cultural Intelligence: The Roles of Implicit Culture Beliefs and Adjustment , 2017 .

[29]  Pratyush Nidhi Sharma,et al.  Prediction-Oriented Model Selection in Partial Least Squares Path Modeling , 2018, Decis. Sci..

[30]  Heungjun Jung,et al.  Relative deprivation of temporary agency workers in the public sector: The role of public service motivation and the possibility of standard employment , 2018 .

[31]  S. Beijer,et al.  The role of employee HR attributions in the relationship between high‐performance work systems and employee outcomes , 2015 .

[32]  D. Shepherd,et al.  Theory Building , 2017 .

[33]  J. Pearl Causality: Models, Reasoning and Inference , 2000 .

[34]  Aki Vehtari,et al.  A survey of Bayesian predictive methods for model assessment, selection and comparison , 2012 .

[35]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[36]  Y. Takane,et al.  Generalized structured component analysis , 2004 .

[37]  Edward E. Rigdon,et al.  Rethinking Partial Least Squares Path Modeling: In Praise of Simple Methods , 2012 .

[38]  Ken Aho,et al.  Model selection for ecologists: the worldviews of AIC and BIC. , 2014, Ecology.

[39]  Rob J. Hyndman,et al.  Another Look at Forecast Accuracy Metrics for Intermittent Demand , 2006 .

[40]  P. Teague,et al.  Recessionary Bundles: HR Practices in the Irish Economic Crisis , 2014 .

[41]  Daniel A. Newman,et al.  Narcissism and Leadership: A Meta-Analytic Review of Linear and Nonlinear Relationships , 2015 .

[42]  P. Holland Statistics and Causal Inference , 1985 .

[43]  Richard P. Bagozzi,et al.  Specification, evaluation, and interpretation of structural equation models , 2012 .

[44]  Rob J Hyndman,et al.  Automatic Time Series Forecasting: The forecast Package for R , 2008 .

[45]  Fotios Petropoulos,et al.  forecast: Forecasting functions for time series and linear models , 2018 .

[46]  Soumya Ray,et al.  Domain-Specific Language for Building and Estimating Structural Equation Models [R package seminr version 1.1.0] , 2020 .

[47]  International Social Survey Programme: ISSP 2015 - Work Orientations IV; Questionnaire Development , 2017 .

[48]  Galit Shmueli,et al.  The elephant in the room: Predictive performance of PLS models , 2016 .

[49]  T. Chai,et al.  Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature , 2014 .

[50]  Malcolm R. Forster,et al.  How to Tell When Simpler, More Unified, or Less Ad Hoc Theories will Provide More Accurate Predictions , 1994, The British Journal for the Philosophy of Science.

[51]  Edgar Erdfelder,et al.  GPOWER: A general power analysis program , 1996 .

[52]  Andrew B. Speer Quantifying with words: An investigation of the validity of narrative‐derived performance scores , 2018 .

[53]  Jakob Lauring,et al.  Can leadership compensate for deficient inclusiveness in global virtual teams , 2018 .

[54]  Galit Shmueli,et al.  To Explain or To Predict? , 2010, 1101.0891.

[55]  J. Mackie,et al.  The Conduct of Inquiry: Methodology for Behavioural Science , 1965 .

[56]  Margarita Mayo,et al.  The Lure of Work-Life Benefits: Perceived Person-Organization Fit As A Mechanism Explaining Job Seeker Attraction To Organizations , 2017 .

[57]  Marko Sarstedt,et al.  Rethinking some of the rethinking of partial least squares , 2019, European Journal of Marketing.

[58]  Rob J Hyndman,et al.  Forecasting Functions for Time Series and Linear Models [R package forecast version 8.13] , 2020 .

[59]  P. Burman A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methods , 1989 .

[60]  Christy Glass,et al.  Do women leaders promote positive change? Analyzing the effect of gender on business practices and diversity initiatives , 2018 .

[61]  Max Kuhn,et al.  Building Predictive Models in R Using the caret Package , 2008 .

[62]  Pratyush Nidhi Sharma,et al.  PLS-Based Model Selection: The Role of Alternative Explanations in Information Systems Research , 2019, J. Assoc. Inf. Syst..

[63]  Heungsun Hwang,et al.  Out-of-bag Prediction Error: A Cross Validation Index for Generalized Structured Component Analysis , 2019, Multivariate behavioral research.

[64]  Jenna R. Pieper Uncovering the Nuances of Referral Hiring: How Referrer Characteristics Affect Referral Hires’ Performance and Likelihood of Voluntary Turnover , 2015 .

[65]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[66]  Sven Hauff,et al.  Job satisfaction in aging workforces: an analysis of the USA, Japan and Germany , 2015 .

[67]  G. Box Science and Statistics , 1976 .

[68]  Hui Wang,et al.  Organizational Inducements and Employee Citizenship Behavior: The Mediating Role of Perceived Insider Status and the Moderating Role of Collectivism , 2015 .

[69]  K. Sanders,et al.  Performance‐based rewards and innovative behaviors , 2018 .

[70]  S. Gudergan,et al.  Partial least squares structural equation modeling in HRM research , 2020 .

[71]  Shirley Gregor,et al.  The Nature of Theory in Information Systems , 2006, MIS Q..

[72]  José Antonio Lozano,et al.  Sensitivity Analysis of k-Fold Cross Validation in Prediction Error Estimation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[73]  A. McQuarrie,et al.  Regression and Time Series Model Selection , 1998 .

[74]  David L. Dowe,et al.  Bayes not Bust! Why Simplicity is no Problem for Bayesians1 , 2007, The British Journal for the Philosophy of Science.

[75]  ShmueliGalit,et al.  Predictive analytics in information systems research , 2011 .

[76]  Spyros Makridakis,et al.  Forecasting and uncertainty in the economic and business world , 2009 .

[77]  Jake M. Hofman,et al.  Prediction and explanation in social systems , 2017, Science.

[78]  Pratyush Nidhi Sharma,et al.  Model selection uncertainty and multimodel inference in partial least squares structural equation modeling (PLS-SEM) , 2020 .

[79]  Joseph F. Hair,et al.  Explanation Plus Prediction—The Logical Focus of Project Management Research , 2021 .

[80]  José Manuel Benítez,et al.  On the use of cross-validation for time series predictor evaluation , 2012, Inf. Sci..

[81]  Galit Shmueli,et al.  The Elephant in the Room: Evaluating the Predictive Performance of Partial Least Squares (PLS) Path Models , 2015 .

[82]  Leisa D. Sargent,et al.  Linking Developmental Experiences to Leader Effectiveness and Promotability: The Mediating Role of Leadership Self‐Efficacy and Mentor Network , 2017 .