Reliance, trust and heuristics in judgmental forecasting

A review in judgmental forecasting was conducted focusing in human judgment.Pervasive computer's advice disuse is moderated by complexity & system performance.Appropriate delivering of computer's advice in forecasting is still controversial.Forecasters usually overforecast but report to prefer underforecast.Heuristics most found were anchor and adjustment, trend damping and noise modelling. Judgmental forecasting gives light to the use of computers in human decision making. This paper reviews studies in judgmental forecasting focusing on what has been learned from human judgment and human-computer interaction. Available information was analyzed in the framework of three dimensions: reliance and trust on computer suggestions and heuristics employed by forecasters to produce forecasts. Results show that computer's advice disuse is pervasive in forecasting; and the disuse increases with higher task complexity and lower perceived system performance. Explanations and past performance are good candidates to increase trust in computer's advice, but the appropriate format to deliver this information is still controversial. Forecasters usually overforecast but report to prefer underforecast, which can lead to a cognitive dissonance and in turn to conflicting goals in the task. Heuristics research in time series forecasting indicates that forecasters heavily assess their own judgment, which in turn tend to be grounded in last outcomes and an overall evaluation of features like mean, trend and autocorrelation. It appears that heuristics not always lead to harmful biases for the forecast.

[1]  Mark W. Scerbo,et al.  Automation-induced complacency for monitoring highly reliable systems: the role of task complexity, system experience, and operator trust , 2007 .

[2]  Michael Lawrence,et al.  Judgmental Adjustments of Previously Adjusted Forecasts , 2008, Decis. Sci..

[3]  Michael Lawrence,et al.  Wisdom of group forecasts: Does role-playing play a role? , 2012 .

[4]  Robert Fildes,et al.  Influence of user participation on DSS use and decision accuracy , 2002 .

[5]  M. Osman Controlling uncertainty: a review of human behavior in complex dynamic environments. , 2010, Psychological bulletin.

[6]  Philip Hans Franses,et al.  Do statistical forecasting models for SKU-level data benefit from including past expert knowledge? , 2013 .

[7]  Mary E. Thomson,et al.  Effects of trend strength and direction on performance and consistency in judgmental exchange rate forecasting , 2013 .

[8]  Soo Young Rieh,et al.  Credibility: A multidisciplinary framework , 2007, Annu. Rev. Inf. Sci. Technol..

[9]  Robert Fildes,et al.  Restrictiveness and guidance in support systems , 2011 .

[10]  Michael J. Prietula,et al.  The Turing effect: the nature of trust in expert systems advice , 1997 .

[11]  Dietrich Manzey,et al.  Misuse of automated decision aids: Complacency, automation bias and the impact of training experience , 2008, Int. J. Hum. Comput. Stud..

[12]  Philip Hans Franses,et al.  Combining SKU-level sales forecasts from models and experts , 2009, Expert Syst. Appl..

[13]  Ardion Beldad,et al.  How shall I trust the faceless and the intangible? A literature review on the antecedents of online trust , 2010, Comput. Hum. Behav..

[14]  Paul Goodwin,et al.  Providing support for decisions based on time series information under conditions of asymmetric loss , 2005, Eur. J. Oper. Res..

[15]  Robert Fildes,et al.  Forecasting support systems: What we know, what we need to know , 2013 .

[16]  Michael Workman,et al.  Expert decision support system use, disuse, and misuse: a study using the theory of planned behavior , 2005, Comput. Hum. Behav..

[17]  Ji Young Lee,et al.  Does source matter? Examining source effects in online product reviews , 2012, Comput. Hum. Behav..

[18]  B. Kemal Büyükkurt,et al.  An Experimental Study of the Effectiveness of Three Debiasing Techniques , 1991 .

[19]  Linda R. Elliott,et al.  Managing workload in human-robot interaction: A review of empirical studies , 2010, Comput. Hum. Behav..

[20]  M. O'Connor,et al.  Judgemental adjustment of initial forecasts: Its effectiveness and biases , 1995 .

[21]  N. Epley,et al.  The Anchoring-and-Adjustment Heuristic , 2006, Psychological science.

[22]  Chris Roberts,et al.  Correlations Among Variables in Message and Messenger Credibility Scales , 2010 .

[23]  John T. Mentzer,et al.  Forecasting Technique Familiarity, Satisfaction, Usage, and Application , 1995 .

[24]  Christopher D Wickens,et al.  Effects of conflict alerting system reliability and task difficulty on pilots' conflict detection with cockpit display of traffic information , 2007, Ergonomics.

[25]  Nigel Harvey,et al.  Sensitivity to autocorrelation in judgmental time series forecasting , 2011 .

[26]  Linda G. Pierce,et al.  Predicting Misuse and Disuse of Combat Identification Systems , 2001 .

[27]  Patric R. Spence,et al.  How much Klout do you have ... A test of system generated cues on source credibility , 2013, Comput. Hum. Behav..

[28]  Konstantinos Nikolopoulos,et al.  The process of using a forecasting support system , 2007 .

[29]  S. Bonaccio,et al.  Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences , 2006 .

[30]  Donald R. Jones,et al.  Understanding and attenuating decision bias in the use of model advice and other relevant information , 2006, Decis. Support Syst..

[31]  Silvia Knobloch-Westerwick,et al.  News cues: Information scent and cognitive heuristics: Research Articles , 2007 .

[32]  J. Armstrong,et al.  Findings from Evidence-Based Forecasting: Methods for Reducing Forecast Error , 2006 .

[33]  Nada R. Sanders,et al.  The efficacy of using judgmental versus quantitative forecasting methods in practice , 2003 .

[34]  Peter A. O'Donnell,et al.  A note on an experimental study of DSS and forecasting exponential growth , 2008, Decis. Support Syst..

[35]  Na Li,et al.  An assessment of human-computer interaction research in management information systems: topics and methods , 2004, Comput. Hum. Behav..

[36]  Gareth R. Jones,et al.  The experience and evolution of trust: Implications for cooperation and teamwork , 1998 .

[37]  Matthew L. Jensen,et al.  Credibility of Anonymous Online Product Reviews: A Language Expectancy Perspective , 2013, J. Manag. Inf. Syst..

[38]  Raja Parasuraman,et al.  Humans and Automation: Use, Misuse, Disuse, Abuse , 1997, Hum. Factors.

[39]  Mary E. Thomson,et al.  The relative influence of advice from human experts and statistical methods on forecast adjustments , 2009 .

[40]  R. Fildes,et al.  Providing support for the use of analogies in demand forecasting tasks , 2007 .

[41]  Meredith Lawley,et al.  Factors influencing decision support system acceptance , 2013, Decis. Support Syst..

[42]  Jean-Michel Hoc,et al.  Towards a cognitive approach to human-machine cooperation in dynamic situations , 2001, Int. J. Hum. Comput. Stud..

[43]  Desmond Saunders-Newton,et al.  “But the Computer Said!” , 2001 .

[44]  Paul Goodwin,et al.  Expectations, use and judgmental adjustment of external financial and economic forecasts: an empirical investigation , 2009 .

[45]  A. Tversky,et al.  Judgment under Uncertainty , 1982 .

[46]  Poornima Madhavan,et al.  Effects of computer self-efficacy and system reliability on user interaction with decision support systems , 2010, Comput. Hum. Behav..

[47]  Alexis Tsoukiàs,et al.  From decision theory to decision aiding methodology , 2008, Eur. J. Oper. Res..

[48]  Robert C. Blattberg,et al.  Database Models And Managerial Intuition: 50% Model + 50% Manager , 1990 .

[49]  Spyros Makridakis,et al.  Forecasting and Planning: An Evaluation , 1981 .

[50]  Eric J. Johnson,et al.  The adaptive decision maker , 1993 .

[51]  Nigel Harvey,et al.  Effects of judges' forecasting on their later combination of forecasts for the same outcomes , 2004 .

[52]  M. O'Connor,et al.  Judgemental and statistical time series forecasting: a review of the literature , 1996 .

[53]  R. Fildes,et al.  Effective forecasting and judgmental adjustments: an empirical evaluation and strategies for improvement in supply-chain planning , 2009 .

[54]  Spyros Makridakis,et al.  Factors affecting judgmental forecasts and confidence intervals , 1989 .

[55]  John D. Lee,et al.  Review of a Pivotal Human Factors Article: “Humans and Automation: Use, Misuse, Disuse, Abuse” , 2008, Hum. Factors.

[56]  John D. Lee,et al.  Trust in Automation: Designing for Appropriate Reliance , 2004 .

[57]  Nadine B. Sarter,et al.  Supporting Trust Calibration and the Effective Use of Decision Aids by Presenting Dynamic System Confidence Information , 2006, Hum. Factors.

[58]  James J. Jiang,et al.  The Impact of Model Performance History Information on Users' Confidence in Decision Models: An Experimental Examination , 1996 .

[59]  Larry P. Ritzman,et al.  Bringing judgment into combination forecasts , 1995 .

[60]  Marcus O'Connor,et al.  The anchor and adjustment heuristic in time‐series forecasting , 1995 .

[61]  S. Shyam Sundar,et al.  Anthropomorphism of computers: Is it mindful or mindless? , 2012, Comput. Hum. Behav..

[62]  Sunil Gupta,et al.  Managerial judgment and forecast combination: An experimental study , 1994 .

[63]  H. Simon,et al.  A Behavioral Model of Rational Choice , 1955 .

[64]  Gerd Gigerenzer,et al.  Heuristic decision making. , 2011, Annual review of psychology.

[65]  Paul Goodwin,et al.  Do forecasts expressed as prediction intervals improve production planning decisions? , 2010, Eur. J. Oper. Res..

[66]  John T. Mentzer,et al.  Forecasting Task-technology Fit: The Influence of Individuals, Systems and Procedures on Forecast Performance , 2010 .

[67]  J. H. Davis,et al.  An integrative model of organizational trust, Academy of Management Review, : . , 1995 .

[68]  Adamantios Diamantopoulos,et al.  Managerial evaluation of sales forecasting effectiveness: A MIMIC modeling approach , 2002 .

[69]  Robert Fildes,et al.  The design features of forecasting support systems and their effectiveness , 2006, Decis. Support Syst..

[70]  Marcus O'Connor,et al.  Forecasting support systems for the incorporation of event information: An empirical investigation , 2005 .

[71]  Robert Ramberg,et al.  People's Perception of Human and Computer Advice. , 1996 .

[72]  Abdul V. Roudsari,et al.  Automation bias: a systematic review of frequency, effect mediators, and mitigators , 2012, J. Am. Medical Informatics Assoc..

[73]  Albert L. Lederer,et al.  Individual Differences and Decision-making Using Various Levels of Aggregation of Information , 1989, J. Manag. Inf. Syst..

[74]  R. H. Edmundson Decomposition; a strategy for judgemental forecasting , 1990 .

[75]  D. Wiegmann,et al.  Similarities and differences between human–human and human–automation trust: an integrative review , 2007 .

[76]  Fred Collopy,et al.  The role and validity of judgment in forecasting , 1996 .

[77]  Peter M. Todd,et al.  How much information do we need? , 2007, Eur. J. Oper. Res..

[78]  Spyros Makridakis,et al.  The M3-Competition: results, conclusions and implications , 2000 .

[79]  N. Sanders,et al.  Forecasting Practices in US Corporations: Survey Results , 1994 .

[80]  Nigel Harvey,et al.  Use of heuristics: Insights from forecasting research , 2007, Thinking & Reasoning.

[81]  P. Goodwin,et al.  Judgmental forecasting: A review of progress over the last 25 years , 2006 .

[82]  Nigel Harvey,et al.  Why Are Judgments Less Consistent in Less Predictable Task Situations , 1995 .

[83]  Robert Fildes,et al.  Judgmental forecasts of time series affected by special events: does providing a statistical forecast improve accuracy? , 1999 .

[84]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[85]  Chanthika Pornpitakpan The Persuasiveness of Source Credibility: A Critical Review of Five Decades' Evidence , 2004 .

[86]  Fredric M. Wolf,et al.  Are Clinicians Correct When They Believe They are Correct? Implications for Medical Decision Support , 2001, MedInfo.

[87]  Nirit Yuviler-Gavish,et al.  Effect of Descriptive Information and Experience on Automation Reliance , 2011, Hum. Factors.

[88]  F. Bolger,et al.  Graphs versus tables: Effects of data presentation format on judgemental forecasting , 1996 .

[89]  Joa Sang Lim,et al.  Judgmental forecasting with time series and causal information , 1996 .

[90]  Anthony J. Jakeman,et al.  Environmental decision support systems (EDSS) development - Challenges and best practices , 2011, Environ. Model. Softw..

[91]  David W. Wilson,et al.  A Picture is Worth a Thousand Words: Source Credibility Theory Applied to Logo and Website Design for Heightened Credibility and Consumer Trust , 2014, Int. J. Hum. Comput. Interact..

[92]  J. Tukey,et al.  Transformations Related to the Angular and the Square Root , 1950 .

[93]  R. Fildes,et al.  Analysis of judgmental adjustments in the presence of promotions , 2013 .

[94]  R. Fildes,et al.  The effects of integrating management judgement into intermittent demand forecasts , 2009 .

[95]  Paul Goodwin,et al.  Antecedents and effects of trust in forecasting advice , 2013 .

[96]  David Arnott,et al.  Cognitive biases and decision support systems development: a design science approach , 2006, Inf. Syst. J..

[97]  S. Shyam Sundar,et al.  Authority vs. peer: how interface cues influence users , 2009, CHI Extended Abstracts.

[98]  Michael Lawrence,et al.  A field study of sales forecasting accuracy and processes , 2000, Eur. J. Oper. Res..

[99]  S. Bonaccio,et al.  What types of advice do decision-makers prefer? , 2010 .

[100]  Clifford Nass,et al.  Source Orientation in Human-Computer Interaction , 2000, Commun. Res..

[101]  Marcus O'Connor,et al.  Judgmental forecasting in the presence of loss functions , 2005 .

[102]  Michael Lawrence,et al.  The effects of structural characteristics of explanations on use of a DSS , 2006, Decis. Support Syst..

[103]  J. H. Davis,et al.  An Integrative Model Of Organizational Trust , 1995 .

[104]  Daniel M. Oppenheimer,et al.  Heuristics made easy: an effort-reduction framework. , 2008, Psychological bulletin.

[105]  S. Shyam Sundar,et al.  News cues: Information scent and cognitive heuristics , 2007, J. Assoc. Inf. Sci. Technol..

[106]  Kathleen L. Mosier,et al.  Accountability and automation bias , 2000, Int. J. Hum. Comput. Stud..

[107]  George M. Kasper,et al.  The Effect of Human-Computer Interchange Protocol on Decision Performance , 1989, J. Manag. Inf. Syst..

[108]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.