An approximate dynamic programming method for the multi-period technician scheduling problem with experience-based service times and stochastic customers

Abstract In this paper, we study how an organization can recognize that individuals learn when assigning employees to tasks. By doing so, an organization can meet current demands and position the capabilities of their workforce for the yet unknown demands in future days. Specifically, we study a variant of the technician and task scheduling problem in which the tasks to be performed in the current day are known, but there is uncertainty regarding the tasks to be performed in subsequent days. To solve this problem, we present an Approximate Dynamic Programming-based approach that incorporates into daily assignment decisions estimates of the long-term benefits associated with experience accumulation. We benchmark this approach against an approach that only considers the impact of experience accumulation on just the next day's productivity and show that the ADP approach outperforms this one-step lookahead approach. Finally, based on the results from an extensive computational study we derive insights into how an organization can schedule their employees in a manner that enables meeting both near and long-term demands.

[1]  Avishai Mandelbaum,et al.  Statistical Analysis of a Telephone Call Center , 2005 .

[2]  Xi Chen,et al.  The technician routing problem with experience-based service times , 2016 .

[3]  Mohamad Y. Jaber,et al.  Learning and forgetting models and their applications , 2013 .

[4]  Scott E. Grasman,et al.  Integer programming techniques for solving non-linear workforce planning models with learning , 2015, Eur. J. Oper. Res..

[5]  Warrren B Powell,et al.  An Adaptive, Distribution-Free Algorithm for the Newsvendor Problem with Censored Demands, with Applications to Inventory and Distribution , 2001 .

[6]  Matthew S. Maxwell,et al.  Approximate Dynamic Programming for Ambulance Redeployment , 2010, INFORMS J. Comput..

[7]  Adam Dubrowski,et al.  Teaching Surgical Skills: What Kind of Practice Makes Perfect?: A Randomized, Controlled Trial , 2006, Annals of surgery.

[8]  David A. Nembhard,et al.  Selection, grouping, and assignment policies with learning-by-doing and knowledge transfer , 2015, Comput. Ind. Eng..

[9]  Itzhak Venezia.  On the statistical origins of the learning curve , 1985 .

[10]  John N. Tsitsiklis,et al.  Regression methods for pricing complex American-style options , 2001, IEEE Trans. Neural Networks.

[11]  G Laporte,et al.  An emergency vehicle dispatching system for an electric utility in Chile , 1999, J. Oper. Res. Soc..

[12]  G. Hendrickson,et al.  Transfer of training in learning to hit a submerged target. , 1941 .

[13]  Dimitri J. Papageorgiou,et al.  Approximate Dynamic Programming for a Class of Long-Horizon Maritime Inventory Routing Problems , 2015, Transp. Sci..

[14]  Xi Chen,et al.  Multi-period technician scheduling with experience-based service times and stochastic customers , 2017, Comput. Oper. Res..

[15]  P. Scardino,et al.  Fellowship Training as a Modifier of the Surgical Learning Curve , 2010, Academic medicine : journal of the Association of American Medical Colleges.

[16]  Warren B. Powell,et al.  Dynamic-Programming Approximations for Stochastic Time-Staged Integer Multicommodity-Flow Problems , 2006, INFORMS J. Comput..

[17]  Jordi Olivella.  An Experiment on Task Performance Forecasting Based on the Experience of Different Tasks , 2007 .

[18]  David Lesaint,et al.  Dynamic Workforce Scheduling for British Telecommunications plc , 2000, Interfaces.

[19]  Barrett W. Thomas,et al.  Balancing flexibility and inventory in workforce planning with learning , 2017 .

[20]  Martin W. P. Savelsbergh,et al.  Dynamic Programming Approximations for a Stochastic Inventory Routing Problem , 2004, Transp. Sci..

[21]  Warren B. Powell,et al.  “Approximate dynamic programming: Solving the curses of dimensionality” by Warren B. Powell , 2007, Wiley Series in Probability and Statistics.

[22]  Warren B. Powell,et al.  An Adaptive Dynamic Programming Algorithm for Dynamic Fleet Management, I: Single Period Travel Times , 2002, Transp. Sci..

[23]  T. P. Wright,et al.  Factors affecting the cost of airplanes , 1936 .

[24]  David A. Nembhard,et al.  Learning and forgetting-based worker selection for tasks of varying complexity , 2005, J. Oper. Res. Soc..

[25]  John W. Fowler,et al.  Modelling inherent worker differences for workforce planning , 2007 .

[26]  Barrett W. Thomas,et al.  Restocking-Based Rollout Policies for the Vehicle Routing Problem with Stochastic Demand and Duration Limits , 2016, Transp. Sci..

[27]  D A Nembhard,et al.  Heuristic approach for assigning workers to tasks based on individual learning rates , 2001 .

[28]  Per Medbo,et al.  Simulating operator learning during production ramp-up in parallel vs. serial flow production , 2017, Int. J. Prod. Res..

[29]  Christian Terwiesch,et al.  Operations Management , 2019 .

[30]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[31]  Benjamin Van Roy,et al.  Solitaire: Man Versus Machine , 2004, NIPS.

[32]  Dan Zhang,et al.  An Approximate Dynamic Programming Approach to Network Revenue Management with Customer Choice , 2009, Transp. Sci..

[33]  Ezey M. Dar-Ei,et al.  Human learning : from learning curves to learning organizations , 2000 .

[34]  Flávio Sanson Fogliatto,et al.  Learning curve models and applications: Literature review and research directions , 2011 .

[35]  Mohamad Y. Jaber,et al.  A numerical comparison of three potential learning and forgetting models , 2004 .

[36]  Pierre Dejax,et al.  Multiperiod Planning and Routing on a Rolling Horizon for Field Force Optimization Logistics , 2008 .

[37]  Warren B. Powell,et al.  Approximate dynamic programming algorithms for optimal dosage decisions in controlled ovarian hyperstimulation , 2012, Eur. J. Oper. Res..

[38]  Christelle Guéret,et al.  On the dynamic technician routing and scheduling problem , 2012 .

[39]  Daniel Adelman,et al.  Dynamic Bid Prices in Revenue Management , 2007, Oper. Res..