Learning Curves of Agents with Diverse Skills in Information Technology-Enabled Physician Referral Systems

To improve operational efficiencies while providing state of the art healthcare services, hospitals rely on information technology enabled physician referral systems (IT-PRS). This study examines learning curves in an IT-PRS setting to determine whether agents achieve performance improvements from cumulative experience at different rates and how information technologies transform the learning dynamics in this setting. We present a hierarchical Bayes model that accounts for different agent skills (domain and system) and estimate learning rates for three types of referral requests: emergency (EM), nonemergency (NE), and nonemergency out of network (NO). Furthermore, the model accounts for learning spillovers among the three referral request types and the impact of system upgrade on learning rates. We estimate this model using data from more than 80,000 referral requests to a large IT-PRS. We find that: (1) The IT-PRS exhibits a learning rate of 4.5% for EM referrals, 7.2% for NE referrals, and 12.3% for NO referrals. This is slower than the learning rate of manufacturing (on average 20%) and more comparable to other service settings (on average, 8%). (2) Domain and system experts are found to exhibit significantly different learning behaviors. (3) Significant and varying learning spillovers among the three referral request types are also observed. (4) The performance of domain experts is affected more adversely in comparison to system experts immediately after system upgrade. (5) Finally, the learning rate change subsequent to system upgrade is also higher for system experts in comparison to domain experts. Overall, system upgrades are found to have a long-term positive impact on the performance of all agents. This study contributes to the development of theoretically grounded understanding of learning behaviors of domain and system experts in an IT-enabled critical healthcare service setting.