Which Channel to Ask My Question?: Personalized Customer Service Request Stream Routing Using Deep Reinforcement Learning

Customer services are critical to all companies, as they may directly connect to the brand reputation. Due to a great number of customers, e-commerce companies often employ multiple communication channels to answer customers’ questions, for example, Chatbot and Hotline. On one hand, each channel has limited capacity to respond to customers’ requests; on the other hand, customers have different preferences over these channels. The current production systems are mainly built based on business rules that merely consider the tradeoffs between the resources and customers’ satisfaction. To achieve the optimal tradeoff between the resources and customers’ satisfaction, we propose a new framework based on deep reinforcement learning that directly takes both resources and user model into account. In addition to the framework, we also propose a new deep-reinforcement-learning-based routing method—double dueling deep Q-learning with prioritized experience replay ( $\mathsf{PER-DoDDQN}$ ). We evaluate our proposed framework and method using both synthetic and a real customer service log data from a large financial technology company. We show that our proposed deep-reinforcement-learning-based framework is superior to the existing production system. Moreover, we also show that our proposed $\mathsf{PER-DoDDQN}$ is better than all other deep Q-learning variants in practice, which provides a more optimal routing plan. These observations suggest that our proposed method can seek the tradeoff, where both channel resources and customers’ satisfaction are optimal.