Dialogue Act Classification in Reference Interview Using Convolutional Neural Network with Byte Pair Encoding

Dialogue act classification is an important component of dialogue management, which captures the user’s intention and chooses the appropriate response action. In this paper, we focus on the dialogue act classification in reference interviews to model the behaviors of librarians in the information seeking dialogues. Reference interviews sometimes include rare words and phrases. Therefore, the existing approaches that use words as units of input often do not work well here. We used the byte pair encoding compression algorithm to build a new vocabulary for the inputs of the classifier. By using this new unit as a feature of the convolutional neural network-based classifier, we improved the accuracy of the dialogue act classification while suppressing the size of vocabulary.