Links with Answers: Query Answering for Customer Support

The increasing complexity of today’s systems has triggered a tremendous demand for customer support [2]. Customers often ask about standard procedures or common issues that are well documented somewhere so that their questions could be answered efficiently in an automated fashion. But current solutions often fail in addressing their problems – we assume that every reader has enough experience with unhelpful chat bots, automated phone systems, and standardized e-mail replies in order to see that there is room for improvement. One main challenge for automated solutions is to understand (enough of) the customer question to either link it to the answer, if it is known by the system, or forward it to special, e.g., human support. Natural language understanding is not easy in general and an entire area of research by itself. However, in the context of a concrete system, the most relevant vocabulary can be narrowed down to the one of the documents containing the answers and its specific semantics are known. For example, the user types and relevant tasks and tools are usually fixed and thus could be described semantically and applied for answering the queries. In this paper, we focus on a set of natural language questions asking for IT support from within IBM. Each question is associated with an URL, the link to a document containing the answer. These documents are available in HTML but, alas, not annotated in the form of Semantic Web (SW) pages; we assume that this is a scenario which is common, also outside of IBM. In this initial investigation, we show that ontologies and reasoning using SW technologies can help in resolving problems with underspecification, ambiguity, and variety in the questions, and in finding answers which are missed by standard information retrieval (IR) approaches. We compare a standard TF-IDF-based approach to a semantic solution based on a custom ontology, the state-of-the-art rule learning system AMIE [3], and SWRL reasoning, using the learned rules. Although the benefits of including semantic knowledge become evident, we point out several issues with and gaps in the current tool set; challenges for future development of semantic technologies. In this sense, we propose a semantic baseline to motivate research in an area of rising importance where, in our opinion, SW technologies could provide fundamental benefits.