Does Similarity Matter? The Case of Answer Extraction from Technical Discussion Forums

Extracting question‐answer pairs from social media discussions has garnered much attention in recent times. Several methods have been proposed in the past that pose this task as a post or sentence classification problem, which label each entry as an answer or not. This paper makes the first attempt at the following two‐fold objectives: (a) In all classification based approaches towards this direction, one of the foremost signals used to identify answers is their similarity to the question. We study the contribution of content similarity specifically in the context of technical problem‐solving domain. (b) We introduce hitherto unexplored features that aid in high‐precision extraction of answers, and present a thorough study of the contribution of all features to this task. Our results show that, it is possible to extract answers using these features with high accuracy, when their similarity to the question is unreliable.