Bringing Why-QA to Web Search

We investigated to what extent users could be satisfied by a web search engine for answering causal questions. We used an assessment environment in which a web search interface was simulated. For 1 401 why-queries from a search engine log we pre-retrieved the first 10 results using Bing. 311 queries were assessed by human judges. We found that even without clicking a result, 25.2% of the why-questions is answered on the first result page. If we count an intended click on a result as a vote for relevance, then 74.4% of the why-questions gets at least one relevant answer in the top-10. 10% of why-queries asked to web search engines are not answerable according to human assessors.