Analysis and optimization of question answering systems

!oueSllon Answering (OA) syslems are 'ppHcaHons that search Info<matlon on large cOlleeHons of documenls, whIch are enhanced with nalurat ,language processing capabilities. This additional processing adds an important cost 10 the system,and thus it Is necessary to find techniques that improve the time performance of the system. This thesis is focused on how to improve the performance of OA systems. in either a single computer or in 'distributed systems, proposing new generic techniques related to the implementation of in memory caches that can take advantage of previous 'computations. ! IThe fi rst contribution of this thesis is the proposal of multilayer-caches. This type of cache are able to sha re the available memory of one computer in different parts, depending on the distribution of the queries and the computa tional cos t of each block. Our results show that the use of mulllple layers for iOA provide an speedup of up to 1.25 over systems with a single layer. lOur second contribution is the proposal of the Evolutive Summary Counters (ESC) data structure. The ESC aim at storing the access frequency to documents in each node of a distributed system during a recent window of time. The ESC are implemented as a list of count bloom filters. which are ~ery efficient to build summaries (ESC-summaries) that can be broadcasted among the nodes in the distributed system In order to provide a global view of the system. I ~he third main contribution is the proposal of an algorithm that places the data in the coopera tive cache, which we call ESC-placemenl. This strategy distributes the conlents of the cache according to the ESC-summaries taking into accoun tlhe following objectives: keep a local copy of the documents \that are very frequently accessed in the cache of the node that is accessing them, keep a copy of the frequent documents in some node of the 'distributed system, and avoid that the redis tribution of the contents generate an excessive number of message chaining. lour fourth contribution is the proposal of ESC·search, which is an algorithm that decides if a documenl is avaitabla in the nel\vork and in which node. Since the larger the access frequency, the more prObable that the document is available in the cache of a compu ter, ESC·search estimates the Iprobability that a document is available for a given access frequency indicated by the ESC-surnmaries. According to previous searches. ESC·search :updales this probability: ESC-search increases it if the document is found in the remote cache, and otherwise it is decreased. Using these probabilities, ESC-search decides dynamically which are the nodes that are more probably sloring a certain cache content, and how many nodes are queried in order Ito reduce the risk that a document is available In the cooperative cache is found but it is not retrieved because the wrong nodes are queried. Our results ;ShOW that for typical search engine distributions, ESC-search retrieves 98% of the cache contents but only queries 14% of the available nodes. Finally, we propose two techniques that balanco the load of a distributed system with a cooperative cache. Caches reduce the cost to compute a query, but this may imbalance the overall system load if the hi l rale is large. Probability cost is a strategy based on costs, that takes the probabilities obtained by ESC-search, and reestimates the cost of query considering the probability of having the data already cached. The second technique. Affini ty. adds a Inew term that measures the similarity among the cache contents in one node and the query. Then, Affinity replica tes the documents that are frequent in Ithe cooperative cache, 10 improve the data access locality and reduce the overa ll computation cost. . ~he complete analysis of our techniques achieves a performance of over 6 q/s (queries per second) with 16 computers. We obtain a superlinear speedup and a t\'.'O order of magnitude improvement with respect to the original system in one computer which was able 10 answer 0.05 q/s. Lloc i