Sequence Pre-processing : Focusing Analysis of Log Event Data

Many computational systems are generating log event data as a way to help developers understand the usage of applications in the wild. While many commercial analysis tools exist, they tend to treat log event data as a “bag of events” instead of collections of observed sequences, where each sequence represents an individual session. While recent work can support the visual analysis of event sequence data, log files tend to contain complexity in scale and noise that can foul downstream analyses. In this work, we identify common recurring problems of noise that arise from the analysis of this data, and assert that methods for preprocessing can be a valuable tool to both focus data for downstream analysis and provide provenance support for visual analytics tools. These pre-processing methods can be performed interactively and in conjunction with analysis tools to iteratively refine rules to streamline visual analysis. Through several case studies, we identify the common sources of noise in log files and demonstrate how our proposed pre-processing methods can help to minimize excess data reaching downstream analysis tools.