Understanding the Role of Reporting in Work Item Tracking Systems for Software Development: An Industrial Case Study

Work item tracking systems such as Visual Studio Team Services, JIRA, BugZilla and GitHub issue tracker are widely used by software engineers. These systems are used to track work items such as features, user stories, bugs, plan sprints, distribute tasks across the team and prioritize the team's work. Such systems can help teams track the progress and manage the shipping of software. While these tracking systems give data about different work items in tabular format, using a reporting tool on top of them can help teams visualize the data related to their projects such as how many bugs are open and closed and which work items are assigned to a team member. While tools like Visual Studio and JIRA provide reporting services, it is important to understand how users leverage them in their projects to help improve the reporting services. In this study, we conduct an empirical investigation on the usage of Analytics Service - a reporting service provided by Visual Studio Team Services (VSTS) to build dashboards and reports out of their work item tracking data. In particular, we want to understand why and how users interact with Analytics Service and what are the outcomes and business decisions taken by stakeholders from reports built using Analytics Service. We perform semi-structured interviews and survey with users of Analytics Service to understand usage and challenges. Our report on qualitative and quantitative analysis can help organizations and engineers building similar tools or services.