Design and validation of precooked developer dashboards

Despite increasing popularity of developer dashboards, the effectiveness of dashboards is still in question. In order to design a dashboard that is effective and useful for developers, it is important to know (a) what information developers need to see in a dashboard, and (b) how developers want to use a dashboard with that necessary information. To answer these questions, we conducted two series of face-to-face individual interviews with developers. In the first step we analyzed answers, build a Goal-Question-Metric model and designed a precooked developer dashboard. Then, during the second separate series of interviews, we validated the GQM and derived feedback on the designed dashboard. Given that the cost of dashboard customization prevents developers from utilizing dashboards, we believe that our findings can provide a solid starting point to build precooked developer dashboards that can be readily utilized by software companies.

[1]  Oana Velcu,et al.  A review of dashboards in performance management: Implications for design and research , 2012, Int. J. Account. Inf. Syst..

[2]  Riaz Khan,et al.  Does Self-Selection Affect Samples’ Representativeness in Online Surveys? An Investigation in Online Video Game Research , 2014, Journal of medical Internet research.

[3]  Wilhelm Meding.  Effective monitoring of progress of agile software development teams in modern software companies: an industrial case study , 2017, IWSM-Mensura.

[4]  Giancarlo Succi,et al.  Lean Software Development in Action , 2014, Springer Berlin Heidelberg.

[5]  Ginger Levin,et al.  Validation of a generic GQM based measurement framework for software projects from industry practitioners , 2011, IEEE 10th International Conference on Cognitive Informatics and Cognitive Computing (ICCI-CC'11).

[6]  Dieter Hoffmann,et al.  Online, face-to-face and telephone surveys—Comparing different sampling methods in wine consumer research , 2013 .

[7]  Arie van Deursen,et al.  Software metrics: Pitfalls and best practices , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[8]  P. Rosenbaum.  Design of Observational Studies , 2009, Springer Series in Statistics.

[9]  H. D. Rombach,et al.  The Goal Question Metric Approach , 1994 .

[10]  Christoph Treude,et al.  Awareness 2.0: staying aware of projects, developers and tasks using dashboards and feeds , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[11]  Jan Bosch,et al.  Toward Evidence-Based Organizations: Lessons from Embedded Systems, Online Games, and the Internet of Things , 2017, IEEE Software.

[12]  Michael W. Godfrey,et al.  Developer Dashboards: The Need for Qualitative Analytics , 2013, IEEE Software.

[13]  Patrik Berander,et al.  A goal question metric based approach for efficient measurement framework definition , 2006, ISESE '06.

[14]  Jon A. Krosnick,et al.  Answering Questions: A Comparison of Survey Satisficing and Mindlessness , 2014 .

[15]  Giancarlo Succi,et al.  What do software engineers care about? gaps between research and practice , 2017, ESEC/SIGSOFT FSE.

[16]  Scott B. MacKenzie,et al.  Common method biases in behavioral research: a critical review of the literature and recommended remedies. , 2003, The Journal of applied psychology.

[17]  M. R. Espejo.  Applying the Rasch Model: Fundamental Measurement in the Human Sciences , 2004 .

[18]  Alberto Sillitti,et al.  Managing non-invasive measurement tools , 2006, J. Syst. Archit..

[19]  V. Basili.  Software modeling and measurement: the Goal/Question/Metric paradigm , 1992 .

[20]  Victor R. Basili,et al.  A Methodology for Collecting Valid Software Engineering Data , 1984, IEEE Transactions on Software Engineering.

[21]  Mary Czerwinski,et al.  FASTDash: a visual dashboard for fostering awareness in software teams , 2007, CHI.

[22]  A. Furnham.  Response bias, social desirability and dissimulation , 1986 .

[23]  Mary Czerwinski,et al.  WIPDash: Work Item and People Dashboard for Software Development Teams , 2009, INTERACT.

[24]  Rainer Koschke,et al.  On the Comprehension of Program Comprehension , 2014, TSEM.