The Medium of Visualization for Software Comprehension

Although abundant studies have shown how visualization can help software developers to understand software systems, visualization is still not a common practice since developers (i) have little support to find a proper visualization for their needs, and once they find a suitable visualization tool, they (ii) are unsure of its effectiveness. We aim to offer support for identifying proper visualizations, and to increase the effectiveness of visualization techniques. In this dissertation, we characterize proposed software visualizations. To fill the gap between proposed visualizations and their practical application, we encapsulate such characteristics in an ontology, and propose a meta-visualization approach to find suitable visualizations. Amongst others characteristics of software visualizations, we identify that the medium used to display them can be a means to increase the effectiveness of visualization techniques for particular comprehension tasks.We implement visualization prototypes and validate our thesis via experiments. We found that even though developers using a physical 3D model medium required the least time to deal with tasks that involve identifying outliers, they perceived the least difficulty when visualizing systems based on the standard computer screen medium. Moreover, developers using immersive virtual reality obtained the highest recollection. We conclude that the effectiveness of software visualizations that use the city metaphor to support comprehension tasks can be increased when city visualizations are rendered in an appropriate medium. Furthermore, that visualization of software visualizations can be a suitable means for exploring their multiple characteristics that can be properly encapsulated in an ontology.

[1]  Andrian Marcus,et al.  A task oriented view of software visualization , 2002, Proceedings First International Workshop on Visualizing Software for Understanding and Analysis.

[2]  Simon Colton,et al.  Capturing player experience with post-game commentaries , 2010, CGAMES 2010.

[3]  Frédéric Tronel,et al.  Kayrebt: An activity diagram extraction and visualization toolset designed for the Linux codebase , 2015, 2015 IEEE 3rd Working Conference on Software Visualization (VISSOFT).

[4]  Sophia Krasikov,et al.  Execution patterns for visualizing web services , 2006, SoftVis '06.

[5]  Oscar Nierstrasz,et al.  The story of moose: an agile reengineering environment , 2005, ESEC/FSE-13.

[6]  Enrico Bertini,et al.  Quality Metrics in High-Dimensional Data Visualization: An Overview and Systematization , 2011, IEEE Transactions on Visualization and Computer Graphics.

[7]  Lucian Voinea,et al.  CVSscan: visualization of code evolution , 2005, SoftVis '05.

[8]  Amela Karahasanovic,et al.  A survey of controlled experiments in software engineering , 2005, IEEE Transactions on Software Engineering.

[9]  Andrian Marcus,et al.  CFB: A Call for Benchmarks - for Software Visualization , 2003, VISSOFT.

[10]  Claus Lewerentz,et al.  Representing development history in software cities , 2010, SOFTVIS '10.

[11]  Rudolf K. Keller,et al.  Software visualization tools: survey and analysis , 2001, Proceedings 9th International Workshop on Program Comprehension. IWPC 2001.

[12]  Nils Faltin,et al.  Structure and Constraints in Interactive Exploratory Algorithm Learning , 2001, Software Visualization.

[13]  Hironori Washizaki,et al.  Metrics Visualization Technique Based on the Origins and Function Layers for OSS-Based Development , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[14]  Oscar Nierstrasz,et al.  Embedding spatial software visualization in the IDE: an exploratory study , 2010, SOFTVIS '10.

[15]  Y. B. Wah,et al.  Power comparisons of Shapiro-Wilk , Kolmogorov-Smirnov , Lilliefors and Anderson-Darling tests , 2011 .

[16]  Hans-Peter Kriegel,et al.  Visualization Techniques for Mining Large Databases: A Comparison , 1996, IEEE Trans. Knowl. Data Eng..

[17]  John Stasko,et al.  BEST PAPER: A Knowledge Task-Based Framework for Design and Evaluation of Information Visualizations , 2004 .

[18]  Daniel A. Keim,et al.  Information Visualization and Visual Data Mining , 2002, IEEE Trans. Vis. Comput. Graph..

[19]  J. J. Higgins,et al.  The aligned rank transform for nonparametric factorial analyses using only anova procedures , 2011, CHI.

[20]  Arthur-Jozsef Molnar Live visualization of GUI application code coverage with GUITracer , 2015, 2015 IEEE 3rd Working Conference on Software Visualization (VISSOFT).

[21]  Alexander Egyed,et al.  A systematic mapping study of information visualization for software product line engineering , 2018, J. Softw. Evol. Process..

[22]  Jock D. Mackinlay,et al.  Automating the design of graphical presentations of relational information , 1986, TOGS.

[23]  Richard W. Vuduc,et al.  Communicating Software Architecture using a Unified Single-View Visualization , 2007, 12th IEEE International Conference on Engineering Complex Computer Systems (ICECCS 2007).

[24]  James A. Jones,et al.  Chronos: Visualizing slices of source-code history , 2013, 2013 First IEEE Working Conference on Software Visualization (VISSOFT).

[25]  R. Likert “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.

[26]  Manoel G. Mendonça,et al.  Software evolution visualization: A systematic mapping study , 2013, Inf. Softw. Technol..

[27]  Marcelo R. Campo,et al.  An Overview of 3D Software Visualization , 2009, IEEE Transactions on Visualization and Computer Graphics.

[28]  Steven P. Reiss,et al.  Automatic categorization and visualization of lock behavior , 2013, 2013 First IEEE Working Conference on Software Visualization (VISSOFT).

[29]  Michele Lanza,et al.  Visualizing the Evolution of Working Sets , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[30]  Jan Stage,et al.  Use Case Evaluation (UCE): A Method for Early Usability Evaluation in Software Development , 2007, INTERACT.

[31]  Claes Wohlin,et al.  Using Students as Subjects—A Comparative Study of Students and Professionals in Lead-Time Impact Assessment , 2000, Empirical Software Engineering.

[32]  Tobias Isenberg,et al.  A Systematic Review on the Practice of Evaluating Visualization , 2013, IEEE Transactions on Visualization and Computer Graphics.

[33]  Christine Nadel,et al.  Case Study Research Design And Methods , 2016 .

[34]  Tobias Schüle,et al.  A Visualization Framework for Parallelization , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[35]  Wilhelm Hasselbring,et al.  Hierarchical software landscape visualization for system comprehension: A controlled experiment , 2015, 2015 IEEE 3rd Working Conference on Software Visualization (VISSOFT).

[36]  Mohammad Ghafari,et al.  Towards actionable visualization for software developers , 2018, J. Softw. Evol. Process..

[37]  Ewan D. Tempero,et al.  User evaluation of polymetric views using a large visualization wall , 2010, SOFTVIS '10.

[38]  Craig Anslow,et al.  CodeSurveyor: Mapping large-scale software to aid in code comprehension , 2015, 2015 IEEE 3rd Working Conference on Software Visualization (VISSOFT).

[39]  Edward M. Reingold,et al.  Graph drawing by force‐directed placement , 1991, Softw. Pract. Exp..

[40]  Ronie Salgado,et al.  CuboidMatrix: Exploring Dynamic Structural Connections in Software Components Using Space-Time Cube , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[41]  Richard Wettel,et al.  Software Systems as Cities , 2010 .

[42]  Melanie Tory,et al.  How Information Visualization Novices Construct Visualizations , 2010, IEEE Trans. Vis. Comput. Graph..

[43]  Steffen Staab,et al.  What Is an Ontology? , 2009, Handbook on Ontologies.

[44]  Michele Lanza,et al.  Package patterns for visual architecture recovery , 2006, Conference on Software Maintenance and Reengineering (CSMR'06).

[45]  Matthias Hauswirth,et al.  Vestige: A visualization framework for engineering geometry-related software , 2015, 2015 IEEE 3rd Working Conference on Software Visualization (VISSOFT).

[46]  Mark A. Musen,et al.  The protégé project: a look back and a look forward , 2015, SIGAI.

[47]  Gruia-Catalin Roman,et al.  A taxonomy of program visualization systems , 1993, Computer.

[48]  Shen Lin,et al.  Towards anomaly comprehension: using structural compression to navigate profiling call-trees , 2010, SOFTVIS '10.

[49]  Emerson R. Murphy-Hill,et al.  Perquimans: A Tool for Visualizing Patterns of Spreadsheet Function Combinations , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[50]  Tamara Munzner,et al.  Visualization Analysis and Design , 2014, A.K. Peters visualization series.

[51]  Sally A. McKee,et al.  Interactive locality optimization on NUMA architectures , 2003, SoftVis '03.

[52]  Daniel A. Keim,et al.  ColorCAT: Guided Design of Colormaps for Combined Analysis Tasks , 2015, EuroVis.

[53]  Houari A. Sahraoui,et al.  What You See is What You Asked for: An Effort-Based Transformation of Code Analysis Tasks into Interactive Visualization Scenarios , 2011, 2011 IEEE 11th International Working Conference on Source Code Analysis and Manipulation.

[54]  Cheng Zhou,et al.  Graph Visualization for the Analysis of the Structure and Dynamics of Extreme-Scale Supercomputers , 2003, SoftVis '03.

[55]  Alexandre Bergel,et al.  Visually Exploring Object Mutation , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[56]  Oscar Nierstrasz,et al.  Explora: A visualisation tool for metric analysis of software corpora , 2015, 2015 IEEE 3rd Working Conference on Software Visualization (VISSOFT).

[57]  Paul A. Cairns,et al.  Measuring and defining the experience of immersion in games , 2008, Int. J. Hum. Comput. Stud..

[58]  Jarke J. van Wijk,et al.  Views on Visualization , 2006, IEEE Transactions on Visualization and Computer Graphics.

[59]  Mario Piattini,et al.  Gamification in software engineering - A systematic mapping , 2015, Inf. Softw. Technol..

[60]  Marja Kuittinen,et al.  Program animation based on the roles of variables , 2003, SoftVis '03.

[61]  Thomas Fritz,et al.  Using information fragments to answer the questions developers ask , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[62]  Mohammad Ghafari,et al.  MetaVis: Exploring Actionable Visualization , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[63]  James Noble,et al.  Exploring the inventor's paradox: applying jigsaw to software visualization , 2010, SOFTVIS '10.

[64]  Daniel M. Germán,et al.  Merge-Tree: Visualizing the Integration of Commits into Linux , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[65]  Malcolm Munro,et al.  Virtual but visible software , 2000, 2000 IEEE Conference on Information Visualization. An International Conference on Computer Visualization and Graphics.

[66]  Wilhelm Hasselbring,et al.  Exploring software cities in virtual reality , 2015, 2015 IEEE 3rd Working Conference on Software Visualization (VISSOFT).

[67]  Frank Maurer,et al.  Information Visualization for Agile Software Development , 2014, 2014 Second IEEE Working Conference on Software Visualization.

[68]  Steven P. Reiss,et al.  Jove: java as it happens , 2005, SoftVis '05.

[69]  Doug A. Bowman,et al.  Virtual Reality: How Much Immersion Is Enough? , 2007, Computer.

[70]  Ying Zhu,et al.  A Visualization Tool for 3D Graphics Program Comprehension and Debugging , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[71]  Anthony Cleve,et al.  DAHLIA 2.0: A Visual Analyzer of Database Usage in Dynamic and Heterogeneous Systems , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[72]  Keith Brian Gallagher,et al.  A Framework for Software Architecture Visualisation Assessment , 2005, 3rd IEEE International Workshop on Visualizing Software for Understanding and Analysis.

[73]  Patrick Ogao,et al.  Evaluation of software visualization tools: Lessons learned , 2009, 2009 5th IEEE International Workshop on Visualizing Software for Understanding and Analysis.

[74]  Wilhelm Hasselbring,et al.  Research perspective on supporting software engineering via physical 3D models , 2015, 2015 IEEE 3rd Working Conference on Software Visualization (VISSOFT).

[75]  Stéphane Ducasse,et al.  Polymetric Views - A Lightweight Visual Approach to Reverse Engineering , 2003, IEEE Trans. Software Eng..

[76]  Marvin V. Zelkowitz,et al.  Experimental Models for Validating Technology , 1998, Computer.

[77]  Thomas W. Malone,et al.  What makes things fun to learn? heuristics for designing instructional computer games , 1980, SIGSMALL '80.

[78]  Andrew M. Sutton,et al.  MosaiCode: Visualizing large scale software: A tool demonstration , 2011, 2011 6th International Workshop on Visualizing Software for Understanding and Analysis (VISSOFT).

[79]  Michael Kaufmann,et al.  Visualizing structural properties of irregular parallel computations , 2005, SoftVis '05.

[80]  Günther Ruhe,et al.  DEVis: A tool for visualizing software document evolution , 2013, 2013 First IEEE Working Conference on Software Visualization (VISSOFT).

[81]  Andrei Chis,et al.  GTInspector: a moldable domain-aware object inspector , 2015, SPLASH.

[82]  Christian Jacob,et al.  evoExplore: Multiscale Visualization of Evolutionary Histories in Virtual Reality , 2018, EvoMUSART.

[83]  Hausi A. Müller,et al.  Requirements of Software Visualization Tools: A Literature Survey , 2007, 2007 4th IEEE International Workshop on Visualizing Software for Understanding and Analysis.

[84]  Peter Young,et al.  Visualising software in virtual reality , 1998, Proceedings. 6th International Workshop on Program Comprehension. IWPC'98 (Cat. No.98TB100242).

[85]  Mohammad Ghafari,et al.  On the Impact of the Medium in the Effectiveness of 3D Software Visualizations , 2017, 2017 IEEE Working Conference on Software Visualization (VISSOFT).

[86]  John T. Stasko,et al.  An evaluation of space-filling information visualizations for depicting hierarchical structures , 2000, Int. J. Hum. Comput. Stud..

[87]  Wei Wang,et al.  EVolve: an open extensible software visualization framework , 2003, SoftVis '03.

[88]  Alfa Yohannis,et al.  Sort Attack: Visualization and Gamification of Sorting Algorithm Learning , 2015, 2015 7th International Conference on Games and Virtual Worlds for Serious Applications (VS-Games).

[89]  Lisa Amini,et al.  Streamsight: a visualization tool for large-scale streaming applications , 2008, SoftVis '08.

[90]  T. Dean Hendrix,et al.  Experimental evaluation of animated-verifying object viewers for Java , 2006, SoftVis '06.

[91]  Christian S. Collberg,et al.  A system for graph-based visualization of the evolution of software , 2003, SoftVis '03.

[92]  Cláudia Maria Lima Werner,et al.  Using a Task-Oriented Framework to Characterize Visualization Approaches , 2014, 2014 Second IEEE Working Conference on Software Visualization.

[93]  Gerik Scheuermann,et al.  A structured approach for conducting a series of controlled experiments in software visualization , 2015, 2014 International Conference on Information Visualization Theory and Applications (IVAPP).

[94]  Pourang Irani,et al.  Using Perceptual Syntax to Enhance Semantic Content in Diagrams , 2001, IEEE Computer Graphics and Applications.

[95]  Tibor Gyimóthy,et al.  Using the City Metaphor for Visualizing Test-Related Metrics , 2016, 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[96]  Ciaran O'Reilly,et al.  The war room command console: shared visualizations for inclusive team coordination , 2005, SoftVis '05.

[97]  Robert DeLine,et al.  Information Needs in Collocated Software Development Teams , 2007, 29th International Conference on Software Engineering (ICSE'07).

[98]  M. Sheelagh T. Carpendale,et al.  Empirical Studies in Information Visualization: Seven Scenarios , 2012, IEEE Transactions on Visualization and Computer Graphics.

[99]  Timothy A. Davis,et al.  KScope: A Modularized Tool for 3D Visualization of Object-Oriented Programs , 2003, VISSOFT.

[100]  Gerald C. Gannod,et al.  vizSlice: Visualizing Large Scale Software Slices , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[101]  Teemu Sirkiä,et al.  Jsvee & Kelmu: Creating and Tailoring Program Animations for Computing Education , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[102]  S. Diehl,et al.  Software visualization , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[103]  James Noble,et al.  Towards Visual Software Analytics , 2008 .

[104]  Jürgen Döllner,et al.  Visual exploration of function call graphs for feature location in complex software systems , 2006, SoftVis '06.

[105]  Daniel Baldauf,et al.  Time perception as a workload measure in simulated car driving. , 2009, Applied ergonomics.

[106]  James Everett Young,et al.  ChronoTwigger: A Visual Analytics Tool for Understanding Source and Test Co-evolution , 2014, 2014 Second IEEE Working Conference on Software Visualization.

[107]  Katsuro Inoue,et al.  SoL Mantra: Visualizing Update Opportunities Based on Library Coexistence , 2017, 2017 IEEE Working Conference on Software Visualization (VISSOFT).

[108]  Camilla Forsell,et al.  A Guide to Scientific Evaluation in Information Visualization , 2010, 2010 14th International Conference Information Visualisation.

[109]  Malcolm Munro,et al.  Comprehension with[in] virtual environment visualisations , 1999, Proceedings Seventh International Workshop on Program Comprehension.

[110]  Gail C. Murphy,et al.  Questions programmers ask during software evolution tasks , 2006, SIGSOFT '06/FSE-14.

[111]  Samuel Z. Guyer,et al.  An Interactive Microarray Call-Graph Visualization , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[112]  Xiaomin Wu,et al.  Plugging-in visualization: experiences integrating a visualization tool with Eclipse , 2003, SoftVis '03.

[113]  Chris Parnin,et al.  A catalogue of lightweight visualizations to support code smell inspection , 2008, SOFTVIS.

[114]  Mohammad Ghafari,et al.  CommunityExplorer: A Framework for Visualizing Collaboration Networks , 2016, IWST.

[115]  Wilhelm Hasselbring,et al.  Software landscape and application visualization for system comprehension with ExplorViz , 2017, Inf. Softw. Technol..

[116]  David W. Binkley,et al.  Dependence cluster visualization , 2010, SOFTVIS '10.

[117]  Ahmed Seffah,et al.  Visualization Patterns: A Context-Sensitive Tool to Evaluate Visualization Techniques , 2007, 2007 4th IEEE International Workshop on Visualizing Software for Understanding and Analysis.

[118]  Joseph A. Cottam,et al.  Pixel-oriented techniques for visualizing next-generation HPC systems , 2015, 2015 IEEE 3rd Working Conference on Software Visualization (VISSOFT).

[119]  Alessandro Orso,et al.  GAMMATELLA: visualization of program-execution data for deployed software , 2004, Proceedings. 26th International Conference on Software Engineering.

[120]  Alexandru Telea,et al.  Visual support for porting large code bases , 2011, 2011 6th International Workshop on Visualizing Software for Understanding and Analysis (VISSOFT).

[121]  Oscar Nierstrasz,et al.  Overcoming Issues of 3D Software Visualization through Immersive Augmented Reality , 2018, 2018 IEEE Working Conference on Software Visualization (VISSOFT).

[122]  Romain Robbes,et al.  Software systems as cities: a controlled experiment , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[123]  Samuel S. Silva,et al.  Head-mounted display versus desktop for 3D navigation in virtual reality: a user study , 2008, Multimedia Tools and Applications.

[124]  Blaine A. Price,et al.  A Principled Taxonomy of Software Visualization , 1993, J. Vis. Lang. Comput..

[125]  George Valença,et al.  Accepted Manuscript Requirements Engineering for Software Product Lines: a Systematic Literature Review Accepted Manuscript Requirements Engineering for Software Product Lines: a Systematic Literature Review Accepted Manuscript , 2022 .

[126]  Hausi A. Müller,et al.  The Tools Perspective on Software Reverse Engineering: Requirements, Construction, and Evaluation , 2010, Adv. Comput..

[127]  Ben Shneiderman,et al.  Tree-maps: a space-filling approach to the visualization of hierarchical information structures , 1991, Proceeding Visualization '91.

[128]  Edward Rolf Tufte,et al.  The visual display of quantitative information , 1985 .

[129]  Jing Li,et al.  The Qualitas Corpus: A Curated Collection of Java Code for Empirical Studies , 2010, 2010 Asia Pacific Software Engineering Conference.

[130]  H. Levene Robust tests for equality of variances , 1961 .

[131]  Mohammad Ghafari,et al.  Walls, Pillars and Beams: A 3D Decomposition of Quality Anomalies , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[132]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[133]  Luciana Porcher Nedel,et al.  Immersive Visualization of Abstract Information: An Evaluation on Dimensionally-Reduced Data Scatterplots , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[134]  Mohammad Ghafari,et al.  CityVR: Gameful Software Visualization , 2017, 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[135]  Wim De Pauw,et al.  Zinsight: a visual and analytic environment for exploring large event traces , 2010, SOFTVIS '10.

[136]  Michele Lanza,et al.  Visualizing Software Systems as Cities , 2007, 2007 4th IEEE International Workshop on Visualizing Software for Understanding and Analysis.

[137]  M. Sheelagh T. Carpendale,et al.  Constructive visualization , 2014, Conference on Designing Interactive Systems.

[138]  Peng Liang,et al.  A systematic review of software architecture visualization techniques , 2014, J. Syst. Softw..

[139]  John D. Kelleher,et al.  On Using Tree Visualisation Techniques to Support Source Code Comprehension , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[140]  Bernd Fischer,et al.  Interactive tag cloud visualization of software version control repositories , 2015, 2015 IEEE 3rd Working Conference on Software Visualization (VISSOFT).

[141]  Neil A. Ernst,et al.  The Journal of Systems and Software , 2022 .

[142]  PlaisantCatherine,et al.  Empirical Studies in Information Visualization , 2012 .

[143]  Jason Leigh,et al.  Visualizing object-oriented software in virtual reality , 2001, Proceedings 9th International Workshop on Program Comprehension. IWPC 2001.

[144]  Benjamin Cerat,et al.  Validation of Software Visualization Tools: A Systematic Mapping Study , 2014, 2014 Second IEEE Working Conference on Software Visualization.

[145]  Ameya Vivek Datey Experiments in the Use of Immersion for Information Visualization , 2002 .

[146]  Michele Lanza,et al.  Program Comprehension through Software Habitability , 2007, 15th IEEE International Conference on Program Comprehension (ICPC '07).

[147]  Mohammad Ghafari,et al.  Towards Actionable Visualisation in Software Development , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[148]  Thomas D. LaToza,et al.  Hard-to-answer questions about code , 2010, PLATEAU '10.

[149]  Daniel M. German,et al.  On the use of visualization to support awareness of human activities in software development: a survey and a framework , 2005, SoftVis '05.

[150]  Carlos Jensen,et al.  Beyond pretty pictures: Examining the benefits of code visualization for Open Source newcomers , 2009, 2009 5th IEEE International Workshop on Visualizing Software for Understanding and Analysis.

[151]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[152]  Tamara Munzner,et al.  Process and Pitfalls in Writing Information Visualization Research Papers , 2008, Information Visualization.

[153]  Steven P. Reiss Visualizing Java in action , 2003, SoftVis '03.

[154]  Paul A. Cairns,et al.  A grounded investigation of game immersion , 2004, CHI EA '04.

[155]  Steven P. Reiss The Challenge of Helping the Programmer during Debugging , 2014, 2014 Second IEEE Working Conference on Software Visualization.

[156]  James Noble,et al.  SourceVis: Collaborative software visualization for co-located environments , 2013, 2013 First IEEE Working Conference on Software Visualization (VISSOFT).

[157]  Abram Hindle,et al.  Visualizing Project Evolution through Abstract Syntax Tree Analysis , 2016, 2016 IEEE Working Conference on Software Visualization (VISSOFT).

[158]  Bernd Hamann,et al.  Stacked-widget visualization of scheduling-based algorithms , 2008, SoftVis '08.

[159]  Katsuro Inoue,et al.  Visualizing the Evolution of Systems and Their Library Dependencies , 2014, 2014 Second IEEE Working Conference on Software Visualization.

[160]  Andrew P. Black,et al.  An interactive ambient visualization for code smells , 2010, SOFTVIS '10.

[161]  Lucian Voinea,et al.  Visual exploration of program structure, dependencies and metrics with SolidSX , 2011, 2011 6th International Workshop on Visualizing Software for Understanding and Analysis (VISSOFT).

[162]  Martin Pinzger,et al.  "A Bug's Life" Visualizing a Bug Database , 2007, 2007 4th IEEE International Workshop on Visualizing Software for Understanding and Analysis.

[163]  Christoph Wysseier,et al.  Visualizing live software systems in 3D , 2006, SoftVis '06.

[164]  Stéphane Ducasse,et al.  Analyzing software evolution through feature views , 2006, J. Softw. Maintenance Res. Pract..

[165]  Lucian Voinea,et al.  Multiscale and multivariate visualizations of software evolution , 2006, SoftVis '06.

[166]  A. Adithya Parallel Coordinates , 2015 .

[167]  W. Kruskal,et al.  Use of Ranks in One-Criterion Variance Analysis , 1952 .

[168]  Pierre Poulin,et al.  Visualizing software dynamicities with heat maps , 2013, 2013 First IEEE Working Conference on Software Visualization (VISSOFT).

[169]  Claes Wohlin,et al.  Experimentation in Software Engineering , 2000, The Kluwer International Series in Software Engineering.

[170]  Martin Wittiger,et al.  Advancing data race investigation and classification through visualization , 2015, 2015 IEEE 3rd Working Conference on Software Visualization (VISSOFT).

[171]  Lauri Malmi,et al.  Visualising concurrent programs with dynamic dependence graphs , 2011, 2011 6th International Workshop on Visualizing Software for Understanding and Analysis (VISSOFT).

[172]  Alexandru Telea,et al.  Visual Exploration of Combined Architectural and Metric Information , 2005, 3rd IEEE International Workshop on Visualizing Software for Understanding and Analysis.

[173]  Andrei Chis,et al.  Spotter: towards a unified search interface in IDEs , 2015, SPLASH.

[174]  N. F. Noy,et al.  Ontology Development 101: A Guide to Creating Your First Ontology , 2001 .

[175]  Alexandru Telea,et al.  Visualizing Dynamic Memory Allocations , 2007, 2007 4th IEEE International Workshop on Visualizing Software for Understanding and Analysis.

[176]  Felix Raab CodeSmellExplorer: Tangible exploration of code smells and refactorings , 2012, 2012 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[177]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[178]  Mary Czerwinski,et al.  Towards understanding programs through wear-based filtering , 2005, SoftVis '05.

[179]  Niklas Elmqvist,et al.  Patterns for visualization evaluation , 2015, Inf. Vis..

[180]  Pierre Poulin,et al.  Generating visualization-based analysis scenarios from maintenance task descriptions , 2009, 2009 5th IEEE International Workshop on Visualizing Software for Understanding and Analysis.

[181]  Mitchel Resnick,et al.  Scratch: A Sneak Preview , 2004 .

[182]  Sophia Kluge,et al.  The Survey Handbook , 2016 .

[183]  John F. Lucas,et al.  Exploring the Benefits of Immersion in Abstract Information Visualization , 2004 .

[184]  Joseph A. Cottam,et al.  Representing unit test data for large scale software development , 2008, SoftVis '08.

[185]  Roland H. C. Yap,et al.  Visualizing windows system traces , 2010, SOFTVIS '10.

[186]  Sara L. Su,et al.  Heapviz: interactive heap visualization for program understanding and debugging , 2010, SOFTVIS '10.

[187]  Ayellet Tal,et al.  Visualization of mobile object environments , 2005, SoftVis '05.

[188]  Francisco J. García-Peñalvo,et al.  Supporting the understanding of the evolution of software items , 2008, SoftVis '08.

[189]  Terhi Kilamo,et al.  Software visualization today: systematic literature review , 2016, MindTrek.

[190]  Patrick Ogao,et al.  Classifying desirable features of software visualization tools for corrective maintenance , 2008, SOFTVIS.

[191]  Matthias Hauswirth,et al.  Trevis: a context tree visualization & analysis framework and its use for classifying performance failure reports , 2010, SOFTVIS '10.

[192]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[193]  Gerardo Canfora,et al.  Achievements and challenges in software reverse engineering , 2011, Commun. ACM.

[194]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[195]  Giordano Tamburrelli,et al.  Understanding gamification mechanisms for software development , 2013, ESEC/FSE 2013.

[196]  Hausi A. Müller,et al.  Dependencies Analysis of Azureus with Rigi: Tool Demo Challenge , 2007, 2007 4th IEEE International Workshop on Visualizing Software for Understanding and Analysis.