Insight-Based Studies for Pathway and Microarray Visualization Tools

Pathway diagrams, similar to the graph diagrams using a node-link representation, are used by biologists to represent complex interactions at the molecular level in living cells. The recent shift towards data-intensive bioinformatics and systems-level science has created a strong need for advanced pathway visualization tools that support exploratory data analysis. User studies suggest that an important requirement for biologists is the need to associate microarray data to pathway diagrams. A design space for visualization tools that allow analysis of microarray data in pathway context was identified for a systematic evaluation of the visualization alternatives. The design space is divided into two dimensions. Dimension 1 is based on the method used to overlay data attributes onto pathway nodes. The three possible approaches are: overlay of data on pathway nodes one data attribute at a time by manipulating a visual property (e.g. color) of the node, along with sliders or some such mechanism to animate the pathway for other timepoints. In another approach data from all the attributes in data can be overlaid simultaneously by embedding small charts (e.g., line charts or heatmap) into pathway nodes. The third approach uses miniature version of the pathways-as-glyph view for each attribute in the data. Dimension 2 decides if additional view besides pathway diagrams were used. These pathway visualizations are often linked to other type of visualization methods (e.g., parallel co-ordinates) using the concept of brushing and linking. The visualization alternatives from pathway + microarray data design space were evaluated by conducting two independent user studies. Both the studies used timeseries datasets. The first study used visualization alternatives from both dimension 1 and dimension 2. The results suggest that the method to overlay multidimensional data on pathway nodes has a non trivial influence on accuracy of participants’ responses, whereas the number of visualizations affect participants’ performance time for pre-selected tasks. The second study used visualization alternatives from dimension 1 that focuses on method used to overlay data attributes on pathway nodes. The study suggests that participants using pathway visualization that display data one attribute at a time on nodes have more controlled performance for all type of tasks as compared to the participants using other alternatives. Participants using pathway visualization that display data in node-as-glyphs view have better performance for tasks that require analysis for a single node, and identifying outlier nodes. Whereas, pathway visualizations with pathways-asglyph view provide better performance on tasks that require analysis of overall changes in the pathway, and identifying interesting timepoints in the data. An insight-based method was designed to evaluate visualization tools for real world biologists’ data analysis scenarios. The insight-based method uses different quantifiable characteristics of an ‘insight’ that can be measured uniformly across participants.

[1]  Michael Krauthammer,et al.  GENIES: a natural-language processing system for the extraction of molecular pathways from journal articles , 2001, ISMB.

[2]  J. Hoheisel,et al.  Monitoring the Switch from Housekeeping to Pathogen Defense Metabolism in Arabidopsis thaliana Using cDNA Arrays* , 2002, The Journal of Biological Chemistry.

[3]  D. Stephan,et al.  Integrating microarrays into disease-gene identification strategies , 2003, Expert review of molecular diagnostics.

[4]  Allan Collins,et al.  Design Research: Theoretical and Methodological Issues , 2004 .

[5]  Ivan Herman,et al.  Graph Visualization and Navigation in Information Visualization: A Survey , 2000, IEEE Trans. Vis. Comput. Graph..

[6]  Colin Ware,et al.  Cognitive Measurements of Graph Aesthetics , 2002, Inf. Vis..

[7]  John D. Storey,et al.  Statistical significance for genomewide studies , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Chris North,et al.  Visualizing Biological Pathways: Requirements Analysis, Systems Evaluation and Research Agenda , 2005, Inf. Vis..

[9]  Alfred Kobsa,et al.  A Workplace Study of the Adoption of Information Visualization Systems. Proceedings of I-KNOW'03: 3rd International Conference on Knowledge Management, Graz, Austria , 2003 .

[10]  Mary Beth Rosson,et al.  Usability Engineering: Scenario-based Development of Human-Computer Interaction , 2001 .

[11]  Allan Kuchinsky,et al.  Biological storytelling: a software tool for biological information organization based upon narrative structure , 2002, AVI '02.

[12]  Gary D Bader,et al.  BIND--The Biomolecular Interaction Network Database. , 2001, Nucleic acids research.

[13]  Jock D. Mackinlay,et al.  Automating the design of graphical presentations of relational information , 1986, TOGS.

[14]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[15]  James D. Hollan,et al.  Pad++: a zooming graphical interface for exploring alternate interface physics , 1994, UIST '94.

[16]  Chris North,et al.  Single complex glyphs versus multiple simple glyphs , 2005, CHI Extended Abstracts.

[17]  Melanie Tory,et al.  Evaluating Visualizations: Do Expert Reviews Work? , 2005, IEEE Computer Graphics and Applications.

[18]  T. J. Jankun-Kelly,et al.  MoireGraphs: radial focus+context visualization and interaction for graphs with visual nodes , 2003, IEEE Symposium on Information Visualization 2003 (IEEE Cat. No.03TH8714).

[19]  Peter D. Karp,et al.  EcoCyc: Encyclopedia of Escherichia coli genes and metabolism , 1998, Nucleic Acids Res..

[20]  P. Shannon,et al.  Cytoscape: a software environment for integrated models of biomolecular interaction networks. , 2003, Genome research.

[21]  Naren Ramakrishnan,et al.  The Emerging Landscape of Bioinformatics Software Systems , 2002, Computer.

[22]  Chaomei Chen,et al.  Empirical studies of information visualization: a meta-analysis , 2000, Int. J. Hum. Comput. Stud..

[23]  Mary Czerwinski,et al.  Empirical evaluation of information visualizations: an introduction , 2000, Int. J. Hum. Comput. Stud..

[24]  Kristin A. Cook,et al.  Illuminating the Path: The Research and Development Agenda for Visual Analytics , 2005 .

[25]  J. C. Flanagan.  Psychological Bulletin THE CRITICAL INCIDENT TECHNIQUE , 2022 .

[26]  Chris North,et al.  Toward measuring visualization insight , 2006, IEEE Computer Graphics and Applications.

[27]  G. Churchill.  Fundamentals of experimental design for cDNA microarrays , 2002, Nature Genetics.

[28]  Roberto Tamassia,et al.  Difference Metrics for Interactive Orthogonal Graph Drawing Algorithms , 2000, J. Graph Algorithms Appl..

[29]  S. Rhee,et al.  MAPMAN: a user-driven tool to display genomics data sets onto diagrams of metabolic pathways and other biological processes. , 2004, The Plant journal : for cell and molecular biology.

[30]  David H. Laidlaw,et al.  Thoughts on User Studies: Why, How and When , 1993 .

[31]  Luciana P. Nedel,et al.  Evaluating Usability of Information Visualization Techniques , 2002 .

[32]  C. Pipper,et al.  [''R"--project for statistical computing]. , 2008, Ugeskrift for laeger.

[33]  Donald H. House,et al.  Design and Evaluation in Visualization Research , 2005, IEEE Visualization.

[34]  Min Su Lee,et al.  UniPath: A Knowledge Representation System for Biological Pathways , 2003 .

[35]  John Rieman,et al.  A field study of exploratory learning strategies , 1996, TCHI.

[36]  Akihiko Konagaya,et al.  GSCope: a clipped fisheye viewer effective for highly complicated biomolecular network graphs , 2003, Bioinform..

[37]  Marco Winckler,et al.  On evaluating information visualization techniques , 2002, AVI '02.

[38]  Susumu Goto,et al.  KEGG: Kyoto Encyclopedia of Genes and Genomes , 2000, Nucleic Acids Res..

[39]  Wendy E. Mackay,et al.  The missing link: augmenting biology laboratory notebooks , 2002, UIST '02.

[40]  Austin Henderson,et al.  Interaction Analysis: Foundations and Practice , 1995 .

[41]  Deborah Hix,et al.  Developing user interfaces , 1993 .

[42]  Emek Demir,et al.  PATIKA: an integrated visual environment for collaborative construction and analysis of cellular pathways , 2002, Bioinform..

[43]  Werner Dubitzky,et al.  A Practical Approach to Microarray Data Analysis , 2003, Springer US.

[44]  M. Bittner,et al.  Expression profiling using cDNA microarrays , 1999, Nature Genetics.

[45]  Stuart K. Card,et al.  Visual information foraging in a focus + context visualization , 2001, CHI.

[46]  Ben Shneiderman,et al.  A Rank-by-Feature Framework for Unsupervised Multidimensional Data Exploration Using Low Dimensional Projections , 2004, IEEE Symposium on Information Visualization.

[47]  P. Fayers,et al.  The Visual Display of Quantitative Information , 1990 .

[48]  Jakob Nielsen,et al.  Finding usability problems through heuristic evaluation , 1992, CHI.

[49]  Colin Ware,et al.  Information Visualization: Perception for Design , 2000 .

[50]  Ganesh S. Oak.  Information Visualization Introduction , 2022 .

[51]  Janet Davis,et al.  A comparison of synchronous remote and local usability studies for an expert interface , 2004, CHI EA '04.

[52]  Akihiko Konagaya,et al.  KnowledgeEditor: a new tool for interactive modeling and analyzing biological pathways based on microarray data , 2003, Bioinform..

[53]  Roy Rada,et al.  Modelling Situated Actions in Collaborative Hypertext Databases. , 2006 .

[54]  H. Brown.  The real value of microarray technology. , 2003, The Lancet. Oncology.

[55]  Ben Shneiderman,et al.  Dynamic querying for pattern identification in microarray and genomic data , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[56]  Ben Shneiderman,et al.  Knowledge discovery in high-dimensional data: case studies and a user survey for the rank-by-feature framework , 2006, IEEE Transactions on Visualization and Computer Graphics.

[57]  John Quackenbush.  Microarray data normalization and transformation , 2002, Nature Genetics.

[58]  Chris North,et al.  An Evaluation of Microarray Visualization Tools for Biological Insight , 2004 .

[59]  D. Botstein,et al.  Cluster analysis and display of genome-wide expression patterns. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[60]  Tamara Munzner,et al.  Visualizing the global topology of the MBone , 1996, Proceedings IEEE Symposium on Information Visualization '96.

[61]  Ben Shneiderman,et al.  Interactively Exploring Hierarchical Clustering Results , 2002, Computer.

[62]  Catherine Plaisant,et al.  The challenge of information visualization evaluation , 2004, AVI.

[63]  Alfred Kobsa,et al.  An empirical comparison of three commercial information visualization systems , 2001, IEEE Symposium on Information Visualization, 2001. INFOVIS 2001..

[64]  G. Karypis,et al.  Interferon-inducible gene expression signature in peripheral blood cells of patients with severe lupus , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[65]  Octavio H. Juarez-Espinosa.  CAEVA: cognitive architecture to evaluate visualization applications , 2003, Proceedings on Seventh International Conference on Information Visualization, 2003. IV 2003..

[66]  Ian M. Donaldson,et al.  BIND: the Biomolecular Interaction Network Database , 2001, Nucleic Acids Res..

[67]  Helen C. Purchase,et al.  Metrics for Graph Drawing Aesthetics , 2002, J. Vis. Lang. Comput..

[68]  Nicholas J. Belkin,et al.  Innovation and evaluation of information: a CHI98 workshop , 1999, SGCH.

[69]  Mark V. Springett,et al.  A comparison of usability techniques for evaluating design , 1997, DIS '97.

[70]  Chris North,et al.  An Insight-Based Longitudinal Study of Visual Analytics , 2006, IEEE Transactions on Visualization and Computer Graphics.

[71]  Allan Kuchinsky,et al.  When worlds collide: Molecular biology as interdisciplinary collaboration , 2001, ECSCW.

[72]  Chris North,et al.  An Evaluation of Microarray Visualization Tools for Biological Insight , 2004, IEEE Symposium on Information Visualization.

[73]  Jonathan C. Roberts,et al.  On encouraging multiple views for visualization , 1998, Proceedings. 1998 IEEE Conference on Information Visualization. An International Conference on Computer Visualization and Graphics (Cat. No.98TB100246).

[74]  Carole A. Goble,et al.  A classification of tasks in bioinformatics , 2001, Bioinform..

[75]  Wolmet Barendregt,et al.  A comparison of two analytical evaluation methods for educational computer games for young children , 2007, Cognition, Technology & Work.

[76]  Patty Curthoys,et al.  Developing user interfaces: Ensuring usability through product and process , 1997 .

[77]  Bonnie E. John,et al.  Tracking the effectiveness of usability evaluation methods , 1997, Behav. Inf. Technol..

[78]  Steven C. Lawlor,et al.  MAPPFinder: using Gene Ontology and GenMAPP to create a global gene-expression profile from microarray data , 2003, Genome Biology.

[79]  H. Rex Hartson,et al.  Developing user interfaces: ensuring usability through product & process , 1993 .

[80]  G. Yarrington.  Molecular Cell Biology , 1987, The Yale Journal of Biology and Medicine.

[81]  Ivan Bratko,et al.  GenePath: a system for automated construction of genetic networks from mutant data , 2003, Bioinform..

[82]  Pourang Irani,et al.  Diagramming information structures using 3D perceptual primitives , 2003, TCHI.

[83]  Philippe Castagliola,et al.  A Comparison of the Readability of Graphs Using Node-Link and Matrix-Based Representations , 2004, IEEE Symposium on Information Visualization.

[84]  Robin Jeffries,et al.  Usability testing vs. heuristic evaluation: was there a contest? , 1992, SGCH.

[85]  Issei Fujishiro,et al.  The elements of graphing data , 2005, The Visual Computer.

[86]  Allison Woodruff,et al.  Guidelines for using multiple views in information visualization , 2000, AVI '00.

[87]  D. Lockhart,et al.  Functional Genomics , 1999, Springer Netherlands.

[88]  Carl Gutwin,et al.  A comparison of usage evaluation and inspection methods for assessing groupware usability , 2001, GROUP.

[89]  Steven C. Lawlor,et al.  GenMAPP, a new tool for viewing and analyzing microarray data on biological pathways , 2002, Nature Genetics.

[90]  James R. Eagan,et al.  Low-level components of analytic activity in information visualization , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[91]  Xiaoyan Zhu,et al.  PathwayFinder: Paving the Way Towards Automatic Pathway Extraction , 2004, APBC.

[92]  M. Eisen,et al.  Gene expression informatics —it's all in your mine , 1999, Nature Genetics.

[93]  Edward Rolf Tufte,et al.  The visual display of quantitative information , 1985 .

[94]  Roger E Bumgarner,et al.  Cellular transcriptional profiling in influenza A virus-infected lung epithelial cells: The role of the nonstructural NS1 protein in the evasion of the host innate defense and its potential contribution to pandemic influenza , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[95]  Roberto Tamassia,et al.  A User Study in Similarity Measures for Graph Drawing , 2000, J. Graph Algorithms Appl..

[96]  Purvi Saraiya,et al.  Visualization of graphs with associated timeseries data , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[97]  Stephen A. R. Scrivener,et al.  The Use of Breakdown Analysis in Synchronous CSCW System Design , 1993, ECSCW.

[98]  Chris North,et al.  An insight-based methodology for evaluating bioinformatics visualizations , 2005, IEEE Transactions on Visualization and Computer Graphics.

[99]  Hannah Brow.  The real value of microarray technology. , 2003 .