Visualizing Dataflow Graphs of Deep Learning Models in TensorFlow

We present a design study of the TensorFlow Graph Visualizer, part of the TensorFlow machine intelligence platform. This tool helps users understand complex machine learning architectures by visualizing their underlying dataflow graphs. The tool works by applying a series of graph transformations that enable standard layout techniques to produce a legible interactive diagram. To declutter the graph, we decouple non-critical nodes from the layout. To provide an overview, we build a clustered graph using the hierarchical structure annotated in the source code. To support exploration of nested structure on demand, we perform edge bundling to enable stable and responsive cluster expansion. Finally, we detect and highlight repeated structures to emphasize a model's modular composition. To demonstrate the utility of the visualizer, we describe example usage scenarios and report user feedback. Overall, users find the visualizer useful for understanding, debugging, and sharing the structures of their models.

[1]  Jeffrey Heer,et al.  D³ Data-Driven Documents , 2011, IEEE Transactions on Visualization and Computer Graphics.

[2]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[3]  Kwan-Liu Ma,et al.  Opening the black box - data driven visualization of neural networks , 2005, VIS 05. IEEE Visualization, 2005..

[4]  Jarke J. van Wijk,et al.  Interactive Visualization of Small World Graphs , 2004, IEEE Symposium on Information Visualization.

[5]  Emden R. Gansner,et al.  Graphviz and Dynagraph – Static and Dynamic Graph Drawing Tools , 2003 .

[6]  Stephen C. North,et al.  Online Hierarchical Graph Drawing , 2001, GD.

[7]  Yuan Yu,et al.  Dryad: distributed data-parallel programs from sequential building blocks , 2007, EuroSys '07.

[8]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Samy Bengio,et al.  Torch: a modular machine learning software library , 2002 .

[10]  Peter Christen,et al.  A Survey of Indexing Techniques for Scalable Record Linkage and Deduplication , 2012, IEEE Transactions on Knowledge and Data Engineering.

[11]  Stephen A. Cook,et al.  The complexity of theorem-proving procedures , 1971, STOC.

[12]  James Abello,et al.  ASK-GraphView: A Large Scale Graph Visualization System , 2006, IEEE Transactions on Visualization and Computer Graphics.

[13]  Cláudio T. Silva,et al.  VisTrails: visualization meets data management , 2006, SIGMOD Conference.

[14]  Hiroshi Nagamochi,et al.  Straight-Line Drawing Algorithms for Hierarchical Graphs and Clustered Graphs , 1996, Algorithmica.

[15]  John W. Tukey,et al.  Exploratory Data Analysis. , 1979 .

[16]  Craig Chambers,et al.  FlumeJava: easy, efficient data-parallel pipelines , 2010, PLDI '10.

[17]  Tamara Munzner,et al.  GrouseFlocks: Steerable Exploration of Graph Hierarchy Space , 2008, IEEE Transactions on Visualization and Computer Graphics.

[18]  Michael Forster,et al.  Applying Crossing Reduction Strategies to Layered Compound Graphs , 2002, GD.

[19]  Kim Marriott,et al.  Constrained graph layout by stress majorization and gradient projection , 2009, Discret. Math..

[20]  Martin Wattenberg,et al.  Direct-Manipulation Visualization of Deep Networks , 2017, ArXiv.

[21]  Martin Wattenberg,et al.  Centrality Based Visualization of Small World Graphs , 2008, Comput. Graph. Forum.

[22]  Arvind Satyanarayan,et al.  Declarative interaction design for data visualization , 2014, UIST.

[23]  Kim Marriott,et al.  Topology Preserving Constrained Graph Layout , 2009, GD.

[24]  Zhen Li,et al.  Towards Better Analysis of Deep Convolutional Neural Networks , 2016, IEEE Transactions on Visualization and Computer Graphics.

[25]  Kim Marriott,et al.  HOLA: Human-like Orthogonal Network Layout , 2016, IEEE Transactions on Visualization and Computer Graphics.

[26]  Ben Shneiderman,et al.  Motif simplification: improving network visualization readability with fan, connector, and clique glyphs , 2013, CHI.

[27]  Mitsuhiko Toda,et al.  Methods for Visual Understanding of Hierarchical System Structures , 1981, IEEE Transactions on Systems, Man, and Cybernetics.

[28]  Jeffrey Heer,et al.  Perfopticon: Visual Query Analysis for Distributed Databases , 2015, Comput. Graph. Forum.

[29]  Evan Czaplicki,et al.  Elm : Concurrent FRP for Functional GUIs , 2012 .

[30]  Kim Marriott,et al.  Drawing directed graphs using quadratic programming , 2006, IEEE Transactions on Visualization and Computer Graphics.

[31]  M. Orne Demand Characteristics and the Concept of Quasi-Controls1 , 2009 .

[32]  Steven Hand,et al.  CIEL: A Universal Execution Engine for Distributed Data-Flow Computing , 2011, NSDI.

[33]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[34]  Yehuda Koren,et al.  Topological fisheye views for visualizing large graphs , 2004, IEEE Transactions on Visualization and Computer Graphics.

[35]  Georg Sander,et al.  Layout of compound directed graphs , 1996 .

[36]  Kozo Sugiyama,et al.  Layout Adjustment and the Mental Map , 1995, J. Vis. Lang. Comput..

[37]  Jens Gerken,et al.  IPSep-CoLa: An Incremental Procedure for Separation Constraint Layout of Graphs , 2006 .

[38]  Jeffrey Heer,et al.  SpanningAspectRatioBank Easing FunctionS ArrayIn ColorIn Date Interpolator MatrixInterpola NumObjecPointI Rectang ISchedu Parallel Pause Scheduler Sequen Transition Transitioner Transiti Tween Co DelimGraphMLCon IData JSONCon DataField DataSc Dat DataSource Data DataUtil DirtySprite LineS RectSprite , 2011 .

[39]  Karsten Klein,et al.  High-Quality Ultra-Compact Grid Layout of Grouped Networks , 2016, IEEE Transactions on Visualization and Computer Graphics.

[40]  Y. Koren,et al.  Dig-CoLa: directed graph layout through constrained energy minimization , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[41]  Fei-Fei Li,et al.  Visualizing and Understanding Recurrent Networks , 2015, ArXiv.

[42]  Alexander M. Rush,et al.  Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks , 2016, ArXiv.

[43]  Kozo Sugiyama,et al.  Visualization of structural information: automatic drawing of compound digraphs , 1991, IEEE Trans. Syst. Man Cybern..

[44]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[45]  Daniel W. Archambault,et al.  The Readability of Path‐Preserving Clusterings of Graphs , 2010, Comput. Graph. Forum.

[46]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[47]  Kim Marriott,et al.  IPSep-CoLa: An Incremental Procedure for Separation Constraint Layout of Graphs , 2006, IEEE Transactions on Visualization and Computer Graphics.

[48]  Emden R. Gansner,et al.  A Technique for Drawing Directed Graphs , 1993, IEEE Trans. Software Eng..

[49]  Zheng Zhang,et al.  MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems , 2015, ArXiv.

[50]  Michael Balzer,et al.  Level-of-detail visualization of clustered graph layouts , 2007, 2007 6th International Asia-Pacific Symposium on Visualization.

[51]  Ulrik Brandes,et al.  Fast and Simple Horizontal Coordinate Assignment , 2001, GD.

[52]  Michael Jünger,et al.  Exact and Heuristic Algorithms for 2-Layer Straightline Crossing Minimization , 1995, GD.