Graph Drawing by Stochastic Gradient Descent

A popular method of force-directed graph drawing is multidimensional scaling using graph-theoretic distances as input. We present an algorithm to minimize its energy function, known as stress, by using stochastic gradient descent (SGD) to move a single pair of vertices at a time. Our results show that SGD can reach lower stress levels faster and more consistently than majorization, without needing help from a good initialization. We then show how the unique properties of SGD make it easier to produce constrained layouts than previous approaches. We also show how SGD can be directly applied within the sparse stress approximation of Ortmann et al. [1], making the algorithm scalable up to large graphs.

[1]  J. Leeuw.  Convergence of the majorization method for multidimensional scaling , 1988 .

[2]  Yifan Hu,et al.  Efficient, High-Quality Force-Directed Graph Drawing , 2006 .

[3]  Joshua B. Tenenbaum,et al.  Sparse multidimensional scaling using land-mark points , 2004 .

[4]  Michael Jünger,et al.  Drawing Large Graphs with a Potential-Field-Based Multilevel Algorithm , 2004, GD.

[5]  Tim Dwyer,et al.  Scalable, Versatile and Simple Constrained Graph Layout , 2009, Comput. Graph. Forum.

[6]  Jeffrey Heer,et al.  D³ Data-Driven Documents , 2011, IEEE Transactions on Visualization and Computer Graphics.

[7]  Ulrik Brandes,et al.  Journal of Graph Algorithms and Applications More Flexible Radial Layout , 2022 .

[8]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[9]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[10]  Edward M. Reingold,et al.  Graph drawing by force‐directed placement , 1991, Softw. Pract. Exp..

[11]  Piet Hut,et al.  A hierarchical O(N log N) force-calculation algorithm , 1986, Nature.

[12]  Richard Piper,et al.  An overview of gradient descent optimization algorithms , 2016 .

[13]  Thomas Jakobsen,et al.  Advanced Character Physics , 2003 .

[14]  Ulrik Brandes,et al.  A Sparse Stress Model , 2016, J. Graph Algorithms Appl..

[15]  Léon Bottou,et al.  Stochastic Gradient Descent Tricks , 2012, Neural Networks: Tricks of the Trade.

[16]  Yee Whye Teh,et al.  Bayesian Learning via Stochastic Gradient Langevin Dynamics , 2011, ICML.

[17]  Jeffrey Heer,et al.  SpanningAspectRatioBank Easing FunctionS ArrayIn ColorIn Date Interpolator MatrixInterpola NumObjecPointI Rectang ISchedu Parallel Pause Scheduler Sequen Transition Transitioner Transiti Tween Co DelimGraphMLCon IData JSONCon DataField DataSc Dat DataSource Data DataUtil DirtySprite LineS RectSprite , 2011 .

[18]  J. Kruskal.  Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis , 1964 .

[19]  Ulrik Brandes,et al.  An Experimental Study on Distance-Based Graph Drawing , 2009, GD.

[20]  L. Eon Bottou.  Online Learning and Stochastic Approximations , 1998 .

[21]  John Moody,et al.  Learning rate schedules for faster stochastic gradient search , 1992, Neural Networks for Signal Processing II Proceedings of the 1992 IEEE Workshop.

[22]  Yifan Hu,et al.  A Maxent-Stress Model for Graph Layout , 2012, IEEE Transactions on Visualization and Computer Graphics.

[23]  Timothy A. Davis,et al.  The university of Florida sparse matrix collection , 2011, TOMS.

[24]  Yehuda Koren,et al.  Graph Drawing by Stress Majorization , 2004, GD.

[25]  Asuman E. Ozdaglar,et al.  Why random reshuffling beats stochastic gradient descent , 2015, Mathematical Programming.

[26]  J. Kruskal.  Nonmetric multidimensional scaling: A numerical method , 1964 .

[27]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[28]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[29]  William H. Press,et al.  Numerical recipes in C , 2002 .

[30]  Yong-Xian Wang,et al.  A fast successive over-relaxation algorithm for force-directed network graph drawing , 2011, Science China Information Sciences.

[31]  Satoru Kawai,et al.  An Algorithm for Drawing General Undirected Graphs , 1989, Inf. Process. Lett..

[32]  Ning Qian,et al.  On the momentum term in gradient descent learning algorithms , 1999, Neural Networks.

[33]  Léon Bottou,et al.  On-line learning and stochastic approximations , 1999 .

[34]  Yifan Hu,et al.  Drawing Large Graphs by Low‐Rank Stress Majorization , 2012, Comput. Graph. Forum.

[35]  Ulrik Brandes,et al.  Eigensolver Methods for Progressive Multidimensional Scaling of Large Data , 2006, GD.

[36]  Furong Huang,et al.  Escaping From Saddle Points - Online Stochastic Gradient for Tensor Decomposition , 2015, COLT.

[37]  Michael C. Hout,et al.  Multidimensional Scaling , 2003, Encyclopedic Dictionary of Archaeology.