DRAGON: Dynamic Recurrent Accelerator for Graph Online Convolution

Despite the extraordinary applicative potentiality that dynamic graph inference may entail, its practical-physical implementation has been a topic seldom explored in literature. Although graph inference through neural networks has received plenty of algorithmic innovation, its transfer to the physical world has not found similar development. This is understandable since the most preeminent Euclidean acceleration techniques from CNN have little implication in the non-Euclidean nature of relational graphs. Instead of coping with the challenges arising from forcing naturally sparse structures into more inflexible stochastic arrangements, in DRAGON, we embrace this characteristic in order to promote acceleration. Inspired by high-performance computing approaches like Parallel Multi-moth Flame Optimization for Link Prediction (PMFO-LP), we propose and implement a novel efficient architecture, capable of producing similar speed-up and performance than baseline but at a fraction of its hardware requirements and power consumption. We leverage the hidden parallelistic capacity of our previously developed static graph convolutional processor ACE-GCN and expanded it with RNN structures, allowing the deployment of a multi-processing network referenced around a common pool of proximity-based centroids. Experimental results demonstrate outstanding acceleration. In comparison with the fastest CPU-based software implementation available in the literature, DRAGON has achieved roughly 191× speed-up. Under the largest configuration and dataset, DRAGON was also able to overtake a more power-hungry PMFO-LP by almost 1.59× in speed, and at around 89.59% in power efficiency. More importantly than raw acceleration, we demonstrate the unique functional qualities of our approach as a flexible and fault-tolerant solution that makes it an interesting alternative for an anthology of applicative scenarios.

[1]  Guoyong Shi,et al.  ACE-GCN: A Fast Data-driven FPGA Accelerator for GCN Embedding , 2021, ACM Trans. Reconfigurable Technol. Syst..

[2]  Md. Jakir Hossain,et al.  State Estimation in Smart Grids Using Temporal Graph Convolution Networks , 2021, 2021 North American Power Symposium (NAPS).

[3]  Olfa Kanoun,et al.  Design and implementation of a cloud-based event-driven architecture for real-time data processing in wireless sensor networks , 2021, The Journal of Supercomputing.

[4]  Baochun Li,et al.  Adversarial Attacks on Link Prediction Algorithms Based on Graph Neural Networks , 2020, AsiaCCS.

[5]  Sander Stuijk,et al.  NERO: A Near High-Bandwidth Memory Stencil Accelerator for Weather Prediction Modeling , 2020, 2020 30th International Conference on Field-Programmable Logic and Applications (FPL).

[6]  Christos-Savvas Bouganis,et al.  Caffe Barista: Brewing Caffe with FPGAs in the Training Loop , 2020, 2020 30th International Conference on Field-Programmable Logic and Applications (FPL).

[7]  Zhifeng Bao,et al.  Temporal Network Representation Learning via Historical Neighborhoods Aggregation , 2020, 2020 IEEE 36th International Conference on Data Engineering (ICDE).

[8]  Chang Xu,et al.  K-Core Based Temporal Graph Convolutional Network for Dynamic Graphs , 2020, IEEE Transactions on Knowledge and Data Engineering.

[9]  Dongrui Fan,et al.  HyGCN: A GCN Accelerator with Hybrid Architecture , 2020, 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[10]  Huawei Li,et al.  EnGN: A High-Throughput and Energy-Efficient Accelerator for Large Graph Neural Networks , 2019, IEEE Transactions on Computers.

[11]  Antonino Tumeo,et al.  AWB-GCN: A Graph Convolutional Network Accelerator with Runtime Workload Rebalancing , 2019, 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[12]  Philip T. G. Jackson,et al.  Temporal Neighbourhood Aggregation: Predicting Future Links in Temporal Graphs via Recurrent Variational Graph Convolutions , 2019, 2019 IEEE International Conference on Big Data (Big Data).

[13]  Gengxin Zhang,et al.  Interference alignment schemes for k-user interference channel based on manifold optimization , 2019, EURASIP J. Wirel. Commun. Netw..

[14]  Ahmad Sharieh,et al.  Multi-moth flame optimization for solving the link prediction problem in complex networks , 2019, Evolutionary Intelligence.

[15]  Marco Fiore,et al.  Multi-Service Mobile Traffic Forecasting via Convolutional Long Short-Term Memories , 2019, 2019 IEEE International Symposium on Measurements & Networking (M&N).

[16]  Hao Wang,et al.  Acceleration of LSTM With Structured Pruning Method on FPGA , 2019, IEEE Access.

[17]  Nadeem Javaid,et al.  An Efficient CNN and KNN Data Analytics for Electricity Load Forecasting in the Smart Grid , 2019, AINA Workshops.

[18]  Ido Guy,et al.  Node Embedding over Temporal Graphs , 2019, IJCAI.

[19]  Qi Xuan,et al.  E-LSTM-D: A Deep Learning Framework for Dynamic Network Link Prediction , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[20]  Gong Zhang,et al.  GCN-GAN: A Non-linear Temporal Link Prediction Model for Weighted Dynamic Networks , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[21]  Han Zhang,et al.  Distributed stochastic gradient descent for link prediction in signed social networks , 2019, EURASIP J. Adv. Signal Process..

[22]  Wenwu Zhu,et al.  Deep Learning on Graphs: A Survey , 2018, IEEE Transactions on Knowledge and Data Engineering.

[23]  Aijun An,et al.  dynnode2vec: Scalable Dynamic Network Embedding , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[24]  Shibao Li,et al.  Similarity-based future common neighbors model for link prediction in complex networks , 2018, Scientific Reports.

[25]  Rémy Cazabet,et al.  Systematic Biases in Link Prediction: comparing heuristic and graph embedding based methods , 2018, COMPLEX NETWORKS.

[26]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[27]  Palash Goyal,et al.  dyngraph2vec: Capturing Network Dynamics using Dynamic Graph Representation Learning , 2018, Knowl. Based Syst..

[28]  Xinming Huang,et al.  ChipNet: Real-Time LiDAR Processing for Drivable Region Segmentation on an FPGA , 2018, IEEE Transactions on Circuits and Systems I: Regular Papers.

[29]  Zhengyang Wang,et al.  Large-Scale Learnable Graph Convolutional Networks , 2018, KDD.

[30]  Tarek M. Taha,et al.  Accelerating Inference In Long Short-Term Memory Neural Networks , 2018, NAECON 2018 - IEEE National Aerospace and Electronics Conference.

[31]  Yan Liu,et al.  DynGEM: Deep Embedding Method for Dynamic Graphs , 2018, ArXiv.

[32]  Philip S. Yu,et al.  Deep Dynamic Network Embedding for Link Prediction , 2018, IEEE Access.

[33]  Xiang Cheng,et al.  Mobile Demand Forecasting via Deep Graph-Sequence Spatiotemporal Modeling in Cellular Networks , 2018, IEEE Internet of Things Journal.

[34]  Ryan A. Rossi,et al.  Continuous-Time Dynamic Network Embeddings , 2018, WWW.

[35]  Yoshihiro Yamanishi,et al.  Dual Convolutional Neural Network for Graph of Graphs Link Prediction , 2018, ArXiv.

[36]  Soheil Ghiasi,et al.  Ristretto: A Framework for Empirical Study of Resource-Efficient Inference in Convolutional Neural Networks , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[37]  Amrinder Kaur,et al.  Distribution transformer monitoring for smart grid using FPGA: a case study implementation in Indian conditions , 2017 .

[38]  Bin Song,et al.  Data-driven vs. model-driven: Fast face sketch synthesis , 2017, Neurocomputing.

[39]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[40]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[41]  Song Han,et al.  Deep compression and EIE: Efficient inference engine on compressed deep neural network , 2016, 2016 IEEE Hot Chips 28 Symposium (HCS).

[42]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[43]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[44]  Seyed Mohammad Mirjalili,et al.  Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm , 2015, Knowl. Based Syst..

[45]  Amy Loutfi,et al.  Data-Driven Rule Mining and Representation of Temporal Patterns in Physiological Sensor Data , 2015, IEEE Journal of Biomedical and Health Informatics.

[46]  Jon Rokne,et al.  Encyclopedia of Social Network Analysis and Mining , 2014, Springer New York.

[47]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[48]  Pabitra Mitra,et al.  Link Prediction Using Power Law Clique Distribution and Common Edges Distribution , 2013, PReMI.

[49]  T. Hendershott,et al.  High Frequency Trading and Price Discovery , 2013, SSRN Electronic Journal.

[50]  John W. Lockwood,et al.  A Low-Latency Library in FPGA Hardware for High-Frequency Trading (HFT) , 2012, 2012 IEEE 20th Annual Symposium on High-Performance Interconnects.

[51]  Heiner Litz,et al.  High Frequency Trading Acceleration Using FPGAs , 2011, 2011 21st International Conference on Field Programmable Logic and Applications.

[52]  Farzin Amzajerdian,et al.  Lidar systems for precision navigation and safe landing on planetary bodies , 2011, Other Conferences.

[53]  David J. Hill,et al.  Anomaly detection in streaming environmental sensor data: A data-driven modeling approach , 2010, Environ. Model. Softw..

[54]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[55]  Céline Robardet,et al.  Constraint-Based Pattern Mining in Dynamic Graphs , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[56]  Srinivasan Parthasarathy,et al.  Local Probabilistic Models for Link Prediction , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[57]  Walter Willinger,et al.  Sampling Techniques for Large, Dynamic Graphs , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[58]  Christos Faloutsos,et al.  Graphs over time: densification laws, shrinking diameters and possible explanations , 2005, KDD '05.

[59]  Eros Gian Alessandro Pasero,et al.  Embedded hardware architecture for statistical rain forecast , 2005 .

[60]  Lei Feng,et al.  A FPGA-Oriented Quantization Scheme for MobileNet-SSD , 2019, Advances in Intelligent Information Hiding and Multimedia Signal Processing.

[61]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[62]  Apurva Narayan,et al.  Learning Graph Dynamics using Deep Neural Networks , 2018 .

[63]  D. D. Šiljak,et al.  Dynamic Graphs , 2018, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[64]  Kamesh Munagala,et al.  Data-Driven Processing in Sensor Networks , 2007, CIDR.

[65]  J. Dongarra,et al.  SCOP3: A Rough Guide to Scientific Computing On the PlayStation 3 , 2007 .