HiperView: real-time monitoring of dynamic behaviors of high-performance computing centers

This paper presents HiperView, a visual analytics framework monitoring and characterizing the health status of high-performance computing systems through a RESTful interface in real time. The primary objectives of this visual analytical system are: (1) to provide a graphical interface for tracking the health status of a large number of data center hosts in real-time statistics, (2) to help users visually analyze unusual behavior of a series of events that may have temporal and spatial correlation, and (3) to assist in performing preliminary troubleshooting and maintenance with a visual layout that reflects the actual physical locations. Two use cases were analyzed in detail to assess the effectiveness of the HiperView on a medium-scale, Redfish-enabled production high-performance computing system with a total of 10 racks and 467 hosts. The visualization apparatus has been proven to offer the necessary support for system automation and control. Our framework’s visual components and interfaces are designed to potentially handle a larger-scale data center of thousands of hosts with hundreds of various health services per host.

[1]  Leland Wilkinson,et al.  Visualizing Big Data Outliers Through Distributed Aggregation , 2018, IEEE Transactions on Visualization and Computer Graphics.

[2]  Peter Zadrozny,et al.  Big Data Analytics Using Splunk: Deriving Operational Intelligence from Social Media, Machine Data, Existing Data Warehouses, and Other Real-Time Streaming Sources , 2013 .

[3]  Jie Li,et al.  MonSTer: An Out-of-the-Box Monitoring Tool for High Performance Computing Systems , 2020, 2020 IEEE International Conference on Cluster Computing (CLUSTER).

[4]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[5]  Jeffrey Heer,et al.  SpanningAspectRatioBank Easing FunctionS ArrayIn ColorIn Date Interpolator MatrixInterpola NumObjecPointI Rectang ISchedu Parallel Pause Scheduler Sequen Transition Transitioner Transiti Tween Co DelimGraphMLCon IData JSONCon DataField DataSc Dat DataSource Data DataUtil DirtySprite LineS RectSprite , 2011 .

[6]  T. H. Tse,et al.  5W+1H pattern: A perspective of systematic mapping studies and a case study on cloud software testing , 2016, J. Syst. Softw..

[7]  Robert L. Grossman,et al.  High-Dimensional Visual Analytics: Interactive Exploration Guided by Pairwise Views of Point Distributions , 2006, IEEE Transactions on Visualization and Computer Graphics.

[8]  M Joan Saary,et al.  Radar plots: a useful way for presenting multivariate health care data. , 2008, Journal of clinical epidemiology.

[9]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[10]  Leland Wilkinson,et al.  TimeSeer: Scagnostics for High-Dimensional Time Series , 2013, IEEE Transactions on Visualization and Computer Graphics.

[11]  Peter Zadrozny,et al.  Big Data Analytics Using Splunk , 2013, Apress.

[12]  John T. Stasko,et al.  The Science of Interaction , 2009, Inf. Vis..

[13]  David E. Culler,et al.  The ganglia distributed monitoring system: design, implementation, and experience , 2004, Parallel Comput..

[14]  Jon Stearley,et al.  Bridging the Gaps: Joining Information Sources with Splunk , 2010, SLAML.

[15]  Leland Wilkinson,et al.  TimeExplorer: Similarity Search Time Series by Their Signatures , 2013, ISVC.

[16]  Ben Shneiderman,et al.  A Rank-by-Feature Framework for Unsupervised Multidimensional Data Exploration Using Low Dimensional Projections , 2004 .

[17]  Daniel A. Keim,et al.  Information Visualization : Scope, Techniques and Opportunities for Geovisualization , 2004 .

[18]  Silvia Miksch,et al.  Characterizing Guidance in Visual Analytics , 2017, IEEE Transactions on Visualization and Computer Graphics.

[19]  William E. Allcock,et al.  Challenges of HPC monitoring , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[20]  Rajkumar Buyya,et al.  PARMON: a portable and scalable monitoring system for clusters , 2000 .

[21]  David Carasso,et al.  Exploring Splunk , 2012 .

[22]  Leland Wilkinson,et al.  ScagExplorer: Exploring Scatterplots by Their Scagnostics , 2014, 2014 IEEE Pacific Visualization Symposium.

[23]  Tamara Munzner,et al.  MizBee: A Multiscale Synteny Browser , 2009, IEEE Transactions on Visualization and Computer Graphics.

[24]  Yong Chen,et al.  RadarViewer : Visualizing the dynamics of multivariate data , 2020, PEARC.

[25]  Ngan V. T. Nguyen,et al.  HiperViz: Interactive visualization of CPU Temperatures in High Performance Computing Centers , 2019, PEARC.

[26]  Daniel A. Keim,et al.  Viewing Visual Analytics as Model Building , 2018, Comput. Graph. Forum.

[27]  Leland Wilkinson,et al.  Transforming Scagnostics to Reveal Hidden Features , 2014, IEEE Transactions on Visualization and Computer Graphics.

[28]  Julian M. Kunkel,et al.  Real-Time I/O-Monitoring of HPC Applications with SIOX, Elasticsearch, Grafana and FUSE , 2017, ISC Workshops.

[29]  James R. Eagan,et al.  Low-level components of analytic activity in information visualization , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[30]  R. Grossman,et al.  Graph-theoretic scagnostics , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[31]  Wolfgang Barth,et al.  Nagios: System and Network Monitoring , 2006 .

[32]  Hans-Peter Seidel,et al.  An Edge-Bundling Layout for Interactive Parallel Coordinates , 2014, 2014 IEEE Pacific Visualization Symposium.

[33]  Gennady L. Andrienko,et al.  Exploratory spatio-temporal visualization: an analytical review , 2003, J. Vis. Lang. Comput..