NeuroPigPen: A Scalable Toolkit for Processing Electrophysiological Signal Data in Neuroscience Applications Using Apache Pig

The recent advances in neurological imaging and sensing technologies have led to rapid increase in the volume, rate of data generation, and variety of neuroscience data. This “neuroscience Big data” represents a significant opportunity for the biomedical research community to design experiments using data with greater timescale, large number of attributes, and statistically significant data size. The results from these new data-driven research techniques can advance our understanding of complex neurological disorders, help model long-term effects of brain injuries, and provide new insights into dynamics of brain networks. However, many existing neuroinformatics data processing and analysis tools were not built to manage large volume of data, which makes it difficult for researchers to effectively leverage this available data to advance their research. We introduce a new toolkit called NeuroPigPen that was developed using Apache Hadoop and Pig data flow language to address the challenges posed by large-scale electrophysiological signal data. NeuroPigPen is a modular toolkit that can process large volumes of electrophysiological signal data, such as Electroencephalogram (EEG), Electrocardiogram (ECG), and blood oxygen levels (SpO2), using a new distributed storage model called Cloudwave Signal Format (CSF) that supports easy partitioning and storage of signal data on commodity hardware. NeuroPigPen was developed with three design principles: (a) Scalability—the ability to efficiently process increasing volumes of data; (b) Adaptability—the toolkit can be deployed across different computing configurations; and (c) Ease of programming—the toolkit can be easily used to compose multi-step data processing pipelines using high-level programming constructs. The NeuroPigPen toolkit was evaluated using 750 GB of electrophysiological signal data over a variety of Hadoop cluster configurations ranging from 3 to 30 Data nodes. The evaluation results demonstrate that the toolkit is highly scalable and adaptable, which makes it suitable for use in neuroscience applications as a scalable data processing toolkit. As part of the ongoing extension of NeuroPigPen, we are developing new modules to support statistical functions to analyze signal data for brain connectivity research. In addition, the toolkit is being extended to allow integration with scientific workflow systems. NeuroPigPen is released under BSD license at: https://sites.google.com/a/case.edu/neuropigpen/.

[1]  Samden D. Lhatoo,et al.  The Electroencephalogram in the Investigation of Epilepsy , 2012 .

[2]  R. Henry,et al.  Diffusion Tensor MR Imaging and Fiber Tractography: Theoretic Underpinnings , 2008, American Journal of Neuroradiology.

[3]  Chien-Hung Chen,et al.  Electrophysiological Signal Analysis and Visualization using Cloudwave for Epilepsy Clinical Research , 2013, MedInfo.

[4]  Karthik Ranganathan,et al.  Apache hadoop goes realtime at Facebook , 2011, SIGMOD '11.

[5]  Zheng Shao,et al.  Hive - a petabyte scale data warehouse using Hadoop , 2010, 2010 IEEE 26th International Conference on Data Engineering (ICDE 2010).

[6]  ReedBenjamin,et al.  Building a high-level dataflow system on top of Map-Reduce , 2009, VLDB 2009.

[7]  Fabrice Wendling,et al.  Computational models of epileptiform activity , 2016, Journal of Neuroscience Methods.

[8]  Sanjay Ghemawat,et al.  MapReduce: a flexible data processing tool , 2010, CACM.

[9]  Hairong Kuang,et al.  The Hadoop Distributed File System , 2010, 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST).

[10]  Kenneth A. Loparo,et al.  A scalable neuroinformatics data flow for electrophysiological signals using MapReduce , 2015, Front. Neuroinform..

[11]  Christopher Olston,et al.  Building a HighLevel Dataflow System on top of MapReduce: The Pig Experience , 2009, Proc. VLDB Endow..

[12]  Norden E. Huang,et al.  Ensemble Empirical Mode Decomposition: a Noise-Assisted Data Analysis Method , 2009, Adv. Data Sci. Adapt. Anal..

[13]  J. Thiran,et al.  Understanding diffusion MR imaging techniques: from scalar diffusion-weighted imaging to diffusion tensor imaging and beyond. , 2006, Radiographics : a review publication of the Radiological Society of North America, Inc.

[14]  Bob Kemp,et al.  European data format ‘plus’ (EDF+), an EDF alike standard format for the exchange of physiological data , 2003, Clinical Neurophysiology.

[15]  Kathryn A. Davis,et al.  Data integration: Combined imaging and electrophysiology data in the cloud , 2016, NeuroImage.

[16]  Timothy O. Laumann,et al.  Informatics and Data Mining Tools and Strategies for the Human Connectome Project , 2011, Front. Neuroinform..

[17]  Carole A. Goble,et al.  Taverna, Reloaded , 2010, SSDBM.

[18]  H. Lüders,et al.  Modern technology calls for a modern approach to classification of epileptic seizures and the epilepsies , 2012, Epilepsia.

[19]  K. Shadan,et al.  Available online: , 2012 .

[20]  Susan Redline,et al.  Entering the era of "big data": getting our metrics right. , 2013, Sleep.

[21]  Lizhe Wang,et al.  Massively Parallel Neural Signal Processing on a Many-Core Platform , 2011, Computing in Science & Engineering.

[22]  Yao Sun,et al.  HBase, MapReduce, and Integrated Data Visualization for Processing Clinical Signal Data , 2011, AAAI Spring Symposium: Computational Physiology.

[23]  Nitin Tandon,et al.  Roles for the pre-supplementary motor area and the right inferior frontal gyrus in stopping action: Electrophysiological responses and functional and structural connectivity , 2012, NeuroImage.

[24]  Takashi Kawashima,et al.  Mapping brain activity at scale with cluster computing , 2014, Nature Methods.

[25]  Komal Shringare,et al.  Apache Hadoop Goes Realtime at Facebook , 2015 .

[26]  Michael Stonebraker,et al.  A comparison of approaches to large-scale data analysis , 2009, SIGMOD Conference.

[27]  Carole A. Goble,et al.  Taverna: a tool for building and running workflows of services , 2006, Nucleic Acids Res..

[28]  A. Scherer,et al.  From the Epilepsy Foundation , 2002, Epilepsy & Behavior.

[29]  M. Budoff,et al.  National Institute of Neurological Disorders and Stroke , 2008 .

[30]  Meng Zhao,et al.  Epilepsy and seizure ontology: towards an epilepsy informatics infrastructure for clinical research and patient care , 2014, J. Am. Medical Informatics Assoc..

[31]  François Mauguière,et al.  Clinical Manifestations of Insular Lobe Seizures: A Stereo‐electroencephalographic Study , 2004 .

[32]  Rajiv Ranjan,et al.  Parallel Processing of Massive EEG Data with MapReduce , 2012, 2012 IEEE 18th International Conference on Parallel and Distributed Systems.

[33]  Fabrice Wendling,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[34]  J D Bayliss,et al.  A virtual reality testbed for brain-computer interface research. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[35]  F. L. D. Silva,et al.  EEG signal processing , 2000, Clinical Neurophysiology.

[36]  Edward A. Lee,et al.  Scientific workflow management and the Kepler system , 2006, Concurr. Comput. Pract. Exp..

[37]  F. Mauguière,et al.  Clinical manifestations of insular lobe seizures: a stereo-electroencephalographic study , 2008, Clinical Neurophysiology.