A cloud computing based system for cyber security management

The exponential increase of cyber security has led to an ever-increasing accumulation of big network data for cyber security applications. The big data analysis for cyber security management presents challenges in data capturing, storing and processing. To address these challenges, in this paper we develop a cloud computing based system for cyber security management to fasten the analysis process of big network data. Our developed system is built on the MapReduce framework and consists of end-user devices, cloud infrastructure and a monitoring centre. To make our proposed system efficient, we introduce two key function modules of our system: data storage module and task scheduling module. We conduct the system implementation using Apache Hadoop, and our implemented system consists of data collection, data normalisation, data computation and data visualisation. Using ranking and aggregation as primitives for performing cyber security management, we conducted extensive experiments to show the effectiveness of our developed system. We also discuss how to extend our proposed system to other applications.

[1]  Alan L. Cox,et al.  The Hadoop distributed filesystem: Balancing portability and performance , 2010, 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS).

[2]  Christos Doulkeridis,et al.  On saying "enough already!" in MapReduce , 2012, Cloud-I '12.

[3]  Rajeev Gandhi,et al.  Kahuna: Problem diagnosis for Mapreduce-based cloud computing environments , 2010, 2010 IEEE Network Operations and Management Symposium - NOMS 2010.

[4]  Kristin E. Lauter,et al.  Cryptographic Cloud Storage , 2010, Financial Cryptography Workshops.

[5]  Srinath T. V. Setty,et al.  Depot: Cloud Storage with Minimal Trust , 2010, TOCS.

[6]  Ching-Hsien Hsu,et al.  Cloud Computing and Big Data , 2015, Lecture Notes in Computer Science.

[7]  Paulo Marques,et al.  Flood: elastic streaming MapReduce , 2010, DEBS '10.

[8]  Guobin Xu,et al.  A study of malware detection on smart mobile devices , 2013, Defense, Security, and Sensing.

[9]  Roberto V. Zicari,et al.  Big Data: Challenges and Opportunities , 2013 .

[10]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[11]  Ibrahim Aljarah,et al.  Towards a scalable intrusion detection system based on parallel PSO clustering using mapreduce , 2013, GECCO.

[12]  Yoshio Turner,et al.  Presented as part of the 5th USENIX Workshop on Hot Topics in Cloud Computing , 2013 .

[13]  Hakim Weatherspoon,et al.  RACS: a case for cloud storage diversity , 2010, SoCC '10.

[14]  Moni Naor,et al.  Job Scheduling Strategies for Parallel Processing , 2017, Lecture Notes in Computer Science.

[15]  Chuck Lam,et al.  Hadoop in Action , 2010 .

[16]  Dipti J. Suryawanshi,et al.  Traffic Measurement and Analysis with Hadoop , 2013 .

[17]  Vinicius Cardoso Garcia,et al.  Measuring Distributed Applications through MapReduce and Traffic Analysis , 2012, 2012 IEEE 18th International Conference on Parallel and Distributed Systems.

[18]  Geoffrey C. Fox,et al.  MapReduce in the Clouds for Science , 2010, 2010 IEEE Second International Conference on Cloud Computing Technology and Science.

[19]  Tom White,et al.  Hadoop: The Definitive Guide , 2009 .

[20]  Wenying Zeng,et al.  Research on cloud storage architecture and key technologies , 2009, ICIS.

[21]  Robert J. Connor,et al.  Data challenges and opportunities , 2014 .

[22]  Radu State,et al.  BotCloud: Detecting botnets using MapReduce , 2011, 2011 IEEE International Workshop on Information Forensics and Security.

[23]  Huan Liu,et al.  Cloud MapReduce: A MapReduce Implementation on Top of a Cloud Operating System , 2011, 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[24]  Xin Jiang,et al.  Cloud computing-based forensic analysis for collaborative network security management system , 2013 .

[25]  Yi Pan,et al.  M2M: A simple Matlab-to-MapReduce translator for cloud computing , 2013 .

[26]  Charlie Catlett.  Cloud computing and big data , 2013 .

[27]  Yun Tian,et al.  Improving MapReduce performance through data placement in heterogeneous Hadoop clusters , 2010, 2010 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum (IPDPSW).

[28]  Himabindu Pucha,et al.  Towards Optimizing Hadoop Provisioning in the Cloud , 2009, HotCloud.

[29]  Jan Tore Morken,et al.  Distributed NetFlow Processing Using the Map-Reduce Model , 2010 .

[30]  Wei Yu,et al.  On behavior-based detection of malware on Android platform , 2013, 2013 IEEE Global Communications Conference (GLOBECOM).

[31]  Hairong Kuang,et al.  The Hadoop Distributed File System , 2010, 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST).

[32]  Eugene Marinelli,et al.  Hyrax: Cloud Computing on Mobile Devices using MapReduce , 2009 .

[33]  Jie Wu,et al.  Hierarchical attribute-based encryption for fine-grained access control in cloud storage services , 2010, CCS '10.

[34]  Youngseok Lee,et al.  Detecting DDoS attacks with Hadoop , 2011, CoNEXT '11 Student.

[35]  Roland H. C. Yap,et al.  A MapReduce-Based Maximum-Flow Algorithm for Large Small-World Network Graphs , 2011, 2011 31st International Conference on Distributed Computing Systems.

[36]  Thomas Sandholm,et al.  Dynamic Proportional Share Scheduling in Hadoop , 2010, JSSPP.

[37]  Ya Wang,et al.  Cloud Storage as the Infrastructure of Cloud Computing , 2010, 2010 International Conference on Intelligent Computing and Cognitive Informatics.

[38]  Alexandru Iosup,et al.  A Performance Analysis of EC2 Cloud Computing Services for Scientific Computing , 2009, CloudComp.

[39]  Benny Pinkas,et al.  Side Channels in Cloud Services: Deduplication in Cloud Storage , 2010, IEEE Security & Privacy.

[40]  Gerhard Weikum,et al.  Solving Linear Programs in MapReduce , 2011 .