Leveraging In-Transit Computational Capabilities in Federated Ecosystems

Software-defined networks (SDNs) have proven to be an efficacious tool for undertaking complex data analysis and manipulation within data intensive applications. SDN technology allows us to separate the data path from the control path, enabling in-network processing capabilities to be supported as data is migrated across the network. We propose to leverage software defined networking to gain control over the data transport service with the purpose of dynamically establishing data routes such that we can opportunistically exploit the latent computational capabilities located along the network path. This strategy allows us to minimize waiting times at the destination data center and to cope with spikes in demand for computational capability. We validate our approach using a smart building application in a multi cloud infrastructure. We show how the in-transit processing strategy increases the computational capabilities of the infrastructure and influences the percentage of job completion without significantly impacting costs and overheads.

[1]  Manish Parashar,et al.  CometCloud: Enabling Software-Defined Federations for End-to-End Application Workflows , 2015, IEEE Internet Computing.

[2]  Laurent Lefèvre,et al.  Active Networking Support for the Grid , 2001, IWAN.

[3]  Zhi Liu,et al.  LiveCloud: A lucid orchestrator for cloud datacenters , 2012, 4th IEEE International Conference on Cloud Computing Technology and Science Proceedings.

[4]  Sakir Sezer,et al.  Sdn Security: A Survey , 2013, 2013 IEEE SDN for Future Networks and Services (SDN4FNS).

[5]  Jason Lee,et al.  Intra and Interdomain Circuit Provisioning Using the OSCARS Reservation System , 2006, 2006 3rd International Conference on Broadband Communications, Networks and Systems.

[6]  Nick McKeown,et al.  OpenFlow: enabling innovation in campus networks , 2008, CCRV.

[7]  Chase Qishi Wu,et al.  Ultrascience net: network testbed for large-scale science applications , 2005, IEEE Communications Magazine.

[8]  Yacine Rezgui,et al.  Cloud Supported Building Data Analytics , 2014, 2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[9]  Thierry Turletti,et al.  A Survey of Software-Defined Networking: Past, Present, and Future of Programmable Networks , 2014, IEEE Communications Surveys & Tutorials.

[10]  Fan Zhang,et al.  Combining in-situ and in-transit processing to enable extreme-scale scientific analysis , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[11]  Anees Shaikh,et al.  Programming your network at run-time for big data applications , 2012, HotSDN '12.

[12]  Guofei Gu,et al.  Attacking software-defined networks: a first feasibility study , 2013, HotSDN '13.

[13]  Michiaki Hayashi,et al.  Sustainable Network Resource Management System for Virtual Private Clouds , 2010, 2010 IEEE Second International Conference on Cloud Computing Technology and Science.

[14]  Preeti Singh,et al.  Application-aware aggregation and traffic engineering in a converged packet-circuit network , 2011, 2011 Optical Fiber Communication Conference and Exposition and the National Fiber Optic Engineers Conference.

[15]  Min Zhu,et al.  B4: experience with a globally-deployed software defined wan , 2013, SIGCOMM.

[16]  Fan Zhang,et al.  POSTER: Leveraging deep memory hierarchies for data staging in coupled data-intensive simulation workflows , 2014, 2014 IEEE International Conference on Cluster Computing (CLUSTER).

[17]  Manish Parashar,et al.  Integrating Software Defined Networks within a Cloud Federation , 2015, 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[18]  Jeffrey F. Naughton,et al.  A software-defined networking based approach for performance management of analytical queries on distributed data stores , 2014, SIGMOD Conference.

[19]  Scott Klasky,et al.  Examples of in transit visualization , 2011, PDAC '11.

[20]  Zhen Li,et al.  Comet: a scalable coordination space for decentralized distributed environments , 2005, Second International Workshop on Hot Topics in Peer-to-Peer Systems.

[21]  Yu Xie,et al.  Federated Computing for the Masses--Aggregating Resources to Tackle Large-Scale Engineering Problems , 2014, Computing in Science & Engineering.

[22]  Mon-Yen Luo,et al.  Software Defined Networking across Distributed Datacenters over Cloud , 2013, 2013 IEEE 5th International Conference on Cloud Computing Technology and Science.

[23]  David Wetherall,et al.  Towards an active network architecture , 1996, CCRV.