Fog in the Clouds

Internet of Things (IoT) has emerged as a huge paradigm shift by connecting a versatile and massive collection of smart objects to the Internet, coming to play an important role in our daily lives. Data produced by IoT devices can generate a number of computational tasks that cannot be executed locally on the IoT devices. The most common solution is offloading these tasks to external devices with higher computational and storage capabilities, usually provided by centralized servers in remote clouds or on the edge by using the fog computing paradigm. Nevertheless, in some IoT scenarios there are remote or challenging areas where it is difficult to connect an IoT network to a fog platform with appropriate links, especially if IoT devices produce a lot of data that require processing in real-time. To this purpose, in this article, we propose to use unmanned aerial vehicles (UAVs) as fog nodes. Although this idea is not new, this is the first work that considers power consumption of the computing element installed on board UAVs, which is crucial, since it may influence flight mission duration. A System Controller (SC) is in charge of deciding the number of active CPUs at runtime by maximizing an objective function weighing power consumption, job loss probability, and processing latency. Reinforcement Learning (RL) is used to support SC in its decisions. A numerical analysis is carried out in a use case to show how to use the model introduced in the article to decide the computation power of the computing element in terms of number of available CPUs and CPU clock speed, and evaluate the achieved performance gain of the proposed framework.

[1]  Yoshitaka Takahashi,et al.  Switched Batch Bernoulli Process (SBBP) and the Discrete-Time SBBP/G/1 Queue with Application to Statistical Multiplexer Performance , 1991, IEEE J. Sel. Areas Commun..

[2]  Albert Y. Zomaya,et al.  A Framework for Reinforcement-Based Scheduling in Parallel Processor Systems , 1998, IEEE Trans. Parallel Distributed Syst..

[3]  Alfio Lombardo,et al.  Modeling intramedia and intermedia relationships in multimedia network analysis through multiple timescale statistics , 2004, IEEE Transactions on Multimedia.

[4]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[5]  Gerald Tesauro,et al.  Reinforcement Learning in Autonomic Computing: A Manifesto and Case Studies , 2007, IEEE Internet Computing.

[6]  Izhak Rubin,et al.  Placement of UAVs as Communication Relays Aiding Mobile Ad Hoc Wireless Networks , 2007, MILCOM 2007 - IEEE Military Communications Conference.

[7]  A. El Saddik,et al.  Ant Colony-Based Reinforcement Learning Algorithm for Routing in Wireless Sensor Networks , 2007, 2007 IEEE Instrumentation & Measurement Technology Conference IMTC 2007.

[8]  Feng Jiang,et al.  Optimization of UAV Heading for the Ground-to-Air Uplink , 2011, IEEE Journal on Selected Areas in Communications.

[9]  Christian Wietfeld,et al.  Interference Aware Positioning of Aerial Relays for Cell Overload and Outage Compensation , 2012, 2012 IEEE Vehicular Technology Conference (VTC Fall).

[10]  Dusit Niyato,et al.  A Dynamic Offloading Algorithm for Mobile Computing , 2012, IEEE Transactions on Wireless Communications.

[11]  Yong Wang,et al.  Energy-constrained ferry route design for sparse wireless sensor networks , 2013 .

[12]  Chonho Lee,et al.  A survey of mobile cloud computing: architecture, applications, and approaches , 2013, Wirel. Commun. Mob. Comput..

[13]  Pierre St. Juste,et al.  Machine Learning-Based Runtime Scheduler for Mobile Offloading Framework , 2013, 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing.

[14]  David Lillethun,et al.  Mobile fog: a programming model for large-scale applications on the internet of things , 2013, MCC '13.

[15]  Mario Nemirovsky,et al.  Key ingredients in an IoT recipe: Fog Computing, Cloud computing, and more Fog Computing , 2014, 2014 IEEE 19th International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD).

[16]  Jing-Yang Jou,et al.  Scalable Power Management Using Multilevel Reinforcement Learning for Multiprocessors , 2014, TODE.

[17]  Ivan Stojmenovic,et al.  The Fog computing paradigm: Scenarios and security issues , 2014, 2014 Federated Conference on Computer Science and Information Systems.

[18]  Raphaël Couturier,et al.  Dynamic Frequency Scaling for Energy Consumption Reduction in Synchronous Distributed Applications , 2014, 2014 IEEE International Symposium on Parallel and Distributed Processing with Applications.

[19]  Marthony Taguinod,et al.  Policy-driven security management for fog computing: Preliminary framework and a case study , 2014, Proceedings of the 2014 IEEE 15th International Conference on Information Reuse and Integration (IEEE IRI 2014).

[20]  Jameela Al-Jaroodi,et al.  A Framework for Using Unmanned Aerial Vehicles for Data Collection in Linear Wireless Sensor Networks , 2014, J. Intell. Robotic Syst..

[21]  Anang Hudaya Muhamad Amin,et al.  Cloudlet-based cyber foraging framework for distributed video surveillance provisioning , 2014, 2014 4th World Congress on Information and Communication Technologies (WICT 2014).

[22]  Renato J. O. Figueiredo,et al.  MALMOS: Machine Learning-Based Mobile Offloading Scheduler with Online Training , 2015, 2015 3rd IEEE International Conference on Mobile Cloud Computing, Services, and Engineering.

[23]  Songqing Chen,et al.  Help your mobile applications with fog computing , 2015, 2015 12th Annual IEEE International Conference on Sensing, Communication, and Networking - Workshops (SECON Workshops).

[24]  Eui-nam Huh,et al.  Fog Computing Micro Datacenter Based Dynamic Resource Estimation and Pricing Model for IoT , 2015, 2015 IEEE 29th International Conference on Advanced Information Networking and Applications.

[25]  Mohsen Guizani,et al.  Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications , 2015, IEEE Communications Surveys & Tutorials.

[26]  Qun Li,et al.  A Survey of Fog Computing: Concepts, Applications and Issues , 2015, Mobidata@MobiHoc.

[27]  Ivan Stojmenovic,et al.  An overview of Fog computing and its security issues , 2016, Concurr. Comput. Pract. Exp..

[28]  Walid Saad,et al.  Unmanned Aerial Vehicle With Underlaid Device-to-Device Communications: Performance and Tradeoffs , 2015, IEEE Transactions on Wireless Communications.

[29]  Prem Prakash Jayaraman,et al.  Internet of Things and Edge Cloud Computing Roadmap for Manufacturing , 2016, IEEE Cloud Computing.

[30]  Antonio Pescapè,et al.  Integration of Cloud computing and Internet of Things: A survey , 2016, Future Gener. Comput. Syst..

[31]  Mohsen Guizani,et al.  Reinforcement learning for resource provisioning in the vehicular cloud , 2016, IEEE Wireless Communications.

[32]  Bin Zhang,et al.  An Adaptive Decision Making Approach Based on Reinforcement Learning for Self-Managed Cloud Applications , 2016, 2016 IEEE International Conference on Web Services (ICWS).

[33]  Rui Zhang,et al.  Wireless communications with unmanned aerial vehicles: opportunities and challenges , 2016, IEEE Communications Magazine.

[34]  Massoud Pedram,et al.  Model-Free Reinforcement Learning and Bayesian Classification in System-Level Power Management , 2016, IEEE Transactions on Computers.

[35]  Rui Zhang,et al.  Throughput Maximization for UAV-Enabled Mobile Relaying Systems , 2016, IEEE Transactions on Communications.

[36]  Ejaz Ahmed,et al.  A survey on mobile edge computing , 2016, 2016 10th International Conference on Intelligent Systems and Control (ISCO).

[37]  Harpreet S. Dhillon,et al.  Downlink Coverage Analysis for a Finite 3-D Wireless Network of Unmanned Aerial Vehicles , 2017, IEEE Transactions on Communications.

[38]  George Mastorakis,et al.  Efficient Next Generation Emergency Communications over Multi-Access Edge Computing , 2017, IEEE Communications Magazine.

[39]  Weifa Liang,et al.  Optimal Cloudlet Placement and User to Cloudlet Allocation in Wireless Metropolitan Area Networks , 2017, IEEE Transactions on Cloud Computing.

[40]  Xing Zhang,et al.  A Survey on Mobile Edge Networks: Convergence of Computing, Caching and Communications , 2017, IEEE Access.

[41]  Walid Saad,et al.  Caching in the Sky: Proactive Deployment of Cache-Enabled Unmanned Aerial Vehicles for Optimized Quality-of-Experience , 2016, IEEE Journal on Selected Areas in Communications.

[42]  Walid Saad,et al.  Mobile Unmanned Aerial Vehicles (UAVs) for Energy-Efficient Internet of Things Communications , 2017, IEEE Transactions on Wireless Communications.

[43]  Charles C. Byers,et al.  Architectural Imperatives for Fog Computing: Use Cases, Requirements, and Architectural Techniques for Fog-Enabled IoT Networks , 2017, IEEE Communications Magazine.

[44]  Osman S. Unsal,et al.  A Machine Learning Approach for Performance Prediction and Scheduling on Heterogeneous CPUs , 2017, 2017 29th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD).

[45]  Andreas Spanias,et al.  A brief survey of machine learning methods and their sensor and IoT applications , 2017, 2017 8th International Conference on Information, Intelligence, Systems & Applications (IISA).

[46]  Tarik Taleb,et al.  Mobile Edge Computing Potential in Making Cities Smarter , 2017, IEEE Communications Magazine.

[47]  Purnendu Shekhar Pandey,et al.  Machine Learning and IoT for prediction and detection of stress , 2017, 2017 17th International Conference on Computational Science and Its Applications (ICCSA).

[48]  Jameela Al-Jaroodi,et al.  UAVFog: A UAV-based fog computing for Internet of Things , 2017, 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI).

[49]  Amit P. Sheth,et al.  On Using the Intelligent Edge for IoT Analytics , 2017, IEEE Intelligent Systems.

[50]  Zdenek Becvar,et al.  Mobile Edge Computing: A Survey on Architecture and Computation Offloading , 2017, IEEE Communications Surveys & Tutorials.

[51]  Lei Wang,et al.  Optimal bit allocation for UAV-enabled mobile communication , 2017, 2017 3rd IEEE International Conference on Computer and Communications (ICCC).

[52]  Joonhyuk Kang,et al.  Mobile cloud computing with a UAV-mounted cloudlet: optimal bit allocation for communication and computation , 2016, IET Commun..

[53]  Mianxiong Dong,et al.  Learning IoT in Edge: Deep Learning for the Internet of Things with Edge Computing , 2018, IEEE Network.

[54]  Walid Saad,et al.  Online Optimization for UAV-Assisted Distributed Fog Computing in Smart Factories of Industry 4.0 , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[55]  Nan Zhao,et al.  Integrated Networking, Caching, and Computing for Connected Vehicles: A Deep Reinforcement Learning Approach , 2018, IEEE Transactions on Vehicular Technology.

[56]  Kenji Sugawara,et al.  Multiagent-Based Flexible Edge Computing Architecture for IoT , 2018, IEEE Network.

[57]  Bhaskar Krishnamachari,et al.  Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networks , 2018, IEEE Transactions on Cognitive Communications and Networking.

[58]  Rui Wang,et al.  A network traffic flow prediction with deep learning approach for large-scale metropolitan area network , 2018, NOMS 2018 - 2018 IEEE/IFIP Network Operations and Management Symposium.

[59]  Hamed Kebriaei,et al.  A multi-state Q-learning based CSMA MAC protocol for wireless networks , 2018, Wirel. Networks.

[60]  Xianfu Chen,et al.  Deep Reinforcement Learning for Resource Management in Network Slicing , 2018, IEEE Access.

[61]  Assadarat Khurat,et al.  Distinguishing Drone Types Based on Acoustic Wave by IoT Device , 2018, 2018 22nd International Computer Science and Engineering Conference (ICSEC).

[62]  Ejaz Ahmed,et al.  The Role of Edge Computing in Internet of Things , 2018, IEEE Communications Magazine.

[63]  Haijian Sun,et al.  UAV-Enabled Mobile Edge Computing: Offloading Optimization and Trajectory Design , 2018, 2018 IEEE International Conference on Communications (ICC).

[64]  Geoffrey Ye Li,et al.  Deep Reinforcement Learning Based Resource Allocation for V2V Communications , 2018, IEEE Transactions on Vehicular Technology.

[65]  Jie Xu,et al.  Energy Minimization for Wireless Communication With Rotary-Wing UAV , 2018, IEEE Transactions on Wireless Communications.

[66]  Li Li,et al.  IoT-Enabled Machine Learning for an Algorithmic Spectrum Decision Process , 2019, IEEE Internet of Things Journal.

[67]  Koichi Adachi,et al.  Radio and Computing Resource Allocation for Minimizing Total Processing Completion Time in Mobile Edge Computing , 2019, IEEE Access.

[68]  Raja Lavanya,et al.  Fog Computing and Its Role in the Internet of Things , 2019, Advances in Computer and Electrical Engineering.

[69]  Giovanni Schembra,et al.  Green wireless power transfer system for a drone fleet managed by reinforcement learning in smart industry , 2020 .