An IoT-Cloud Based Solution for Real-Time and Batch Processing of Big Data: Application in Healthcare

With the large use of Internet of Things (IoT) today, everything around us seems to generate data. The ever increasing number of connected things or objects (IoT) is coupled with a growing volume of data generated at a continually increasing rate. Especially where data is big or there is a need to process it, cloud infrastructures, with their scalability and easy access, are becoming the solution of choice for storage and processing. In the context of healthcare applications, where medical sensors collect health data from patients and send it to the cloud, two issues frequently appear in relation to “Big Data”. The first issue is related to real-time analysis introduced by the increasing velocity at which data is generated especially from connected devices (IoT). This data should be analyzed continuously in real-time in order to take appropriate actions regarding the patient’s care plan. Moreover, medical data accumulated from different patients over time constitutes an important training dataset that can be used to train machine learning models in order to perform smarter disease prediction and treatment. This gives rise to another issue regarding long-term batch processing of often huge volumes of stored data. To deal with these issues, we propose an IoT-Cloud based framework for real-time and batch processing of Big Data in the healthcare domain. We implement the proposed solution on Amazon Cloud operator known as Amazon Web Services (AWS) and use a Raspberry pi as an IoT device to generate data in real time. We test the solution with the specific application of ECG monitoring and abnormality reporting. We analyze the performance of the implemented system in terms of response time by varying the velocity and volume of the analyzed data. We also discuss how the cloud resources should be provisioned in order to guarantee processing performance for both long-term and real-time scenarios. To ensure a good tradeoff between cost and processing performance, resources provision should be adapted to the exact needs and characteristics of the considered application.