Performance evaluation of IoT middleware

Connected objects in a billion scale are expected in the Internet of Things (IoT). These objects (things) are interacting autonomously, with minimal human intervention and connected to the Internet. Most of these objects are constrained in terms of resources, meaning that their intelligence is delegated to a smarter player, a software, identified as middleware. It is present in most IoT scenarios because it solves interoperability problems, allowing previously incompatible devices to communicate, while also integrates and makes decisions based on collected data. This paper presents a performance evaluation study of open-source middleware solutions, including a proprietary solution developed at Inatel for the Inatel Smart Campus. Performance metrics, both qualitative and quantitative to evaluate middleware solutions objectively are also proposed. The results are analyzed and it is concluded that the proposed metrics are well adjusted for this type of platforms and can play a key role when choosing the best middleware solution to deploy in a given IoT solution. Sitewhere is the middleware platform that obtained better performance in the study. An overview of IoT middleware and their importance was highlighted and presented.Proposal of qualitative and quantitative metrics to objectively evaluate the performance of IoT middleware solutions.A performance evaluation study in a real scenario of open-source middleware solutions is conducted using the proposed metrics.Invaluable learned lessons during the study are included and presented for readers.