Q-Learning Supplemented Response Based Crowdsensing Framework for Resource Constrained Devices

In mobile crowdsensing, the most significant challenge is to enable smart devices to perform various sensing tasks for diverse goal-oriented applications. This can be accomplished by the interaction of task owners with smart devices via a specific platform (application interface) to influence their acceptance for task completion, employing various incentive schemes and techniques mentioned in the existing literatures. However, it becomes critical to handle distinct energy restrictions of participating devices and appropriately assign task loads based upon their capabilities that have mostly been overlooked, even more so in an unknown interaction environment. In this paper we address this issue first by evaluating an optimal task-load assignment that maximizes a participating resource constraint node’s utility at a resourceful node (broker), and then modeling a distributed Q-learning framework of crowdsensing to improve the cumulative reward for participating nodes. Simulation results show that the proposed algorithm converges quickly for the designed framework, and is very efficient to employ.