Nurture: Notifying Users at the Right Time Using Reinforcement Learning

User interaction is an essential part of many mobile devices such as smartphones and wrist bands. Only by interacting with the user can these devices deliver services, enable proper configurations, and learn user preferences. Push notifications are the primary method used to attract user attention in modern devices. However, these notifications can be ineffective and even irritating if they prompt the user at an inappropriate time. The discontent is exacerbated by the large number of applications that target limited user attention. We propose a reinforcement learning-based personalization technique, called Nurture, which automatically identifies the appropriate time to send notifications for a given user context. Through simulations with the crowd-sourcing platform Amazon Mechanical Turk, we show that our approach successfully learns user preferences and significantly improves the rate of notification responses.

[1]  Niels Henze,et al.  In-situ investigation of notifications in multi-device environments , 2016, UbiComp.

[2]  Brian P. Bailey,et al.  Oasis: A framework for linking notification delivery to the perceptual structure of goal-directed tasks , 2010, TCHI.

[3]  Mirco Musolesi,et al.  PrefMiner: mining user's preferences for intelligent mobile notification management , 2016, UbiComp.

[4]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[5]  Lawrence B. Holder,et al.  Thyme: Improving Smartphone Prompt Timing Through Activity Awareness , 2017, 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA).

[6]  Takanori Ichikawa,et al.  Attention and engagement-awareness in the wild: A large-scale study with adaptive notifications , 2017, 2017 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[7]  Tadashi Okoshi,et al.  Reducing users' perceived mental effort due to interruptive notifications in multi-device mobile environments , 2015, UbiComp.

[8]  Gert R. G. Lanckriet,et al.  Recognizing Detailed Human Context in the Wild from Smartphones and Smartwatches , 2016, IEEE Pervasive Computing.

[9]  Mirco Musolesi,et al.  Designing content-driven intelligent notification mechanisms for mobile applications , 2015, UbiComp.

[10]  Susan R. Fussell,et al.  Intelligent Interruption Management using Electro Dermal Activity based Physiological Sensor for Collaborative Sensemaking , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[11]  Martin Pielot,et al.  An in-situ study of mobile phone notifications , 2014, MobileHCI '14.

[12]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[13]  Feng Guo,et al.  Driver crash risk factors and prevalence evaluation using naturalistic driving data , 2016, Proceedings of the National Academy of Sciences.

[14]  Mirco Musolesi,et al.  Intelligent Notification Systems: A Survey of the State of the Art and Research Challenges , 2017, ArXiv.

[15]  Moushumi Sharmin,et al.  Assessing the availability of users to engage in just-in-time intervention in the natural environment , 2014, UbiComp.

[16]  Daniel Moldovan,et al.  Elastic systems: Towards cyber-physical ecosystems of people, processes, and things , 2018, Comput. Stand. Interfaces.

[17]  Jason Flinn,et al.  The Case for Operating System Management of User Attention , 2015, HotMobile.