Decentralized Learning with Average Difference Aggregation for Proactive Online Social Care

The Internet and the Web are being increasingly used in proactive social care to provide people, especially the vulnerable, with a better life and services, and their derived social services generate enormous data. However, privacy concerns require the strict protection of a user's data which leads to a dilemma between pursuing better intelligent services and privacy preservation. To solve this dilemma, this paper develops a decentralized learning framework to enable proactive social care equipped with artificial intelligence and data protection. The proposed learning framework trains a secure local model for each user using their own datasets. Each user's device only shares its local model's parameters with the central model server without exposing any data, and the server integrates multiple users' models to learn a global data-free model. To improve the generalizability of the global model, we further propose a novel model aggregation algorithm, namely the average difference descent aggregation (AvgDiffAgg for short). In particular, to evaluate the effectiveness of the learning algorithm, we use a case study on the early detection and prevention of suicidal ideation, and the experiment results on four datasets derived from social media demonstrate the effectiveness of the proposed learning method.

[1]  Mark Dredze,et al.  Discovering Shifts to Suicidal Ideation from Mental Health Content in Social Media , 2016, CHI.

[2]  Changqin Quan,et al.  Examining Accumulated Emotional Traits in Suicide Blogs With an Emotion Topic Model , 2016, IEEE Journal of Biomedical and Health Informatics.

[3]  Pradeep Dubey,et al.  Distributed Deep Learning Using Synchronous Stochastic Gradient Descent , 2016, ArXiv.

[4]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[5]  Yoshua Bengio,et al.  Zero-data Learning of New Tasks , 2008, AAAI.

[6]  Vitaly Shmatikov,et al.  Privacy-preserving deep learning , 2015, Allerton.

[7]  H. Christensen,et al.  Detecting suicidality on Twitter , 2015 .

[8]  Golan Shahar,et al.  Responses to suicidal messages in an online support group: comparison between trained volunteers and lay individuals , 2012, Social Psychiatry and Psychiatric Epidemiology.

[9]  Alexander J. Smola,et al.  Scaling Distributed Machine Learning with the Parameter Server , 2014, OSDI.

[10]  Joshua Achiam,et al.  On First-Order Meta-Learning Algorithms , 2018, ArXiv.

[11]  Tianqing Zhu,et al.  Detecting Suicidal Ideation with Data Protection in Online Communities , 2019, DASFAA.

[12]  Louis-Philippe Morency,et al.  Adolescent Suicidal Risk Assessment in Clinician-Patient Interaction , 2017, IEEE Transactions on Affective Computing.

[13]  Anthony K. H. Tung,et al.  SINGA: A Distributed Deep Learning Platform , 2015, ACM Multimedia.

[14]  Hiroyuki Ohsaki,et al.  Recognizing Depression from Twitter Activity , 2015, CHI.

[15]  R. Calvo,et al.  Application of Synchronous Text-Based Dialogue Systems in Mental Health Interventions: Systematic Review , 2017, Journal of medical Internet research.

[16]  Ling Feng,et al.  Analyzing and Identifying Teens’ Stressful Periods and Stressor Events From a Microblog , 2017, IEEE Journal of Biomedical and Health Informatics.

[17]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[18]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[19]  H. Christensen,et al.  Ibobbly mobile health intervention for suicide prevention in Australian Indigenous youth: a pilot randomised controlled trial , 2017, BMJ Open.

[20]  Shirui Pan,et al.  Supervised Learning for Suicidal Ideation Detection in Online User Content , 2018, Complex..

[21]  Mir Mustafa Ali,et al.  Detecting Stress Based on Social Interactions in Social Networks , 2019 .

[22]  Pramod Viswanath,et al.  Extremal Mechanisms for Local Differential Privacy , 2014, J. Mach. Learn. Res..

[23]  Tassilo Klein,et al.  Differentially Private Federated Learning: A Client Level Perspective , 2017, ArXiv.

[24]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.

[25]  Eric Horvitz,et al.  Predicting Depression via Social Media , 2013, ICWSM.

[26]  Svetha Venkatesh,et al.  Affective and Content Analysis of Online Depression Communities , 2014, IEEE Transactions on Affective Computing.

[27]  Alexander J. Smola,et al.  Communication Efficient Distributed Machine Learning with the Parameter Server , 2014, NIPS.

[28]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[29]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[30]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[31]  Philip S. Yu,et al.  A Comprehensive Study on Social Network Mental Disorders Detection via Online Social Media Mining , 2018, IEEE Transactions on Knowledge and Data Engineering.