Can Machines Read our Minds?

We explore the question of whether machines can infer information about our psychological traits or mental states by observing samples of our behaviour gathered from our online activities. Ongoing technical advances across a range of research communities indicate that machines are now able to access this information, but the extent to which this is possible and the consequent implications have not been well explored. We begin by highlighting the urgency of asking this question, and then explore its conceptual underpinnings, in order to help emphasise the relevant issues. To answer the question, we review a large number of empirical studies, in which samples of behaviour are used to automatically infer a range of psychological constructs, including affect and emotions, aptitudes and skills, attitudes and orientations (e.g. values and sexual orientation), personality, and disorders and conditions (e.g. depression and addiction). We also present a general perspective that can bring these disparate studies together and allow us to think clearly about their philosophical and ethical implications, such as issues related to consent, privacy, and the use of persuasive technologies for controlling human behaviour.

[1]  J. Hyönä,et al.  Eye fixation patterns among dyslexic and normal readers: effects of word length and word frequency. , 1995, Journal of experimental psychology. Learning, memory, and cognition.

[2]  M. M. Hulle,et al.  Event Related Potential Study of Language Interaction in Bilingual Aphasia Patients , 2018, Front. Hum. Neurosci..

[3]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[4]  Sandra Wachter,et al.  A Right to Reasonable Inferences: Re-Thinking Data Protection Law in the Age of Big Data and AI , 2018 .

[5]  Lada A. Adamic,et al.  Computational Social Science , 2009, Science.

[6]  Karolina Baras,et al.  ‘Smartphine': Supporting students' well-being according to their calendar and mood , 2016, 2016 International Multidisciplinary Conference on Computer and Energy Science (SpliTech).

[7]  Alessio Medda,et al.  Disentangling Multispectral Functional Connectivity With Wavelets , 2018, Front. Neurosci..

[8]  Yu-Hsuan Lin,et al.  Development and Validation of the Smartphone Addiction Inventory (SPAI) , 2014, PloS one.

[9]  Pan Wang,et al.  Using Support Vector Machine on EEG for Advertisement Impact Assessment , 2018, Front. Neurosci..

[10]  C. Barr Taylor,et al.  Evaluation of computerized text analysis in an Internet breast cancer support group , 2005, Comput. Hum. Behav..

[11]  D. Boyd,et al.  CRITICAL QUESTIONS FOR BIG DATA , 2012 .

[12]  S. Schwartz Chapter 7 A Proposal for Measuring Value Orientations across Nations , 2007 .

[13]  Luz Rello,et al.  Detecting readers with dyslexia using machine learning with eye tracking measures , 2015, W4A.

[14]  Lin Li,et al.  Sensing Subjective Well-Being from Social Media , 2014, AMT.

[15]  Michelle Karg,et al.  Recognition of Affect Based on Gait Patterns , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[16]  Jill Burstein,et al.  AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0 , 2004 .

[17]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[18]  Vince D. Calhoun,et al.  Machine learning of structural magnetic resonance imaging predicts psychopathic traits in adolescent offenders , 2017, NeuroImage.

[19]  D. Borsboom Measuring the mind: Conceptual issues in contemporary psychometrics , 2005 .

[20]  P. Ekman An argument for basic emotions , 1992 .

[21]  Andrew Sears,et al.  Automated stress detection using keystroke and linguistic features: An exploratory study , 2009, Int. J. Hum. Comput. Stud..

[22]  D. Eignor The standards for educational and psychological testing. , 2013 .

[23]  R. Hogan,et al.  New Talent Signals: Shiny New Objects or a Brave New World? , 2016, Industrial and Organizational Psychology.

[24]  Kimmo Kaski,et al.  Absence makes the heart grow fonder: social compensation when failure to interact risks weakening a relationship , 2016, EPJ Data Science.

[25]  David Watson,et al.  The PANAS-X manual for the positive and negative affect schedule , 1994 .

[26]  L. Radloff The CES-D Scale , 1977 .

[27]  Boyang Li,et al.  Predicting Personality from Book Preferences with User-Generated Content Labels , 2017, IEEE Transactions on Affective Computing.

[28]  S. Gosling,et al.  Personality in its natural habitat: manifestations and implicit folk theories of personality in daily life. , 2006, Journal of personality and social psychology.

[29]  Jeffrey T. Hancock,et al.  Experimental evidence of massive-scale emotional contagion through social networks , 2014, Proceedings of the National Academy of Sciences.

[30]  Eric Horvitz,et al.  Predicting Depression via Social Media , 2013, ICWSM.

[31]  Wessel Kraaij,et al.  Detecting Work Stress in Offices by Combining Unobtrusive Sensors , 2018, IEEE Transactions on Affective Computing.

[32]  T. Graepel,et al.  Private traits and attributes are predictable from digital records of human behavior , 2013, Proceedings of the National Academy of Sciences.

[33]  Maike Luhmann,et al.  Using Big Data to study subjective well-being , 2017, Current Opinion in Behavioral Sciences.

[34]  Renaud Lambiotte,et al.  Tracking the Digital Footprints of Personality , 2014, Proceedings of the IEEE.

[35]  Karolina Baras,et al.  Overview of context-sensitive technologies for well-being , 2017, 2017 International Conference on Internet of Things for the Global Community (IoTGC).

[36]  Nello Cristianini,et al.  An Analysis of the Interaction Between Intelligent Software Agents and Human Users , 2018, Minds and Machines.

[37]  G. Arbanas Diagnostic and Statistical Manual of Mental Disorders (DSM-5) , 2015 .

[38]  P. Ekman,et al.  What the face reveals : basic and applied studies of spontaneous expression using the facial action coding system (FACS) , 2005 .

[39]  Misha Pavel,et al.  Home-based cognitive monitoring using embedded measures of verbal fluency in a computer word game , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[40]  A. Pentland,et al.  Computational Social Science , 2009, Science.

[41]  Rada Mihalcea,et al.  Values in Words: Using Language to Evaluate and Understand Personal Values , 2015, ICWSM.

[42]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[43]  Shrikanth S. Narayanan,et al.  A Robust Unsupervised Arousal Rating Framework using Prosody with Cross-Corpora Evaluation , 2012, INTERSPEECH.

[44]  Gloria Mark,et al.  Digital footprints: predicting personality from temporal patterns of technology use , 2017, UbiComp/ISWC Adjunct.

[45]  S. Schwartz Universals in the Content and Structure of Values: Theoretical Advances and Empirical Tests in 20 Countries , 1992 .

[46]  Gilbert Harman,et al.  XIV—Moral Philosophy Meets Social Psychology: Virtue Ethics and the Fundamental Attribution Error , 1999 .

[47]  Daniel M. Haybron,et al.  Is Construct Validation Valid? , 2016, Philosophy of Science.

[48]  R. Calvo,et al.  Positive Computing: Technology for Wellbeing and Human Potential , 2014, Psychology Teaching Review.

[49]  J. Gratch,et al.  The Oxford Handbook of Affective Computing , 2014 .

[50]  Anna Alexandrova,et al.  A Philosophy for the Science of Well-Being , 2017 .

[51]  Christopher M. Danforth,et al.  Instagram photos reveal predictive markers of depression , 2016, EPJ Data Science.

[52]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[53]  Daniel Nettle,et al.  Personality: What Makes You the Way You Are , 2007 .

[54]  Jordan A. Litman,et al.  The measurement of perceptual curiosity , 2004 .

[55]  J. Pennebaker,et al.  The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods , 2010 .

[56]  Guohua Shen,et al.  Deep image reconstruction from human brain activity , 2017, bioRxiv.

[57]  Seth Flaxman,et al.  Multimodal Sentiment Analysis To Explore the Structure of Emotions , 2018, KDD.

[58]  Shanmuganathan Raman,et al.  Facial Expression Recognition Using Visual Saliency and Deep Learning , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[59]  R. M. Tobin,et al.  Measuring emotional expression with the Linguistic Inquiry and Word Count. , 2007, The American journal of psychology.

[60]  Derek Ruths,et al.  Classifying Political Orientation on Twitter: It's Not Easy! , 2013, ICWSM.

[61]  Jeffrey Nichols,et al.  Understanding individuals' personal values from social media word use , 2014, CSCW.

[62]  Katherine E Henson,et al.  Risk of Suicide After Cancer Diagnosis in England , 2018, JAMA psychiatry.

[63]  J. Cohn,et al.  Automated Face Analysis for Affective Computing , 2015 .

[64]  Margaret L. Kern,et al.  Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach , 2013, PloS one.

[65]  J. Rust,et al.  Modern Psychometrics: The Science of Psychological Assessment , 1989 .

[66]  Cindy K. Chung,et al.  The development and psychometric properties of LIWC2007 , 2007 .

[67]  Padmini Srinivasan,et al.  Life Satisfaction and the Pursuit of Happiness on Twitter , 2016, PloS one.

[68]  S. Gosling,et al.  Facebook as a research tool for the social sciences: Opportunities, challenges, ethical considerations, and practical guidelines. , 2015, The American psychologist.

[69]  T. Yarkoni,et al.  Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning , 2017, Perspectives on psychological science : a journal of the Association for Psychological Science.

[70]  T. Loetscher,et al.  Eye Movements During Everyday Behavior Predict Personality Traits , 2018, Front. Hum. Neurosci..

[71]  M. Kosinski,et al.  Deep Neural Networks Are More Accurate Than Humans at Detecting Sexual Orientation From Facial Images , 2018, Journal of personality and social psychology.

[72]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .

[73]  Cindy K. Chung,et al.  Revealing Dimensions of Thinking in Open-Ended Self-Descriptions: An Automated Meaning Extraction Method for Natural Language. , 2008, Journal of research in personality.

[74]  P. Costa,et al.  Validation of the five-factor model of personality across instruments and observers. , 1987, Journal of personality and social psychology.

[75]  Yong Tao,et al.  Compound facial expressions of emotion , 2014, Proceedings of the National Academy of Sciences.

[76]  M. Kosinski,et al.  Psychological targeting as an effective approach to digital mass persuasion , 2017, Proceedings of the National Academy of Sciences.

[77]  J. Pennebaker,et al.  Language use of depressed and depression-vulnerable college students , 2004 .

[78]  A. Beck,et al.  An inventory for measuring depression. , 1961, Archives of general psychiatry.

[79]  D. Ariely,et al.  Neuromarketing: the hope and hype of neuroimaging in business , 2010, Nature Reviews Neuroscience.

[80]  J. Pennebaker,et al.  The Secret Life of Pronouns , 2003, Psychological science.

[81]  E. Diener,et al.  Subjective well-being. , 1984, Psychological bulletin.

[82]  Darren Schreiber,et al.  Neuropolitics: Twenty years later , 2017, Politics & Life Sciences.

[83]  Ryan L. Boyd,et al.  The Development and Psychometric Properties of LIWC2015 , 2015 .

[84]  R. Sherman,et al.  The datafication of talent: how technology is advancing the science of human potential at work , 2017, Current Opinion in Behavioral Sciences.

[85]  Johan Bollen,et al.  The happiness paradox: your friends are happier than you , 2016, EPJ Data Science.

[86]  J. Russell A circumplex model of affect. , 1980 .

[87]  Tetsuya Takiguchi,et al.  Detecting Abnormal Word Utterances in Children With Autism Spectrum Disorders , 2017, Perceptual and motor skills.

[88]  Daniel Marcu,et al.  Finding the WRITE Stuff: Automatic Identification of Discourse Structure in Student Essays , 2003, IEEE Intell. Syst..

[89]  Jing Wang,et al.  Predicting the brain activation pattern associated with the propositional content of a sentence: Modeling neural representations of events and states , 2017, Human brain mapping.

[90]  John P A Ioannidis,et al.  Informed Consent, Big Data, and the Oxymoron of Research That Is Not Research , 2013, The American journal of bioethics : AJOB.

[91]  F. Huppert,et al.  Flourishing Across Europe: Application of a New Conceptual Framework for Defining Well-Being , 2011, Social indicators research.

[92]  C. Keyes,et al.  The structure of psychological well-being revisited. , 1995, Journal of personality and social psychology.

[93]  Skyler T. Hawk,et al.  Presentation and validation of the Radboud Faces Database , 2010 .