Face-to-BMI: Using Computer Vision to Infer Body Mass Index on Social Media

A person's weight status can have profound implications on their life, ranging from mental health, to longevity, to financial income. At the societal level, "fat shaming" and other forms of "sizeism" are a growing concern, while increasing obesity rates are linked to ever raising healthcare costs. For these reasons, researchers from a variety of backgrounds are interested in studying obesity from all angles. To obtain data, traditionally, a person would have to accurately self-report their body-mass index (BMI) or would have to see a doctor to have it measured. In this paper, we show how computer vision can be used to infer a person's BMI from social media images. We hope that our tool, which we release, helps to advance the study of social aspects related to body weight.

[1]  Weisi Lin,et al.  Do Others Perceive You As You Want Them To?: Modeling Personality based on Selfies , 2015, ASM@ACM Multimedia.

[2]  K. Brownell,et al.  Perceptions of weight discrimination: prevalence and comparison to race and gender discrimination in America , 2008, International Journal of Obesity.

[3]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[4]  David I. Perrett,et al.  Perception of health from facial cues , 2016, Philosophical Transactions of the Royal Society B: Biological Sciences.

[5]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[6]  Lei Huang,et al.  Social Media Profiler: Inferring Your Social Media Personality from Visual Attributes in Portrait , 2016, PCM.

[7]  Hazim Kemal Ekenel,et al.  How Transferable Are CNN-Based Features for Age and Gender Classification? , 2016, 2016 International Conference of the Biometrics Special Interest Group (BIOSIG).

[8]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[9]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[11]  D. Perrett,et al.  Using composite images to assess accuracy in personality attribution to faces. , 2007, British journal of psychology.

[12]  S. Daniels,et al.  The Use of BMI in the Clinical Setting , 2009, Pediatrics.

[13]  Guodong Guo,et al.  A computational approach to body mass index prediction from face images , 2013, Image Vis. Comput..

[14]  A. Prentice,et al.  Beyond body mass index , 2001, Obesity reviews : an official journal of the International Association for the Study of Obesity.

[15]  Lyle H. Ungar,et al.  Analyzing Personality through Social Media Profile Picture Choice , 2016, ICWSM.

[16]  Ingmar Weber,et al.  Crowdsourcing Health Labels: Inferring Body Weight from Profile Pictures , 2016, Digital Health.

[17]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[18]  D. Perrett,et al.  Facial Adiposity: A Cue to Health? , 2009, Perception.

[19]  J. Chrisler,et al.  Sizeism is a health hazard , 2017 .

[20]  Ralph B D'Agostino,et al.  Body mass index, metabolic syndrome, and risk of type 2 diabetes or cardiovascular disease. , 2006, The Journal of clinical endocrinology and metabolism.

[21]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[22]  Jesse Hoey,et al.  First Impressions - Predicting User Personality from Twitter Profile Images , 2016, HBU.

[24]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.