Mirror mirror on the wall… An intelligent multisensory mirror for well-being self-assessment

The face reveals the healthy status of an individual, through a combination of physical signs and facial expressions. The project SEMEOTICONS is translating the semeiotic code of the human face into computational descriptors and measures, automatically extracted from videos, images, and 3D scans of the face. SEMEOTICONS is developing a multisensory platform, in the form of a smart mirror, looking for signs related to cardio-metabolic risk. The goal is to enable users to self-monitor their well-being status over time and improve their life-style via tailored user guidance. Building the multisensory mirror requires addressing significant scientific and technological challenges, from touch-less data acquisition, to real-time processing and integration of multimodal data.

[1]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[2]  Kin-Man Lam,et al.  An efficient algorithm for human face detection and facial feature extraction under different conditions , 2001 .

[3]  Silvia Biasotti,et al.  Mathematical Tools for Shape Analysis and Description , 2014, Synthesis Lectures on Computer Graphics and Animation.

[4]  Hanna Karlsson,et al.  Microcirculation assessment using an individualized model for diffuse reflectance spectroscopy and conventional laser Doppler flowmetry , 2014, Journal of biomedical optics.

[5]  Sara Colantonio,et al.  A Wize Mirror for Lifestyle Improvement , 2014, InMed.

[6]  Behnoosh Hariri,et al.  YawDD: a yawning detection dataset , 2014, MMSys '14.

[7]  Marc Garbey,et al.  Interacting with human physiology , 2007, Comput. Vis. Image Underst..

[8]  Daniel McDuff,et al.  Remote measurement of cognitive stress via heart rate variability , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[9]  Sami Romdhani,et al.  A 3D Face Model for Pose and Illumination Invariant Face Recognition , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[10]  T. Rabelink,et al.  Endothelial function and dysfunction: testing and clinical relevance. , 2007, Circulation.

[11]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[12]  Sara Colantonio,et al.  Moving Medical Semeiotics to the Digital Realm - SEMEOTICONS Approach to Face Signs of Cardiometabolic Risk , 2014, HEALTHINF.

[13]  Ioannis Pitas,et al.  Extraction of facial regions and features using color and shape information , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[14]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[15]  G. Heo,et al.  Topological Analysis of Variance and the Maxillary Complex , 2012 .

[16]  C. Minson,et al.  Highlighted Topic Mechanisms and Modulators of Temperature Regulation Thermal Provocation to Evaluate Microvascular Reactivity in Human Skin , 2022 .

[17]  Simon Lucey,et al.  Deformable Model Fitting by Regularized Landmark Mean-Shift , 2010, International Journal of Computer Vision.

[18]  Lisa Kuramoto,et al.  Association of skin cholesterol content, measured by a noninvasive method, with markers of inflammation and Framingham risk prediction. , 2002, The American journal of cardiology.

[19]  Christian Huitema,et al.  Real-time 3D face tracking based on active appearance model constrained by depth data , 2014, Image Vis. Comput..

[20]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[21]  Federica Marcolin,et al.  3D human face description: landmarks measures and geometrical features , 2012, Image Vis. Comput..

[22]  Daniela Giorgi,et al.  Describing shapes by geometrical-topological properties of real functions , 2008, CSUR.

[23]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[24]  A. Barden,et al.  Advanced Glycation End Products: A Review , 2013 .

[25]  J. Y. Kim,et al.  Predicting visceral obesity based on facial characteristics , 2014, BMC Complementary and Alternative Medicine.

[26]  V. Ferrario,et al.  Soft tissue facial morphology in obese adolescents: a three-dimensional noninvasive assessment. , 2009, The Angle orthodontist.

[27]  Luc Van Gool,et al.  Real Time Head Pose Estimation from Consumer Depth Cameras , 2011, DAGM-Symposium.

[28]  Bogdan J. Matuszewski,et al.  Head pose tracking for immersive applications , 2014, 2014 IEEE International Conference on Image Processing (ICIP).