SentiBooks: Enhancing Audiobooks via Affective Computing and Smart Light Bulbs

We present SentiBooks, a smartphone application to enhance the audiobook listening experience via affective computing and smart light bulbs. Users can connect to Philips Hue Light Bulbs with a smartphone app while listening to an audiobook. The app analyzes the emotional expression of the narrator of the audiobook using speech emotion recognition and adjusts the colors of the lighting settings according to the expression of the narrator in 10-seconds intervals. By transitioning between colors that are connected to the specific emotion that is currently dominant in the reading, the overall audiobook experience is intensified.

[1]  Helen H. Epps,et al.  Color-emotion associations: Past experience and personal preference , 2004 .

[2]  Christian Wolff,et al.  Toward Multimodal Sentiment Analysis of Historic Plays: A Case Study with Text and Audio for Lessing's Emilia Galotti , 2019, DHN.

[3]  Tieniu Tan,et al.  Affective Computing: A Review , 2005, ACII.

[4]  Iben Have,et al.  Digital Audiobooks: New Media, Users, and Experiences , 2015 .

[5]  Remco Magielse,et al.  Future lighting systems , 2013, CHI Extended Abstracts.

[6]  Eva Hudlicka,et al.  To feel or not to feel: The role of affect in human-computer interaction , 2003, Int. J. Hum. Comput. Stud..

[7]  Eyal Ofek,et al.  IllumiRoom: peripheral projected illusions for interactive experiences , 2013, SIGGRAPH '13.

[8]  Kang Liu,et al.  Book Review: Sentiment Analysis: Mining Opinions, Sentiments, and Emotions by Bing Liu , 2015, CL.

[9]  Mohammad Obaid,et al.  Preface to the focus section on 'Transforming Books and the Reading Experience through Interactive Technologies' - guest editorial , 2017, IxD&A.

[10]  Erik Cambria,et al.  A review of affective computing: From unimodal analysis to multimodal fusion , 2017, Inf. Fusion.

[11]  Victor M. Ruiz Penichet,et al.  Emotion detection: a technology review , 2017, Interacción.

[12]  Astrid Paeschke,et al.  A database of German emotional speech , 2005, INTERSPEECH.

[13]  Hyeon-Jeong Suk,et al.  Bon Appetit! An Investigation About Color Combination Between Lighting and Food , 2012 .

[14]  Thomas B. Moeslund,et al.  Controlling urban lighting by human motion patterns results from a full scale experiment , 2012, ACM Multimedia.

[15]  P. Valdez,et al.  Effects of color on emotions. , 1994, Journal of experimental psychology. General.

[16]  Mohammad Obaid,et al.  Transforming books and the reading experience through interactive technologies , 2017 .