Training on the job: behavioral analysis of job interviews in hospitality

First impressions play a critical role in the hospitality industry and have been shown to be closely linked to the behavior of the person being judged.In this work, we implemented a behavioral training framework for hospitality students with the goal of improving the impressions that other people make about them. We outline the challenges associated with designing such a framework and embedding it in the everyday practice of a real hospitality school. We collected a dataset of 169 laboratory sessions where two role-plays were conducted, job interviews and reception desk scenarios, for a total of 338 interactions. For job interviews, we evaluated the relationship between automatically extracted nonverbal cues and various perceived social variables in a correlation analysis. Furthermore, our system automatically predicted first impressions from job interviews in a regression task, and was able to explain up to 32% of the variance, thus extending the results in existing literature, and showing gender differences, corroborating previous findings in psychology. This work constitutes a step towards applying social sensing technologies to the real world by designing and implementing a living lab for students of an international hospitality management school.

[1]  B. R. Schlenker Impression Management: The Self-Concept, Social Identity, and Interpersonal Relations , 1980 .

[2]  Daniel Gatica-Perez,et al.  Automatic nonverbal analysis of social interaction in small groups: A review , 2009, Image Vis. Comput..

[3]  Bilge Mutlu,et al.  MACH: my automated conversation coach , 2013, UbiComp.

[4]  Nicu Sebe,et al.  Please, tell me about yourself: automatic personality assessment using short self-presentations , 2011, ICMI '11.

[5]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[6]  Christian M. End,et al.  Short Communication: Powerless and Jobless? Comparing the Effects of Powerless Speech and Speech Disorders on an Applicant’s Employability , 2013 .

[7]  Daniel Gatica-Perez Signal Processing in the Workplace [Social Sciences] , 2015, IEEE Signal Processing Magazine.

[8]  S. J. Motowidlo,et al.  Why visual and vocal interview cues can affect interviewers' judgments and predict job performance. , 1999 .

[9]  Daniel Gatica-Perez,et al.  Hire me: Computational Inference of Hirability in Employment Interviews Based on Nonverbal Behavior , 2014, IEEE Transactions on Multimedia.

[10]  Shamsi T. Iqbal,et al.  Reducing disruption from subtle information delivery during a conversation: mode and bandwidth investigation , 2013, CHI.

[11]  Campbell Leaper,et al.  Women Are More Likely Than Men to Use Tentative Language, Aren’t They? A Meta-Analysis Testing for Gender Differences and Moderators , 2011 .

[12]  Mohammed E. Hoque,et al.  Rhema: A Real-Time In-Situ Intelligent Interface to Help People with Public Speaking , 2015, IUI.

[13]  Allen I. Huffcutt,et al.  Identification and meta-analytic assessment of psychological constructs measured in employment interviews. , 2001, The Journal of applied psychology.

[14]  T. DeGroot,et al.  Can Nonverbal Cues be Used to Make Meaningful Personality Attributions in Employment Interviews? , 2009 .

[15]  Daniel Gatica-Perez,et al.  I Would Hire You in a Minute: Thin Slices of Nonverbal Behavior in Job Interviews , 2015, ICMI.

[16]  Alex Pentland,et al.  Honest Signals - How They Shape Our World , 2008 .

[17]  A. Pentland,et al.  Thin slices of negotiation: predicting outcomes from conversational dynamics within the first 5 minutes. , 2007, The Journal of applied psychology.

[18]  N. Ambady,et al.  On judging and being judged accurately in zero-acquaintance situations. , 1995 .

[19]  Jean-Marc Odobez,et al.  Head Nod Detection from a Full 3D Model , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[20]  Richard Kazelskis,et al.  Relative Contributions of Verbal, Articulative, and Nonverbal Communication to Employment Decisions in the Job Interview Setting. , 1979 .

[21]  Nadia Mana,et al.  Multimodal recognition of personality traits in social interactions , 2008, ICMI '08.

[22]  Alvaro Marcos-Ramiro,et al.  Multimodal analysis of body communication cues in employment interviews , 2013, ICMI '13.

[23]  N. Ambady,et al.  Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis. , 1992 .

[24]  Gatica-PerezDaniel Automatic nonverbal analysis of social interaction in small groups , 2009 .

[25]  Alex Pentland,et al.  Honest signals: how social networks shape human behavior , 2011, ACM Multimedia.

[26]  Daniel Gatica-Perez,et al.  The YouTube Lens: Crowdsourced Personality Impressions and Audiovisual Analysis of Vlogs , 2013, IEEE Transactions on Multimedia.

[27]  M. Knapp,et al.  Nonverbal communication in human interaction , 1972 .

[28]  A. Imada,et al.  Influence of nonverbal communication and rater proximity on impressions and decisions in simulated employment interviews. , 1977 .

[29]  Daniel Gildea,et al.  Automated prediction and analysis of job interview performance: The role of what you say and how you say it , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[30]  Johannes Schöning,et al.  Augmenting Social Interactions: Realtime Behavioural Feedback using Social Signal Processing Techniques , 2015, CHI.

[31]  S. Rothmann,et al.  THE BIG FIVE PERSONALITY DIMENSIONS AND JOB PERFORMANCE , 2003 .

[32]  Thomas V. McGovern,et al.  Interviewer evaluations of interviewee nonverbal behavior , 1978 .

[33]  P. Jackson,et al.  Non‐verbal behaviour and the outcome of selection interviews , 1980 .