Explainable and Interpretable Models in Computer Vision and Machine Learning

This book compiles leading research on the development of explainable and interpretable machine learning methods in the context of computer vision and machine learning. Research progress in computer vision and pattern recognition has led to a variety of modeling techniques with almost human-like performance. Although these models have obtained astounding results, they are limited in their explainability and interpretability: what is the rationale behind the decision made? what in the model structure explains its functioning? Hence, while good performance is a critical required characteristic for learning machines, explainability and interpretability capabilities are needed to take learning machines to the next step to include them in decision support systems involving human supervision.

[1]  M. Pietikäinen,et al.  A Malaria Diagnostic Tool Based on Computer Vision Screening and Visualization of Plasmodium falciparum Candidate Areas in Digitized Blood Smears , 2014, PloS one.

[2]  Alexandre Tkatchenko,et al.  Quantum-chemical insights from deep tensor neural networks , 2016, Nature Communications.

[3]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[4]  William W. Cohen Fast Effective Rule Induction , 1995, ICML.

[5]  Tianming Liu,et al.  Predicting eye fixations using convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Avanti Shrikumar,et al.  Learning Important Features Through Propagating Activation Differences , 2017, ICML.

[7]  Gabriel J. Brostow,et al.  Becoming the expert - interactive multi-class machine teaching , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Fernando De la Torre,et al.  Supervised Descent Method and Its Applications to Face Alignment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Trevor Darrell,et al.  Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding , 2016, EMNLP.

[10]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[11]  Johannes Fürnkranz,et al.  On the quest for optimal rule learning heuristics , 2010, Machine Learning.

[12]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Robert M. Guion,et al.  Assessment, Measurement, and Prediction for Personnel Decisions , 1997 .

[14]  Dan-Olof Rooth,et al.  The role of automatic obesity stereotypes in real hiring discrimination. , 2011, The Journal of applied psychology.

[15]  Jongwook Choi,et al.  Supervising Neural Attention Models for Video Captioning by Human Gaze Data , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[17]  Sanjiv Singh,et al.  The DARPA Urban Challenge: Autonomous Vehicles in City Traffic, George Air Force Base, Victorville, California, USA , 2009, The DARPA Urban Challenge.

[18]  R. P. Fishburne,et al.  Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel , 1975 .

[19]  Johannes Fürnkranz,et al.  Multi-Label Classification with Label Constraints , 2008 .

[20]  Yoshua Bengio,et al.  Challenges in representation learning: A report on three machine learning contests , 2013, Neural Networks.

[21]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[22]  Tara S. Behrend,et al.  The viability of crowdsourcing for survey research , 2011, Behavior research methods.

[23]  Hugo Larochelle,et al.  Recurrent Mixture Density Network for Spatiotemporal Visual Attention , 2016, ICLR.

[24]  Donato Malerba,et al.  A Multistrategy Approach to Learning Multiple Dependent Concepts , 1996 .

[25]  Pratik Rane,et al.  Self-Critical Sequence Training for Image Captioning , 2018 .

[26]  Chandrima Sarkar,et al.  Feature Analysis for Computational Personality Recognition Using YouTube Personality Data set , 2014, WCPR '14.

[27]  K. McKeown,et al.  Justification Narratives for Individual Classifications , 2014 .

[28]  R. A. Bradley,et al.  Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons , 1952 .

[29]  Peter I. Cowling,et al.  MMAC: a new multi-class, multi-label associative classification approach , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[30]  Sudeep Sarkar,et al.  Spatially Coherent Interpretations of Videos Using Pattern Theory , 2016, International Journal of Computer Vision.

[31]  Fabio Valente,et al.  Annotation and Recognition of Personality Traits in Spoken Conversations from the AMI Meetings Corpus , 2012, INTERSPEECH.

[32]  Christoph H. Lampert,et al.  Attribute-Based Classification for Zero-Shot Visual Object Categorization , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Sergio Escalera,et al.  Multimodal First Impression Analysis with Deep Residual Networks , 2018, IEEE Transactions on Affective Computing.

[34]  Lior Rokach,et al.  Exploiting label dependencies for improved sample complexity , 2013, Machine Learning.

[35]  Johannes Fürnkranz,et al.  From Local to Global Patterns: Evaluation Issues in Rule Learning Algorithms , 2004, Local Pattern Detection.

[36]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Peter I. Cowling,et al.  Knowledge and Information Systems , 2006 .

[38]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[39]  Peter Robinson,et al.  Cross-dataset learning and person-specific normalisation for automatic Action Unit detection , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[40]  Marc'Aurelio Ranzato,et al.  Sequence Level Training with Recurrent Neural Networks , 2015, ICLR.

[41]  Kate Saenko,et al.  Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering , 2015, ECCV.

[42]  Christopher Kanan,et al.  Answer-Type Prediction for Visual Question Answering , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Siqi Liu,et al.  Improved Image Captioning via Policy Gradient optimization of SPIDEr , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[44]  Johannes Fürnkranz,et al.  Multi-label LeGo - Enhancing Multi-label Classifiers with Local Patterns , 2012, IDA.

[45]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[46]  A. Knobbe,et al.  Supervised descriptive local pattern mining with complex target concepts , 2016 .

[47]  Albert Ali Salah,et al.  Multimodal fusion of audio, scene, and face features for first impression estimation , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[48]  Murray R. Barrick,et al.  THE BIG FIVE PERSONALITY DIMENSIONS AND JOB PERFORMANCE: A META-ANALYSIS , 1991 .

[49]  Navdeep Jaitly,et al.  Towards End-To-End Speech Recognition with Recurrent Neural Networks , 2014, ICML.

[50]  Dan Klein,et al.  Neural Module Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  W. Lewis Johnson,et al.  Agents that Learn to Explain Themselves , 1994, AAAI.

[52]  Jonathan Anderson Lix and Rix: Variations on a Little-Known Readability Index. , 1983 .

[53]  Adam Tauman Kalai,et al.  Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings , 2016, NIPS.

[54]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[55]  Bohyung Han,et al.  Image Question Answering Using Convolutional Neural Network with Dynamic Parameter Prediction , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Alexander Binder,et al.  Evaluating the Visualization of What a Deep Neural Network Has Learned , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[57]  Sunita Sarawagi,et al.  Discriminative Methods for Multi-labeled Classification , 2004, PAKDD.

[58]  R. Ledley,et al.  Reasoning foundations of medical diagnosis. , 1991, M.D. computing : computers in medical practice.

[59]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[60]  Srinivasan Parthasarathy,et al.  New Algorithms for Fast Discovery of Association Rules , 1997, KDD.

[61]  H. A. Pines,et al.  No Fat Persons Need Apply , 1979 .

[62]  Alex Alves Freitas,et al.  Comprehensible classification models: a position paper , 2014, SKDD.

[63]  Frank J. Bernieri,et al.  Toward a histology of social behavior: Judgmental accuracy from thin slices of the behavioral stream , 2000 .

[64]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[65]  R. L. Dipboye,et al.  RECONSIDERING THE USE OF PERSONALITY TESTS IN PERSONNEL SELECTION CONTEXTS , 2007 .

[66]  William Whittaker,et al.  Autonomous driving in urban environments: Boss and the Urban Challenge , 2008, J. Field Robotics.

[67]  Mariusz Bojarski,et al.  VisualBackProp: visualizing CNNs for autonomous driving , 2016, ArXiv.

[68]  Sergio Escalera,et al.  ChaLearn Joint Contest on Multimedia Challenges Beyond Visual Analysis: An overview , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[69]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[70]  Grigorios Tsoumakas,et al.  Mining Multi-label Data , 2010, Data Mining and Knowledge Discovery Handbook.

[71]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[72]  Wojciech Kotlowski,et al.  ENDER: a statistical framework for boosting decision rules , 2010, Data Mining and Knowledge Discovery.

[73]  M. Mount,et al.  Validity of observer ratings of the five-factor model of personality traits: a meta-analysis. , 2011, The Journal of applied psychology.

[74]  Alex Fridman,et al.  DeepTraffic: Driving Fast through Dense Traffic with Deep Reinforcement Learning , 2018, ArXiv.

[75]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[76]  Ruslan Salakhutdinov,et al.  Multimodal Neural Language Models , 2014, ICML.

[77]  R. Gunning The Technique of Clear Writing. , 1968 .

[78]  Yash Goyal,et al.  Towards Transparent AI Systems: Interpreting Visual Question Answering Models , 2016, 1608.08974.

[79]  Klaus-Robert Müller,et al.  Nonlinear gradient denoising: Finding accurate extrema from inaccurate functional derivatives , 2015 .

[80]  Raymond J. Mooney,et al.  Stacked Ensembles of Information Extractors for Knowledge-Base Population , 2015, ACL.

[81]  Li Fei-Fei,et al.  Every Moment Counts: Dense Detailed Labeling of Actions in Complex Videos , 2015, International Journal of Computer Vision.

[82]  M. Friedman The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[83]  Eyke Hüllermeier,et al.  On label dependence and loss minimization in multi-label classification , 2012, Machine Learning.

[84]  M. A. Campion,et al.  APPLICANT REACTIONS TO SELECTION: DEVELOPMENT OF THE SELECTION PROCEDURAL JUSTICE SCALE (SPJS) , 2001 .

[85]  Amy J. C. Cuddy,et al.  Universal dimensions of social cognition: warmth and competence , 2007, Trends in Cognitive Sciences.

[86]  Raymond J. Mooney,et al.  Stacking With Auxiliary Features , 2016, IJCAI.

[87]  Nuno Vasconcelos,et al.  Anomaly detection in crowded scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[88]  Or Biran,et al.  Explanation and Justification in Machine Learning : A Survey Or , 2017 .

[89]  Max Welling,et al.  Visualizing Deep Neural Network Decisions: Prediction Difference Analysis , 2017, ICLR.

[90]  P. Borkenau,et al.  Extraversion is accurately perceived after a 50-ms exposure to a face. , 2009 .

[91]  JOHANNES FÜRNKRANZ,et al.  Separate-and-Conquer Rule Learning , 1999, Artificial Intelligence Review.

[92]  Margaret Mitchell,et al.  VQA: Visual Question Answering , 2015, International Journal of Computer Vision.

[93]  Grigorios Tsoumakas,et al.  Multi-Label Classification of Music into Emotions , 2008, ISMIR.

[94]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[95]  Raymond J. Mooney,et al.  Stacking with Auxiliary Features for Visual Question Answering , 2018, NAACL.

[96]  Grigorios Tsoumakas,et al.  Introduction to the special issue on learning from multi-label data , 2012, Machine Learning.

[97]  Geoff Holmes,et al.  Classifier chains for multi-label classification , 2009, Machine Learning.

[98]  A. Pentland Social Dynamics: Signals and Behavior , 2004 .

[99]  Leo Lebanov,et al.  Random Forests machine learning applied to gas chromatography - Mass spectrometry derived average mass spectrum data sets for classification and characterisation of essential oils. , 2020, Talanta.

[100]  M. Mori THE UNCANNY VALLEY , 2020, The Monster Theory Reader.

[101]  Arvind Narayanan,et al.  Semantics derived automatically from language corpora contain human-like biases , 2016, Science.

[102]  Jianxiong Xiao,et al.  DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[103]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[104]  M. Cook,et al.  Personnel Selection , 2004 .

[105]  Trevor Darrell,et al.  Multimodal Explanations: Justifying Decisions and Pointing to the Evidence , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[106]  Joy E. Beatty,et al.  The Effects of Video and Paper Resumes on Assessments of Personality, Applied Social Skills, Mental Capability, and Resume Outcomes , 2014 .

[107]  Rickard Carlsson,et al.  Warm and Competent Hassan = Cold and Incompetent Eric: A Harsh Equation of Real-Life Hiring Discrimination , 2012 .

[108]  Wei Xu,et al.  ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering , 2015, ArXiv.

[109]  R. Hogan,et al.  New Talent Signals: Shiny New Objects or a Brave New World? , 2016, Industrial and Organizational Psychology.

[110]  John F. Canny,et al.  Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[111]  David D. Lewis,et al.  An evaluation of phrasal and clustered representations on a text categorization task , 1992, SIGIR '92.

[112]  Foster J. Provost,et al.  Explaining Data-Driven Document Classifications , 2013, MIS Q..

[113]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[114]  Ryan Calo,et al.  There is a blind spot in AI research , 2016, Nature.

[115]  Seth Flaxman,et al.  European Union Regulations on Algorithmic Decision-Making and a "Right to Explanation" , 2016, AI Mag..

[116]  S. Gosling,et al.  Personality and Social Psychology Bulletin Personality Judgments Based on Physical Appearance Personality Judgments Based on Physical Appearance , 2022 .

[117]  Douglas Eck,et al.  Learning Features from Music Audio with Deep Belief Networks , 2010, ISMIR.

[118]  Khalil Sima'an,et al.  Wired for Speech: How Voice Activates and Advances the Human-Computer Relationship , 2006, Computational Linguistics.

[119]  M. Coleman,et al.  A computer readability formula designed for machine scoring. , 1975 .

[120]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[121]  Scott Nowson,et al.  Look! Who's Talking?: Projection of Extraversion Across Different Social Contexts , 2014, WCPR '14.

[122]  Matthias Bethge,et al.  Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet , 2014, ICLR.

[123]  Robert E. Ployhart,et al.  Solving the Supreme Problem: 100 years of selection and recruitment at the Journal of Applied Psychology. , 2017, The Journal of applied psychology.

[124]  Albert Ali Salah,et al.  Continuous Mapping of Personality Traits: A Novel Challenge and Failure Conditions , 2014, MAPTRAITS '14.

[125]  Shichao Zhang,et al.  Association Rule Mining: Models and Algorithms , 2002 .

[126]  Bernard C. K. Choi,et al.  Multidisciplinarity, interdisciplinarity and transdisciplinarity in health research, services, education and policy: 1. Definitions, objectives, and evidence of effectiveness. , 2006, Clinical and investigative medicine. Medecine clinique et experimentale.

[127]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[128]  Ning Zhang,et al.  Deep Reinforcement Learning-Based Image Captioning with Embedding Reward , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[129]  Johannes Fürnkranz,et al.  Shorter Rules Are Better, Aren't They? , 2016, DS.

[130]  Soraia Raupp Musse,et al.  Crowd Analysis Using Computer Vision Techniques , 2010, IEEE Signal Processing Magazine.

[131]  Toniann Pitassi,et al.  Fairness through awareness , 2011, ITCS '12.

[132]  Albert Ali Salah,et al.  Combining Deep Facial and Ambient Features for First Impression Estimation , 2016, ECCV Workshops.

[133]  Geoffrey I. Webb Efficient search for association rules , 2000, KDD '00.

[134]  Bolei Zhou,et al.  Network Dissection: Quantifying Interpretability of Deep Visual Representations , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[135]  Sridha Sridharan,et al.  Going Deeper: Autonomous Steering with Neural Memory Networks , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[136]  Li Fei-Fei,et al.  DenseCap: Fully Convolutional Localization Networks for Dense Captioning , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[137]  Yang Gao,et al.  End-to-End Learning of Driving Models from Large-Scale Video Datasets , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[138]  Alexander Binder,et al.  Explaining nonlinear classification decisions with deep Taylor decomposition , 2015, Pattern Recognit..

[139]  Gary N. Burns,et al.  The Good Judge Revisited: Individual Differences in the Accuracy of Personality Judgments , 2005 .

[140]  Fabio Valente,et al.  The INTERSPEECH 2013 computational paralinguistics challenge: social signals, conflict, emotion, autism , 2013, INTERSPEECH.

[141]  Michael Smith A theory of the validity of predictors in selection , 1994 .

[142]  G. Harry McLaughlin,et al.  SMOG Grading - A New Readability Formula. , 1969 .

[143]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[144]  E. Shortliffe,et al.  An analysis of physician attitudes regarding computer-based clinical consultation systems. , 1981, Computers and biomedical research, an international journal.

[145]  Johannes Fürnkranz,et al.  A Comparison of Techniques for Selecting and Combining Class Association Rules , 2008, LWA.

[146]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[147]  Brian McWilliams,et al.  The Shattered Gradients Problem: If resnets are the answer, then what is the question? , 2017, ICML.

[148]  Rob J Hyndman,et al.  Forecasting with Exponential Smoothing: The State Space Approach , 2008 .

[149]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[150]  Francisco Charte,et al.  Multilabel Classification: Problem Analysis, Metrics and Techniques , 2016 .

[151]  Trevor Darrell,et al.  YouTube2Text: Recognizing and Describing Arbitrary Activities Using Semantic Hierarchies and Zero-Shot Recognition , 2013, 2013 IEEE International Conference on Computer Vision.

[152]  Joichi Ito,et al.  Interventions over Predictions: Reframing the Ethical Debate for Actuarial Risk Assessment , 2017, FAT.

[153]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[154]  Jennifer S. Tucker,et al.  Selection in the Information Age: The Impact of Privacy Concerns and Computer Experience on Applicant Reactions , 2006 .

[155]  H. Chad Lane,et al.  Explainable Artificial Intelligence for Training and Tutoring , 2005, AIED.

[156]  Jian Pei,et al.  Mining Frequent Patterns without Candidate Generation: A Frequent-Pattern Tree Approach , 2006, Sixth IEEE International Conference on Data Mining - Workshops (ICDMW'06).

[157]  Eva Derous,et al.  Are they accurate? Recruiters' personality judgments in paper versus video resumes , 2017, Comput. Hum. Behav..

[158]  Bernt Schiele,et al.  Generative Adversarial Text to Image Synthesis , 2016, ICML.

[159]  Michael van Lent,et al.  An Explainable Artificial Intelligence System for Small-unit Tactical Behavior , 2004, AAAI.

[160]  Elmar Nöth,et al.  The INTERSPEECH 2012 Speaker Trait Challenge , 2012, INTERSPEECH.

[161]  Mohammed J. Zaki,et al.  Multi-label Lazy Associative Classification , 2007, PKDD.

[162]  Carmen Lacave,et al.  A review of explanation methods for Bayesian networks , 2002, The Knowledge Engineering Review.

[163]  Firoj Alam,et al.  Personality Traits Recognition on Social Network - Facebook , 2013, Proceedings of the International AAAI Conference on Web and Social Media.

[164]  R. Flesch A new readability yardstick. , 1948, The Journal of applied psychology.

[165]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[166]  Rajesh Rajamani,et al.  Vehicle dynamics and control , 2005 .

[167]  Yang Gao,et al.  Compact Bilinear Pooling , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[168]  Sergio Escalera,et al.  ChaLearn LAP 2016: First Round Challenge on First Impressions - Dataset and Results , 2016, ECCV Workshops.

[169]  Mario Fritz,et al.  A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input , 2014, NIPS.

[170]  Terry A. Beehr,et al.  Effects of applicant sex, applicant physical attractiveness, type of rater and type of job on interview decisions* , 1986 .

[171]  Yaneer Bar-Yam,et al.  The limits of phenomenology: From behaviorism to drug testing and engineering design , 2013, Complex..

[172]  V. Bacharach,et al.  Psychometrics : An Introduction , 2007 .

[173]  Neil Anderson,et al.  The practitioner‐researcher divide in Industrial, Work and Organizational (IWO) psychology: Where are we now, and where do we go from here? , 2001 .

[174]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[175]  Bernt Schiele,et al.  Learning Deep Representations of Fine-Grained Visual Descriptions , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[176]  Amy J. C. Cuddy,et al.  A model of (often mixed) stereotype content: competence and warmth respectively follow from perceived status and competition. , 2002, Journal of personality and social psychology.

[177]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[178]  Tara S. Behrend,et al.  Technology in The employmenT inTerview: A meTA-AnAlysis And FuTure reseArch AgendA , 2016 .

[179]  Jason Weston,et al.  A kernel method for multi-labelled classification , 2001, NIPS.

[180]  Robbert Sanderman,et al.  Correction: Ineffectiveness of Reverse Wording of Questionnaire Items: Let’s Learn from Cows in the Rain , 2013, PLoS ONE.

[181]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[182]  Eva Derous,et al.  Fairness Perceptions of Video Resumes Among Ethnically Diverse Applicants , 2012 .

[183]  Eneldo Loza Mencía,et al.  Learning rules for multi-label classification: a stacking and a separate-and-conquer approach , 2016, Machine Learning.

[184]  Heiko Paulheim,et al.  Learning Semantically Coherent Rules , 2014, DMNLP@PKDD/ECML.

[185]  Filip Lievens,et al.  An In-Depth Look at Dispositional Reasoning and Interviewer Accuracy , 2015 .

[186]  Filip Lievens,et al.  A Closer Look at the Measurement of Dispositional Reasoning: Dimensionality and Invariance Across Assessor Groups , 2017 .

[187]  Albert Ali Salah,et al.  Multi-modal Score Fusion and Decision Trees for Explainable Automatic Job Candidate Screening from Video CVs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[188]  Pericles A. Mitkas,et al.  Effective Rule-Based Multi-label Classification with Learning Classifier Systems , 2013, ICANNGA.

[189]  Sebastian Thrun,et al.  Towards fully autonomous driving: Systems and algorithms , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[190]  L. Cronbach Coefficient alpha and the internal structure of tests , 1951 .

[191]  R. Mike Cameron-Jones,et al.  Avoiding Pitfalls When Learning Recursive Theories , 1993, IJCAI.

[192]  Alan Hanjalic,et al.  One deep music representation to rule them all? A comparative analysis of different representation learning strategies , 2018, Neural Computing and Applications.

[193]  Daniel Gildea,et al.  Automated Analysis and Prediction of Job Interview Performance , 2015, IEEE Transactions on Affective Computing.

[194]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[195]  Firoj Alam,et al.  Predicting Personality Traits using Multimodal Information , 2014, WCPR '14.

[196]  Geoffrey I. Webb,et al.  Supervised Descriptive Rule Discovery: A Unifying Survey of Contrast Set, Emerging Pattern and Subgroup Mining , 2009, J. Mach. Learn. Res..

[197]  Nick Campbell,et al.  Towards Automatic Recognition of Attitudes: Prosodic Analysis of Video Blogs , 2014 .

[198]  Wei Xu,et al.  Explain Images with Multimodal Recurrent Neural Networks , 2014, ArXiv.

[199]  Yann LeCun,et al.  Generalization and network design strategies , 1989 .

[200]  Lars Kai Hansen,et al.  Model sparsity and brain pattern interpretation of classification models in neuroimaging , 2012, Pattern Recognit..

[201]  Dhruv Batra,et al.  Human Attention in Visual Question Answering: Do Humans and Deep Networks look at the same regions? , 2016, EMNLP.

[202]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[203]  Sergio Escalera,et al.  Overcoming Calibration Problems in Pattern Labeling with Pairwise Ratings: Application to Personality Traits , 2016, ECCV Workshops.

[204]  Odette Scharenborg,et al.  Speech perception by humans and machines , 2017 .

[205]  Rita Cucchiara,et al.  Predicting Human Eye Fixations via an LSTM-Based Saliency Attentive Model , 2016, IEEE Transactions on Image Processing.

[206]  Niklas Lavesson,et al.  User-oriented Assessment of Classification Model Understandability , 2011, SCAI.

[207]  Klaus-Robert Müller,et al.  Finding Density Functionals with Machine Learning , 2011, Physical review letters.

[208]  Philip J. Hayes,et al.  CONSTRUE/TIS: A System for Content-Based Indexing of a Database of News Stories , 1990, IAAI.

[209]  Zhanyi Hu,et al.  Modern physiognomy: an investigation on predicting personality traits and intelligence from the human face , 2016, Science China Information Sciences.

[210]  Joshua D. Reiss,et al.  Music Information Technology and Professional Stakeholder Audiences: Mind the Adoption Gap , 2012, Multimodal Music Processing.

[211]  Geoffrey I. Webb Recent Progress in Learning Decision Lists by Prepending Inferred Rules , 1994 .

[212]  Edward H. Shortliffe,et al.  A model of inexact reasoning in medicine , 1990 .

[213]  Sebastián Ventura,et al.  Multi‐label learning: a review of the state of the art and ongoing research , 2014, WIREs Data Mining Knowl. Discov..

[214]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[215]  Yuhong Guo,et al.  Multi-Label Classification Using Conditional Dependency Networks , 2011, IJCAI.

[216]  Emily D. Campion,et al.  Initial investigation into computer scoring of candidate essays for personnel selection. , 2016, The Journal of applied psychology.

[217]  Björn Schuller,et al.  Opensmile: the munich versatile and fast open-source audio feature extractor , 2010, ACM Multimedia.

[218]  Daniel Gatica-Perez,et al.  Hire me: Computational Inference of Hirability in Employment Interviews Based on Nonverbal Behavior , 2014, IEEE Transactions on Multimedia.

[219]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[220]  T. G. I. Fernando,et al.  Persons’ Personality Traits Recognition using Machine Learning Algorithms and Image Processing Techniques , 2016 .

[221]  Yann LeCun,et al.  What is the best multi-stage architecture for object recognition? , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[222]  Markus Langer,et al.  Information as a double-edged sword: The role of computer experience and information on applicant reactions towards novel technologies for personnel selection , 2018, Comput. Hum. Behav..

[223]  Terry B. Gutkin,et al.  Computers in human behavior: perspectives from the departing editors , 1987 .

[224]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[225]  Melanie Mitchell,et al.  Interpreting individual classifications of hierarchical networks , 2013, 2013 IEEE Symposium on Computational Intelligence and Data Mining (CIDM).

[226]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[227]  Shie Mannor,et al.  Graying the black box: Understanding DQNs , 2016, ICML.

[228]  Neil D. Lawrence,et al.  Dataset Shift in Machine Learning , 2009 .

[229]  Johannes Fürnkranz,et al.  On cognitive preferences and the plausibility of rule-based models , 2018, Machine Learning.

[230]  Jiasen Lu,et al.  Hierarchical Question-Image Co-Attention for Visual Question Answering , 2016, NIPS.

[231]  John Riedl,et al.  Explaining collaborative filtering recommendations , 2000, CSCW '00.

[232]  Duane Szafron,et al.  Visual Explanation of Evidence with Additive Classifiers , 2006, AAAI.

[233]  Samy Bengio,et al.  Show and tell: A neural image caption generator , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[234]  Robert E. Ployhart,et al.  Determinants, Detection and Amelioration of Adverse Impact in Personnel Selection Procedures: Issues, Evidence and Lessons Learned , 2001 .

[235]  Trevor Darrell,et al.  Generating Visual Explanations , 2016, ECCV.

[236]  Therese Macan,et al.  Comparison of the factors influencing interviewer hiring decisions for applicants with and those without disabilities , 1997 .

[237]  Eneldo Loza Mencía,et al.  Stacking Label Features for Learning Multilabel Rules , 2014, Discovery Science.

[238]  Sudeep Sarkar,et al.  Temporally coherent interpretations for long videos using pattern theory , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[239]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[240]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[241]  C. B. Colby The weirdest people in the world , 1973 .

[242]  Cornelius J. König,et al.  Are Observer Ratings of Applicants’ Personality Also Faked? Yes, But Less than Self‐Reports , 2017 .

[243]  Dhruv Batra,et al.  Analyzing the Behavior of Visual Question Answering Models , 2016, EMNLP.

[244]  Fulvio Mazzocchi,et al.  Could Big Data be the end of theory in science? , 2015, EMBO reports.

[245]  Tim Miller,et al.  Explanation in Artificial Intelligence: Insights from the Social Sciences , 2017, Artif. Intell..

[246]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[247]  Concha Bielza,et al.  Multi-label classification with Bayesian network-based chain classifiers , 2014, Pattern Recognit. Lett..

[248]  Eyke Hüllermeier,et al.  On the Problem of Error Propagation in Classifier Chains for Multi-label Classification , 2012, GfKl.

[249]  Markus Langer,et al.  Examining Digital Interviews for Personnel Selection: Applicant Reactions and Interviewer Ratings , 2017 .

[250]  J. van Leeuwen,et al.  Neural Networks: Tricks of the Trade , 2002, Lecture Notes in Computer Science.

[251]  P. Costa,et al.  The five-factor theory of personality. , 2008 .

[252]  Andrea Palazzi,et al.  Predicting the Driver's Focus of Attention: The DR(eye)VE Project , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[253]  P. Silver,et al.  Molecular Systems Biology in Drug Development , 2007, Clinical pharmacology and therapeutics.

[254]  Luc De Raedt,et al.  Multiple Predicate Learning , 1993, IJCAI.

[255]  G. Stanley Hall Practical relations between psychology and the war. , 1917 .

[256]  Johannes Gehrke,et al.  Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission , 2015, KDD.

[257]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[258]  Pietro Perona,et al.  The Caltech-UCSD Birds-200-2011 Dataset , 2011 .

[259]  Peter N. Belhumeur,et al.  How Do You Tell a Blackbird from a Crow? , 2013, 2013 IEEE International Conference on Computer Vision.

[260]  Jessica Lowell Neural Network , 2001 .

[261]  Timothée Masquelier,et al.  Deep Networks Can Resemble Human Feed-forward Vision in Invariant Object Recognition , 2015, Scientific Reports.

[262]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[263]  Jane Webster,et al.  Applicant reactions to face-to-face and technology-mediated interviews: a field investigation. , 2003, The Journal of applied psychology.

[264]  Razvan Pascanu,et al.  On the Number of Linear Regions of Deep Neural Networks , 2014, NIPS.

[265]  Eyke Hüllermeier,et al.  On the bayes-optimality of F-measure maximizers , 2013, J. Mach. Learn. Res..

[266]  Stéphane Ayache,et al.  Explaining First Impressions: Modeling, Recognizing, and Explaining Apparent Personality from Videos , 2018, ArXiv.

[267]  Cornelius J. König,et al.  Reasons for Being Selective When Choosing Personnel Selection Procedures , 2010 .

[268]  Bart Goethals,et al.  Frequent Set Mining , 2010, Data Mining and Knowledge Discovery Handbook.

[269]  S. Gilliland The Perceived Fairness of Selection Systems: An Organizational Justice Perspective , 1993 .

[270]  Bo Li,et al.  Multi-label Classification based on Association Rules with Application to Scene Classification , 2008, 2008 The 9th International Conference for Young Computer Scientists.

[271]  Alan L. Yuille,et al.  Generation and Comprehension of Unambiguous Object Descriptions , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[272]  Heikki Mannila,et al.  Fast Discovery of Association Rules , 1996, Advances in Knowledge Discovery and Data Mining.

[273]  Daniel Gatica-Perez,et al.  You Are Known by How You Vlog: Personality Impressions and Nonverbal Behavior in YouTube , 2011, ICWSM.

[274]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[275]  E A Smith,et al.  Automated readability index. , 1967, AMRL-TR. Aerospace Medical Research Laboratories.

[276]  C. Lawrence Zitnick,et al.  CIDEr: Consensus-based image description evaluation , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[277]  Wojciech Samek,et al.  Methods for interpreting and understanding deep neural networks , 2017, Digit. Signal Process..

[278]  Marcel van Gerven,et al.  Deep Impression: Audiovisual Deep Residual Networks for Multimodal Apparent Personality Trait Recognition , 2016, ECCV Workshops.

[279]  Andrew Slavin Ross,et al.  Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations , 2017, IJCAI.

[280]  R. E. Lee,et al.  Distribution-free multiple comparisons between successive treatments , 1995 .

[281]  Francisco Charte,et al.  LI-MLC: A Label Inference Methodology for Addressing High Dimensionality in the Label Space for Multilabel Classification , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[282]  Neal Schmitt,et al.  Experience-based and situational interview questions: Studies of validity. , 1995 .

[283]  Donato Malerba,et al.  Learning Recursive Theories in the Normal ILP Setting , 2003, Fundam. Informaticae.

[284]  Trevor Darrell,et al.  Attentive Explanations: Justifying Decisions and Pointing to the Evidence , 2016, ArXiv.

[285]  Kelly A. Piasentin,et al.  Applicant attraction to organizations and job choice: a meta-analytic review of the correlates of recruiting outcomes. , 2005, The Journal of applied psychology.

[286]  Dan Klein,et al.  Learning to Compose Neural Networks for Question Answering , 2016, NAACL.

[287]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[288]  Sebastián Ventura,et al.  Evolving Multi-label Classification Rules with Gene Expression Programming: A Preliminary Study , 2010, HAIS.

[289]  J. Webster,et al.  The Use of Technologies in the Recruiting, Screening, and Selection Processes for Job Candidates , 2003 .

[290]  Bernard Ghanem,et al.  On the relationship between visual attributes and convolutional networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[291]  Sebastián Ventura,et al.  A Tutorial on Multilabel Learning , 2015, ACM Comput. Surv..

[292]  Céline Robardet,et al.  Local Subgroup Discovery for Eliciting and Understanding New Structure-Odor Relationships , 2016, DS.

[293]  R. Gibson,et al.  What the Face Reveals , 2002 .

[294]  Sonja Gievska,et al.  The Impact of Affective Verbal Content on Predicting Personality Impressions in YouTube Videos , 2014, WCPR '14.

[295]  Gholamreza Anbarjafari,et al.  Automated Screening of Job Candidate Based on Multimodal Video Processing , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[296]  Martine De Cock,et al.  A Multivariate Regression Approach to Personality Impression Recognition of Vloggers , 2014, WCPR '14.

[297]  Stéphane Ayache,et al.  Design of an explainable machine learning challenge for video interviews , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[298]  E. Vincent Cross,et al.  Explaining robot actions , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[299]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[300]  V. Rajamani,et al.  An Evolutionary Multi Label Classification using Associative Rule Mining for Spatial Preferences , 2011 .

[301]  Albert Ali Salah,et al.  Video-based emotion recognition in the wild using deep transfer learning and score fusion , 2017, Image Vis. Comput..

[302]  Catherine Havasi,et al.  ConceptNet 5: A Large Semantic Network for Relational Knowledge , 2013, The People's Web Meets NLP.

[303]  Raymond J. Mooney,et al.  Combining Supervised and Unsupervised Enembles for Knowledge Base Population , 2016, EMNLP.

[304]  Yuandong Tian,et al.  Simple Baseline for Visual Question Answering , 2015, ArXiv.

[305]  Geoffrey E. Hinton,et al.  On the importance of initialization and momentum in deep learning , 2013, ICML.

[306]  Timnit Gebru,et al.  Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification , 2018, FAT.

[307]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[308]  R. Caneel Social signaling in decision making , 2005 .

[309]  M. Gevrey,et al.  Review and comparison of methods to study the contribution of variables in artificial neural network models , 2003 .

[310]  Yoshua Bengio,et al.  Hierarchical Recurrent Neural Networks for Long-Term Dependencies , 1995, NIPS.

[311]  T. Prescott,et al.  The brainstem reticular formation is a small-world, not scale-free, network , 2006, Proceedings of the Royal Society B: Biological Sciences.

[312]  Xiu-Shen Wei,et al.  Deep Bimodal Regression for Apparent Personality Analysis , 2016, ECCV Workshops.

[313]  Sergio Escalera,et al.  First Impressions: A Survey on Computer Vision-Based Apparent Personality Trait Analysis , 2018, ArXiv.

[314]  Yadong Mu,et al.  Deep Steering: Learning End-to-End Driving Model from Spatial and Temporal Visual Cues , 2017, ArXiv.

[315]  Ruslan Salakhutdinov,et al.  Action Recognition using Visual Attention , 2015, NIPS 2015.

[316]  Bolei Zhou,et al.  Object Detectors Emerge in Deep Scene CNNs , 2014, ICLR.

[317]  Stefan Ultes,et al.  Automatic Recognition of Personality Traits: A Multimodal Approach , 2014, MAPTRAITS '14.

[318]  Janine Willis,et al.  First Impressions , 2006, Psychological science.

[319]  M. Kosinski,et al.  Computer-based personality judgments are more accurate than those made by humans , 2015, Proceedings of the National Academy of Sciences.

[320]  Yejin Choi,et al.  Baby talk: Understanding and generating simple image descriptions , 2011, CVPR 2011.

[321]  Hugo Liu,et al.  ConceptNet — A Practical Commonsense Reasoning Tool-Kit , 2004 .

[322]  Douglas Eck,et al.  The need for music information retrieval with user-centered and multimodal strategies , 2011, MIRUM '11.

[323]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[324]  Cornelius J. König,et al.  Personality testing in personnel selection: Love it? Leave it? Understand it! , 2015 .

[325]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[326]  J. Levashina,et al.  Impression Management and Interview and Job Performance Ratings: A Meta-Analysis of Research Design with Tactics in Mind , 2017, Front. Psychol..