Goal-driven Command Recommendations for Analysts

Recent times have seen data analytics software applications become an integral part of the decision-making process of analysts. The users of these software applications generate a vast amount of unstructured log data. These logs contain clues to the user’s goals, which traditional recommender systems may find difficult to model implicitly from the log data. With this assumption, we would like to assist the analytics process of a user through command recommendations. We categorize the commands into software and data categories based on their purpose to fulfill the task at hand. On the premise that the sequence of commands leading up to a data command is a good predictor of the latter, we design, develop, and validate various sequence modeling techniques. In this paper, we propose a framework to provide goal-driven data command recommendations to the user by leveraging unstructured logs. We use the log data of a web-based analytics software to train our neural network models and quantify their performance, in comparison to relevant and competitive baselines. We propose a custom loss function to tailor the recommended data commands according to the goal information provided exogenously. We also propose an evaluation metric that captures the degree of goal orientation of the recommendations. We demonstrate the promise of our approach by evaluating the models with the proposed metric and showcasing the robustness of our models in the case of adversarial examples, where the user activity is misaligned with selected goal, through offline evaluation.

[1]  Sophie Ahrens,et al.  Recommender Systems , 2012 .

[2]  Vincent S. Tseng,et al.  CPT+: Decreasing the Time/Space Complexity of the Compact Prediction Tree , 2015, PAKDD.

[3]  Eric Horvitz,et al.  The Lumière Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users , 1998, UAI.

[4]  Yi Liu,et al.  One-against-all multi-class SVM classification using reliability measures , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[5]  Tova Milo,et al.  REACT: Context-Sensitive Recommendations for Data Analysis , 2016, SIGMOD Conference.

[6]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[7]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[8]  Belle Selene Xia,et al.  Review of business intelligence through data analysis , 2014 .

[9]  Paul Resnick,et al.  Recommender systems , 1997, CACM.

[10]  Brian D. Davison,et al.  Predicting Sequences of User Actions , 1998 .

[11]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Nicolaus Henke,et al.  The age of analytics: competing in a data-driven world , 2016 .

[13]  Gaurav Verma,et al.  Stuck? No worries!: Task-aware Command Recommendation and Proactive Help for Analysts , 2019, UMAP.

[14]  Zhicheng Liu,et al.  Identifying Frequent User Tasks from Application Logs , 2017, IUI.

[15]  Timothy Baldwin,et al.  Topically Driven Neural Language Model , 2017, ACL.

[16]  Chong Wang,et al.  TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency , 2016, ICLR.

[17]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[18]  Biswarup Bhattacharya,et al.  Intent-Aware Contextual Recommendation System , 2017, 2017 IEEE International Conference on Data Mining Workshops (ICDMW).

[19]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[20]  Qun Jin,et al.  Discovery of Action Patterns and User Correlations in Task-Oriented Processes for Goal-Driven Learning Recommendation , 2014, IEEE Transactions on Learning Technologies.

[21]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[22]  Archana Ganapathi,et al.  Analyzing Log Analysis: An Empirical Study of User Log Mining , 2014, LISA.

[23]  Andrew McCallum,et al.  Optimizing Semantic Coherence in Topic Models , 2011, EMNLP.

[24]  Ryan M. Rifkin,et al.  In Defense of One-Vs-All Classification , 2004, J. Mach. Learn. Res..

[25]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[26]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[27]  Zachary A. Pardos,et al.  Time slice imputation for personalized goal-based recommendation in higher education , 2019, RecSys.

[28]  Daniel Barbará,et al.  Topic Significance Ranking of LDA Generative Models , 2009, ECML/PKDD.

[29]  Samy Bengio,et al.  Show and tell: A neural image caption generator , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Ethem Alpaydin,et al.  Support Vector Machine for Multiclass Classification , 1998 .

[31]  BottouLéon,et al.  Natural Language Processing (Almost) from Scratch , 2011 .

[32]  Alec Radford,et al.  Improving Language Understanding by Generative Pre-Training , 2018 .

[33]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[34]  Sung-Bae Cho,et al.  Ensemble Approaches of Support Vector Machines for Multiclass Classification , 2007, PReMI.

[35]  Evgueni A. Haroutunian,et al.  Information Theory and Statistics , 2011, International Encyclopedia of Statistical Science.

[36]  Tovi Grossman,et al.  CommunityCommands: command recommendations for software applications , 2009, UIST '09.

[37]  Timothy Baldwin,et al.  Evaluating topic models for digital libraries , 2010, JCDL '10.

[38]  A. Blandford Intelligent interaction design: the role of human‐computer interaction research in the design of intelligent systems , 2001, Expert Syst. J. Knowl. Eng..

[39]  Kemal Polat,et al.  A novel hybrid intelligent method based on C4.5 decision tree classifier and one-against-all approach for multi-class classification problems , 2009, Expert Syst. Appl..

[40]  Xu Wang,et al.  Leveraging Community-Generated Videos and Command Logs to Classify and Recommend Software Workflows , 2018, CHI.

[41]  Yann Dauphin,et al.  Convolutional Sequence to Sequence Learning , 2017, ICML.

[42]  Xiaohui Yan,et al.  A biterm topic model for short texts , 2013, WWW.

[43]  Zachary A. Pardos,et al.  Goal-based Course Recommendation , 2018, LAK.

[44]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[45]  Michele Banko,et al.  Scaling to Very Very Large Corpora for Natural Language Disambiguation , 2001, ACL.

[46]  George Kingsley Zipf,et al.  Human behavior and the principle of least effort , 1949 .

[47]  Hao Wu,et al.  Hierarchical Neural Language Models for Joint Representation of Streaming Documents and their Content , 2015, WWW.

[48]  Xumin Liu Unraveling and Learning Workflow Models from Interleaved Event Logs , 2014, 2014 IEEE International Conference on Web Services.

[49]  Clyde Kluckhohn,et al.  Human Behavior and the Principle of Least Effort. George Kingsley Zipf , 1950 .

[50]  Xumin Liu,et al.  Incorporating User Behavior Patterns to Discover Workflow Models from Event Logs , 2013, 2013 IEEE 20th International Conference on Web Services.

[51]  Wojciech Czarnecki,et al.  On Loss Functions for Deep Neural Networks in Classification , 2017, ArXiv.