Balancing Familiarity and Curiosity in Data Exploration with Deep Reinforcement Learning

The ability to find a set of records in Exploratory Data Analysis (EDA) hinges on the scattering of objects in the data set and the on users’ knowledge of data and their ability to express their needs. This yields a wide range of EDA scenarios and solutions that differ in the guidance they provide to users. In this paper, we investigate the interplay between modeling curiosity and familiarity in Deep Reinforcement Learning (DRL) and expressive data exploration operators. We formalize curiosity as intrinsic reward and familiarity as extrinsic reward. We examine the behavior of several policies learned for different weights for those rewards. Our experiments on SDSS, a very large sky survey data set1 provide several insights and justify the need for a deeper examination of combining DRL and data exploration operators that go beyond drill-downs and roll-ups.

[1]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[2]  Panos Vassiliadis,et al.  Towards a Benefit-based Optimizer for Interactive Data Analysis , 2019, DOLAP.

[3]  Olga Papaemmanouil,et al.  AIDE: An Active Learning-Based Approach for Interactive Data Exploration , 2016, IEEE Transactions on Knowledge and Data Engineering.

[4]  Pieter Abbeel,et al.  Constrained Policy Optimization , 2017, ICML.

[5]  John T. Stasko,et al.  Interactive Browsing and Navigation in Relational Databases , 2016, Proc. VLDB Endow..

[6]  Mohamed A. Sharaf,et al.  REQUEST: A scalable framework for interactive construction of exploratory queries , 2016, 2016 IEEE International Conference on Big Data (Big Data).

[7]  Gautam Das,et al.  Facetedpedia: enabling query-dependent faceted search for wikipedia , 2010, CIKM '10.

[8]  Alexandre Termier,et al.  Interactive User Group Analysis , 2015, CIKM.

[9]  Arnab Nandi,et al.  Distributed and interactive cube exploration , 2014, 2014 IEEE 30th International Conference on Data Engineering.

[10]  Anthony G. Cohn,et al.  A Spatial Logic based on Regions and Connection , 1992, KR.

[11]  Zi Huang,et al.  Curiosity-driven Reinforcement Learning for Diverse Visual Paragraph Generation , 2019, ACM Multimedia.

[12]  Laks V. S. Lakshmanan,et al.  Exploring Rated Datasets with Rating Maps , 2017, WWW.

[13]  Yi Zhang,et al.  Interactive retrieval based on faceted feedback , 2010, SIGIR '10.

[14]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[15]  C. Lintott,et al.  Galaxy Zoo 2: detailed morphological classifications for 304,122 galaxies from the Sloan Digital Sky Survey , 2013, 1308.3496.

[16]  Vipin Kumar,et al.  Summarization - compressing data into an informative representation , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[17]  Nuttapong Chentanez,et al.  Intrinsically Motivated Reinforcement Learning , 2004, NIPS.

[18]  Neoklis Polyzotis,et al.  QueRIE: Collaborative Database Exploration , 2014, IEEE Transactions on Knowledge and Data Engineering.

[19]  Tova Milo,et al.  Automatically Generating Data Exploration Sessions Using Deep Reinforcement Learning , 2020, SIGMOD Conference.

[20]  Carsten Binnig,et al.  IDEBench: A Benchmark for Interactive Data Exploration , 2018, SIGMOD Conference.

[21]  Sihem Amer-Yahia,et al.  Guided exploration of user groups , 2020, Proc. VLDB Endow..

[22]  Alexei A. Efros,et al.  Curiosity-Driven Exploration by Self-Supervised Prediction , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[23]  Cong Yu,et al.  Who Tags What? An Analysis Framework , 2012, Proc. VLDB Endow..

[24]  Rina Dechter,et al.  Value iteration and policy iteration algorithms for Markov decision problem , 1996 .

[25]  Themis Palpanas,et al.  New Trends on Exploratory Methods for Data Analytics , 2017, Proc. VLDB Endow..

[26]  Peter Geibel,et al.  Reinforcement Learning for MDPs with Constraints , 2006, ECML.

[27]  Sanjiang Li,et al.  Region Connection Calculus: Its models and composition table , 2003, Artif. Intell..

[28]  Hiroki Arimura,et al.  LCM ver. 2: Efficient Mining Algorithms for Frequent/Closed/Maximal Itemsets , 2004, FIMI.