MindMiner: A Mixed-Initiative Interface for Interactive Distance Metric Learning

We present MindMiner, a mixed-initiative interface for capturing subjective similarity measurements via a combination of new interaction techniques and machine learning algorithms. MindMiner collects qualitative, hard to express similarity measurements from users via active polling with uncertainty and example based visual constraint creation. MindMiner also formulates human prior knowledge into a set of inequalities and learns a quantitative similarity distance metric via convex optimization. In a 12-subject peer-review understanding task, we found MindMiner was easy to learn and use, and could capture users’ implicit knowledge about writing performance and cluster target entities into groups that match subjects’ mental models. We also found that MindMiner’s constraint suggestions and uncertainty polling functions could improve both efficiency and the quality of clustering.

[1]  Aniket Kittur,et al.  Apolo: making sense of large network data by combining rich user interaction and machine learning , 2011, CHI.

[2]  Jean-Daniel Fekete,et al.  NodeTrix: a Hybrid Visualization of Social Networks , 2007, IEEE Transactions on Visualization and Computer Graphics.

[3]  Desney S. Tan,et al.  Effective End-User Interaction with Machine Learning , 2011, AAAI.

[4]  Desney S. Tan,et al.  EnsembleMatrix: interactive visualization to support machine learning with multiple classifiers , 2009, CHI.

[5]  Jimeng Sun,et al.  DICON: Interactive Visual Analysis of Multidimensional Clusters , 2011, IEEE Transactions on Visualization and Computer Graphics.

[6]  Alfred Inselberg,et al.  Parallel coordinates: a tool for visualizing multi-dimensional geometry , 1990, Proceedings of the First IEEE Conference on Visualization: Visualization `90.

[7]  Arindam Banerjee,et al.  Active Semi-Supervision for Pairwise Constrained Clustering , 2004, SDM.

[8]  Desney S. Tan,et al.  Overview based example selection in end user interactive concept learning , 2009, UIST '09.

[9]  Matej Novotny,et al.  Visually Effective Information Visualization of Large Data , 2004 .

[10]  Ben Shneiderman,et al.  Direct manipulation vs. interface agents , 1997, INTR.

[11]  Eric Horvitz,et al.  Principles of mixed-initiative user interfaces , 1999, CHI '99.

[12]  Marie desJardins,et al.  Interactive visual clustering , 2007, IUI '07.

[13]  Desney S. Tan,et al.  Interactive optimization for steering machine classification , 2010, CHI.

[14]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[15]  Desney S. Tan,et al.  CueFlik: interactive concept learning in image search , 2008, CHI.

[16]  Tom M. Mitchell,et al.  Exploring Hierarchical User Feedback in Email Clustering , 2008 .

[17]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[18]  Matthew O. Ward,et al.  Hierarchical parallel coordinates for exploration of large datasets , 1999, Proceedings Visualization '99 (Cat. No.99CB37067).

[19]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[20]  Carla E. Brodley,et al.  Visualization and interactive feature selection for unsupervised data , 2000, KDD '00.

[21]  Andrew McCallum,et al.  Semi-Supervised Clustering with User Feedback , 2003 .

[22]  Claire Cardie,et al.  Proceedings of the Eighteenth International Conference on Machine Learning, 2001, p. 577–584. Constrained K-means Clustering with Background Knowledge , 2022 .

[23]  John F. Canny,et al.  CAAD: an automatic task support system , 2007, CHI.

[24]  Ben Shneiderman,et al.  Integrating statistics and visualization: case studies of gaining clarity during exploratory data analysis , 2008, CHI.

[25]  E. Rosch,et al.  Family resemblances: Studies in the internal structure of categories , 1975, Cognitive Psychology.

[26]  Pattie Maes,et al.  Interface agents , 1996, CHI Conference Companion.

[27]  Steven M. Drucker,et al.  Assisting Users with Clustering Tasks by Combining Metric Learning and Classification , 2010, AAAI.

[28]  Ben Shneiderman,et al.  Interactively Exploring Hierarchical Clustering Results , 2002, Computer.