A Variance Minimization Criterion to Active Learning on Graphs

We consider the problem of active learning over the vertices in a graph, without feature representation. Our study is based on the common graph smoothness assumption, which is formulated in a Gaussian random field model. We analyze the probability distribution over the unlabeled vertices conditioned on the label information, which is a multivariate normal with the mean being the harmonic solution over the field. Then we select the nodes to label such that the total variance of the distribution on the unlabeled data, as well as the expected prediction error, is minimized. In this way, the classifier we obtain is theoretically more robust. Compared with existing methods, our algorithm has the advantage of selecting data in a batch offline mode with solid theoretical support. We show improved performance over existing label selection criteria on several real world data sets.

[1]  Gene H. Golub,et al.  Matrix computations , 1983 .

[2]  Editors , 1986, Brain Research Bulletin.

[3]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[4]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[5]  D. Harville.  Matrix Algebra From a Statistician's Perspective , 1998 .

[6]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[7]  Avrim Blum,et al.  Learning from Labeled and Unlabeled Data using Graph Mincuts , 2001, ICML.

[8]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[9]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[10]  J. Lafferty,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[11]  Steve Hanneke,et al.  An analysis of graph cut size for transductive learning , 2006, ICML.

[12]  Nicolas Le Roux,et al.  Label Propagation and Quadratic Criterion , 2006, Semi-Supervised Learning.

[13]  Johan A. K. Suykens,et al.  Margin based Transductive Graph Cuts using Linear Programming , 2007, AISTATS.

[14]  Jianping Yin,et al.  Graph-Based Active Learning Based on Label Propagation , 2008, MDAI.

[15]  En Zhu,et al.  A Scalable Algorithm for Graph-Based Active Learning , 2008, FAW.

[16]  Jeff A. Bilmes,et al.  Label Selection on Graphs , 2009, NIPS.

[17]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[18]  Sofus A. Macskassy.  Using graph-based metrics with empirical risk minimization to speed up active learning on networked data , 2009, KDD.

[19]  Jie Tang,et al.  Combining link and content for collective active learning , 2010, CIKM.

[20]  Lise Getoor,et al.  Active Learning for Networked Data , 2010, ICML.

[21]  Xiaofei He,et al.  Laplacian Regularized D-Optimal Design for Active Learning and Its Application to Image Retrieval , 2010, IEEE Transactions on Image Processing.

[22]  A. Kuwadekar.  Combining Semi-supervised Learning and Relational Resampling for Active Learning in Network Domains , 2010 .

[23]  Hujun Bao,et al.  A Variance Minimization Criterion to Feature Selection Using Laplacian Regularization , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Yi Yang,et al.  Interactive Video Indexing With Statistical Active Learning , 2012, IEEE Transactions on Multimedia.

[25]  Rong Jin,et al.  Active Learning by Querying Informative and Representative Examples , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.