A Randomized Approach for Crowdsourcing in the Presence of Multiple Views

Driven by the dramatic growth of data both in terms of the size and sources, learning from heterogeneous data is emerging as an important research direction for many real applications. One of the biggest challenges of this type of problem is how to meaningfully integrate heterogeneous data to considerably improve the generality and quality of the learning model. In this paper, we first present a unified learning framework that aims to leverage the structural information from two types of data heterogeneity: view heterogeneity (as in multi-view learning) and worker heterogeneity (as in crowdsourcing). The objective follows the principles of view consistency and worker consensus by minimizing the loss term with a regularized prediction tensor. We then propose to relax and solve the optimization framework with an iterative updating method. We also prove that the gradient of the most time-consuming updating block is separable with respect to the workers, which leads to a randomized algorithm with faster speed and better convergence. Finally, we compare the proposed method with several state-of-the-arts and demonstrate its effectiveness on various data sets.

[1]  A. P. Dawid,et al.  Maximum Likelihood Estimation of Observer Error‐Rates Using the EM Algorithm , 1979 .

[2]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[3]  P. Tseng.  Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization , 2001 .

[4]  Shotaro Akaho,et al.  A kernel method for canonical correlation analysis , 2006, ArXiv.

[5]  Vikas Sindhwani,et al.  An RKHS for multi-view learning and manifold co-regularization , 2008, ICML '08.

[6]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[7]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[8]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[9]  Peng Dai,et al.  Decision-Theoretic Control of Crowd-Sourced Workflows , 2010, AAAI.

[10]  Gerardo Hermosillo,et al.  Learning From Crowds , 2010, J. Mach. Learn. Res..

[11]  Jingrui He,et al.  A Graphbased Framework for Multi-Task Multi-View Learning , 2011, ICML.

[12]  Dean P. Foster,et al.  Multi-View Learning of Word Embeddings via CCA , 2011, NIPS.

[13]  Yixin Chen,et al.  Automatic Feature Decomposition for Single View Co-training , 2011, ICML.

[14]  Jian Peng,et al.  Variational Inference for Crowdsourcing , 2012, NIPS.

[15]  L. Andrews,et al.  Is the whole greater than the sum of its parts? , 2012, Hereditary Cancer in Clinical Practice.

[16]  John C. Platt,et al.  Learning from the Wisdom of Crowds by Minimax Entropy , 2012, NIPS.

[17]  Dacheng Tao,et al.  A Survey on Multi-view Learning , 2013, ArXiv.

[18]  Huan Liu,et al.  Unsupervised Feature Selection for Multi-View Data in Social Media , 2013, SDM.

[19]  Vittorio Murino,et al.  A unifying framework for vector-valued manifold regularization and multi-view learning , 2013, ICML.

[20]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2013, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Feiping Nie,et al.  Multi-View Clustering and Feature Learning via Structured Sparsity , 2013, ICML.

[22]  Peter Richtárik,et al.  Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function , 2011, Mathematical Programming.

[23]  Jingrui He,et al.  MUVIR: Multi-View Rare Category Detection , 2015, IJCAI.

[24]  Jingrui He,et al.  Crowdsourcing via Tensor Augmentation and Completion , 2016, IJCAI.

[25]  Hanghang Tong,et al.  FINAL: Fast Attributed Network Alignment , 2016, KDD.

[26]  Lei Xie,et al.  FASCINATE: Fast Cross-Layer Dependency Inference on Multi-layered Networks , 2016, KDD.

[27]  Yang Pei,et al.  Heterogeneous Representation Learning with Structured Sparsity Regularization , 2016 .

[28]  Jingrui He,et al.  Finding Cut from the Same Cloth: Cross Network Link Recommendation via Joint Matrix Factorization , 2017, AAAI.

[29]  Jingrui He,et al.  A Local Algorithm for Structure-Preserving Graph Cut , 2017, KDD.

[30]  Jingrui He,et al.  MultiC2: an Optimization Framework for Learning from Task and Worker Dual Heterogeneity , 2017, SDM.

[31]  Huan Liu,et al.  Attributed Network Embedding for Learning in a Dynamic Environment , 2017, CIKM.