Machine Learning Approach to Task Ranking

There are variety of methods and algorithms that can be used to overcome the ranking problem. Task ranking is one of the problems that can be solved by using a machine learning algorithm ranking problem. This work focuses on finding the right approach and corresponding algorithms in the process of ranking to be able to help people in determining which jobs have a higher priority than others. Our approach is to compare several algorithms performed in the process of ranking that are Bipartite Ranking, k-partite Ranking, and Ranking by pairwise comparison. We're used questionnaires and deployment of prototype of Intelligent Personal Assistant Agent to apply the appropriate algorithm in intelligence agent in arranging task priority in daily activity that must be done by the users. After training dataset and evaluate the validation dataset using NDCG, it is found that the collaborative ranking used have a more accurate value / lower variance test evaluation because it uses a large dataset and smaller training dataset. We found that labeling for more than 2 values it is not recommended to use a bipartite ranking if there are many repetitive data, both k-partite ranking and rank by pairwise comparison are able to be used for multi-dimensional data labeling.

[1]  Hang Li,et al.  A Short Introduction to Learning to Rank , 2011, IEICE Trans. Inf. Syst..

[2]  Dan Roth,et al.  Learnability of Bipartite Ranking Functions , 2005, COLT.

[3]  Timothy W. Finin,et al.  A personal agent application for the semantic web , 2002, AAAI 2002.

[4]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[5]  Shivani Agarwal,et al.  Surrogate regret bounds for bipartite ranking via strongly proper losses , 2012, J. Mach. Learn. Res..

[6]  Pang-Ning Tan,et al.  Recommendation via Query Centered Random Walk on K-Partite Graph , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[7]  Shipeng Yu,et al.  Eliminating Spammers and Ranking Annotators for Crowdsourced Labeling Tasks , 2012, J. Mach. Learn. Res..

[8]  Suhrid Balakrishnan,et al.  Collaborative ranking , 2012, WSDM '12.

[9]  Xin Geng,et al.  Multilabel Ranking with Inconsistent Rankers , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Tsuyoshi Murata,et al.  Detecting Communities in K-Partite K-Uniform (Hyper)Networks , 2011, Journal of Computer Science and Technology.

[11]  Chris Buckley,et al.  OHSUMED: an interactive retrieval evaluation and new large test collection for research , 1994, SIGIR '94.

[12]  Karen L. Myers,et al.  The Design of a Proactive Personal Agent for Task Management , 2012, Int. J. Artif. Intell. Tools.

[13]  Thomas Gärtner,et al.  Label Ranking Algorithms: A Survey , 2010, Preference Learning.

[14]  John Riedl,et al.  GroupLens: an open architecture for collaborative filtering of netnews , 1994, CSCW '94.

[15]  Eyke Hüllermeier,et al.  Label ranking by learning pairwise preferences , 2008, Artif. Intell..