Process Analytics Approach for R&D Project Selection

R&D project selection plays an important role in government funding agencies, as allocation of billions of dollars among the proposals deemed highly influential and contributive solely depend on it. Efficacious assignment of reviewers is one of the most critical processes that controls the quality of the entire project selection and also has a serious implication on business profit. Current methods that focus on workflow automation are more efficient than manual assignment; however, they are not effective, as they fail to consider the real insight of core tasks. Other decision models that analyze core tasks are effective but inefficient when handling large amounts of submissions, and they suffer from irrelevant assignment. Furthermore, they largely ignore real deep insight of back-end data such as quality of the reviewers (e.g., quality and citation impact of their produced research) and the effect of social relationships in project selection processes that are essential for identifying reviewers for interdisciplinary proposal evaluation. In light of these deficiencies, this research proposes a novel hybrid process analytics approach to decompose the complex reviewer assignment process into manageable subprocesses and applies data-driven decision models cum process analytics systematically from a triangular perspective via the research analytics framework to achieve high operational efficiencies and high-quality assignment. It also analyzes big data from scientific databases and generates visualized decision-ready information to support effective decision making. The proposed approach has been implemented to aid the project selection process of the largest funding agency in China and has been tested. The test results show that the proposed approach has the potential to add great benefits, including cost saving, improved effectiveness, and increased business value.

[1]  Yuen-Hsien Tseng,et al.  Text mining techniques for patent analysis , 2007, Inf. Process. Manag..

[2]  M. Newman,et al.  The structure of scientific collaboration networks. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Edward A. Fox,et al.  Combination of Multiple Searches , 1993, TREC.

[4]  Patrick Martin,et al.  Clustering WSDL Documents to Bootstrap the Discovery of Web Services , 2010, 2010 IEEE International Conference on Web Services.

[5]  Jian-Yun Nie,et al.  Inferential language models for information retrieval , 2006, TALIP.

[6]  Shusaku Tsumoto,et al.  Text Categorization with Considering Temporal Patterns of Term Usages , 2010, 2010 IEEE International Conference on Data Mining Workshops.

[7]  Hsinchun Chen,et al.  Recommendation as link prediction in bipartite graphs: A graph kernel-based machine learning approach , 2013, Decis. Support Syst..

[8]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[9]  W.M.P. van der Aalst,et al.  Business Process Management: A Comprehensive Survey , 2013 .

[10]  M E J Newman,et al.  Fast algorithm for detecting community structure in networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[11]  Anil K. Jain Data clustering: 50 years beyond K-means , 2008, Pattern Recognit. Lett..

[12]  Toramatsu Shintani,et al.  A paper recommendation mechanism for the research support system Papits , 2005, International Workshop on Data Engineering Issues in E-Commerce.

[13]  Amy J. C. Trappey,et al.  Clustering patents using non-exhaustive overlaps , 2010 .

[14]  Babajide Osatuyi Information sharing on social media sites , 2014 .

[15]  Akiko Aizawa,et al.  An information-theoretic perspective of tf-idf measures , 2003, Inf. Process. Manag..

[16]  Johan Bollen,et al.  The convergence of digital libraries and the peer-review process , 2005, J. Inf. Sci..

[17]  James Z. Wang,et al.  Concept Forest: A New Ontology-assisted Text Document Similarity Measurement Method , 2007, IEEE/WIC/ACM International Conference on Web Intelligence (WI'07).

[18]  Boaz Golany,et al.  Optimal Allocation of Proposals to Reviewers to Facilitate Effective Ranking , 2005, Manag. Sci..

[19]  Fan Wang,et al.  A Survey on Reviewer Assignment Problem , 2008, IEA/AIE.

[20]  Raymond Y. K. Lau,et al.  Leveraging the web context for context-sensitive opinion mining , 2009, 2009 2nd IEEE International Conference on Computer Science and Information Technology.

[21]  Edward A. Fox,et al.  Recommender Systems Research: A Connection-Centric Survey , 2004, Journal of Intelligent Information Systems.

[22]  M. Hasan,et al.  Using Publications and Domain Knowledge to Build Research Profiles: An Application in Automatic Reviewer Assignment , 2007, 2007 International Conference on Information and Communication Technology.

[23]  Jun Wang,et al.  A Group Decision Support Approach to Evaluate Experts for R&D Project Selection , 2008, IEEE Transactions on Engineering Management.

[24]  Limin Yin,et al.  Application of two fuzzy c-means clustering algorithms in segmenting the sonar image from a small underwater target into multi-regions , 2011, 2011 Second International Conference on Mechanic Automation and Control Engineering.

[25]  Michael J. Pazzani,et al.  Mining for proposal reviewers: lessons learned at the national science foundation , 2006, KDD '06.

[26]  Steven B. Andrews,et al.  Structural Holes: The Social Structure of Competition , 1995, The SAGE Encyclopedia of Research Design.

[27]  Hsinchun Chen,et al.  Business stakeholder analyzer: An experiment of classifying stakeholders on the Web , 2009, J. Assoc. Inf. Sci. Technol..

[28]  Raymond Y. K. Lau,et al.  Text mining and probabilistic language modeling for online review spam detection , 2012, TMIS.

[29]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[30]  W. Bruce Croft,et al.  Cluster-based retrieval using language models , 2004, SIGIR '04.

[31]  Wei Xu,et al.  A decision support approach for assigning reviewers to proposals , 2010, Expert Syst. Appl..

[32]  Norman P. Archer,et al.  Project portfolio selection through decision support , 2000, Decis. Support Syst..

[33]  Gilad Mishne,et al.  Blocking Blog Spam with Language Model Disagreement , 2005, AIRWeb.

[34]  Fan Wang,et al.  A Comprehensive Survey of the Reviewer Assignment Problem , 2010, Int. J. Inf. Technol. Decis. Mak..

[35]  Ray Reagans,et al.  Network Structure and Knowledge Transfer: The Effects of Cohesion and Range , 2003 .

[36]  Vladimir Batagelj,et al.  Exploratory Social Network Analysis with Pajek , 2005 .

[37]  Richard F. Hartl,et al.  Reviewer Assignment for Scientific Articles using Memetic Algorithms , 2007, Metaheuristics.

[38]  Peter Trkman,et al.  The impact of business analytics on supply chain performance , 2010, Decis. Support Syst..

[39]  Ron Chi-Wai Kwok,et al.  An organizational decision support system for effective R&D project selection , 2005, Decis. Support Syst..

[40]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[41]  Fabio Casati,et al.  Business Process Intelligence , 2018, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[42]  Hilary Cheng,et al.  An ontology-based business intelligence application in a financial knowledge management system , 2009, Expert Syst. Appl..

[43]  András A. Benczúr,et al.  Detecting nepotistic links by language model disagreement , 2006, WWW '06.

[44]  Jun Wang,et al.  A Hybrid Knowledge and Model Approach for Reviewer Assignment , 2007, 2007 40th Annual Hawaii International Conference on System Sciences (HICSS'07).

[45]  Pin-Yu Chu,et al.  A fuzzy AHP application in government-sponsored R&D project selection☆ , 2008 .

[46]  N. Black,et al.  Development of the review quality instrument (RQI) for assessing peer reviews of manuscripts. , 1999, Journal of clinical epidemiology.

[47]  Yonggui Dong,et al.  A cosine similarity-based negative selection algorithm for time series novelty detection , 2006 .

[48]  Thomas L. Griffiths,et al.  The Author-Topic Model for Authors and Documents , 2004, UAI.

[49]  Radhika Santhanam,et al.  The Effects of Social Network Structure on Enterprise Systems Success: A Longitudinal Multilevel Analysis , 2012, Inf. Syst. Res..

[50]  M. Stumptner,et al.  Finding Experts By Semantic Matching of User Profiles , 2008 .

[51]  Michalis Vazirgiannis,et al.  Word Sense Disambiguation with Spreading Activation Networks Generated from Thesauri , 2007, IJCAI.

[52]  Wei Zheng,et al.  Social capital and knowledge transfer: A multi-level analysis , 2011 .

[53]  Jian Ma,et al.  A social network-empowered research analytics framework for project selection , 2013, Decis. Support Syst..

[54]  Jian Ma,et al.  An Ontology-Based Text-Mining Method to Cluster Proposals for Research Project Selection , 2012, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[55]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[56]  Qiudan Li,et al.  QuestionHolic: Hot topic discovery and trend analysis in community question answering systems , 2011, Expert Syst. Appl..

[57]  Stefano Ferilli,et al.  Automatic Topics Identification for Reviewer Assignment , 2006, IEA/AIE.