An Instance-Frequency-Weighted Regularization Scheme for Non-Negative Latent Factor Analysis on High-Dimensional and Sparse Data

High-dimensional and sparse (HiDS) data with non-negativity constraints are commonly seen in industrial applications, such as recommender systems. They can be modeled into an HiDS matrix, from which non-negative latent factor analysis (NLFA) is highly effective in extracting useful features. Preforming NLFA on an HiDS matrix is ill-posed, desiring an effective regularization scheme for avoiding overfitting. Current models mostly adopt a standard <inline-formula> <tex-math notation="LaTeX">${L} _{2}$ </tex-math></inline-formula> scheme, which does not consider the imbalanced distribution of known data in an HiDS matrix. From this point of view, this paper proposes an instance-frequency-weighted regularization (IR) scheme for NLFA on HiDS data. It specifies the regularization effects on each latent factors with its relevant instance count, i.e., instance-frequency, which clearly describes the known data distribution of an HiDS matrix. By doing so, it achieves finely grained modeling of regularization effects. The experimental results on HiDS matrices from industrial applications demonstrate that compared with an <inline-formula> <tex-math notation="LaTeX">${L} _{2}$ </tex-math></inline-formula> scheme, an IR scheme enables a resultant model to achieve higher accuracy in missing data estimation of an HiDS matrix.

[1]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[2]  Bradley N. Miller,et al.  GroupLens: applying collaborative filtering to Usenet news , 1997, CACM.

[3]  David Heckerman,et al.  Empirical Analysis of Predictive Algorithms for Collaborative Filtering , 1998, UAI.

[4]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[5]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[6]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[7]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[8]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[9]  Fillia Makedon,et al.  Learning from Incomplete Ratings Using Non-negative Matrix Factorization , 2006, SDM.

[10]  Hyunsoo Kim,et al.  Sparse Non-negative Matrix Factorizations via Alternating Non-negativity-constrained Least Squares , 2006 .

[11]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[12]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[13]  Limsoon Wong,et al.  Author's Personal Copy Increasing the Reliability of Protein Interactomes , 2022 .

[14]  Jimeng Sun,et al.  MetaFac: community discovery via relational hypergraph factorization , 2009, KDD.

[15]  Yihong Gong,et al.  Fast nonparametric matrix factorization for large-scale collaborative filtering , 2009, SIGIR.

[16]  Domonkos Tikk,et al.  Scalable Collaborative Filtering Approaches for Large Recommender Systems , 2009, J. Mach. Learn. Res..

[17]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[18]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Zhu-Hong You,et al.  Using manifold embedding for assessing and predicting protein interactions from high-throughput experimental data , 2010, Bioinform..

[20]  Ruslan Salakhutdinov,et al.  Collaborative Filtering in a Non-Uniform World: Learning with the Weighted Trace Norm , 2010, NIPS.

[21]  Yin Zhang,et al.  An alternating direction algorithm for matrix completion with nonnegative factors , 2011, Frontiers of Mathematics in China.

[22]  Y. Narahari,et al.  A Shapley Value-Based Approach to Discover Influential Nodes in Social Networks , 2011, IEEE Transactions on Automation Science and Engineering.

[23]  Yehuda Koren,et al.  Advances in Collaborative Filtering , 2011, Recommender Systems Handbook.

[24]  Chao Liu,et al.  Recommender systems with social regularization , 2011, WSDM '11.

[25]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[26]  Zibin Zheng,et al.  QoS-Aware Web Service Recommendation by Collaborative Filtering , 2011, IEEE Transactions on Services Computing.

[27]  Charles A. Bouman,et al.  The Sparse Matrix Transform for Covariance Estimation and Analysis of High Dimensional Signals , 2011, IEEE Transactions on Image Processing.

[28]  Ning Zhou,et al.  A Hybrid Probabilistic Model for Unified Collaborative and Content-Based Image Tagging , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Qiang Yang,et al.  Tracking Mobile Users in Wireless Networks via Semi-Supervised Colocalization , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Yung-Yu Chuang,et al.  Collaborative video reindexing via matrix factorization , 2012, TOMCCAP.

[31]  Tommy W. S. Chow,et al.  M-Isomap: Orthogonal Constrained Marginal Isomap for Nonlinear Dimensionality Reduction , 2013, IEEE Transactions on Cybernetics.

[32]  Zibin Zheng,et al.  Predicting Quality of Service for Selection by Neighborhood-Based Collaborative Filtering , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[33]  Sotirios Chatzis,et al.  Nonparametric bayesian multitask collaborative filtering , 2013, CIKM.

[34]  Jing Liu,et al.  A Multiobjective Evolutionary Algorithm Based on Similarity for Community Detection From Signed Social Networks , 2014, IEEE Transactions on Cybernetics.

[35]  Lin Wu,et al.  A Fast Algorithm for Nonnegative Matrix Factorization and Its Convergence , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[36]  N. Latha,et al.  Personalized Recommendation Combining User Interest and Social Circle , 2015 .

[37]  Jun Zhou,et al.  Multitask Sparse Nonnegative Matrix Factorization for Joint Spectral–Spatial Hyperspectral Imagery Denoising , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Xiaochun Cao,et al.  A Unified Semi-Supervised Community Detection Framework Using Latent Space Graph Regularization , 2015, IEEE Transactions on Cybernetics.

[39]  Johan A. K. Suykens,et al.  Kernelized Elastic Net Regularization: Generalization Bounds, and Sparse Recovery , 2016, Neural Computation.

[40]  Jane You,et al.  Correlated Logistic Model With Elastic Net Regularization for Multilabel Image Classification , 2016, IEEE Transactions on Image Processing.

[41]  MengChu Zhou,et al.  A Nonnegative Latent Factor Model for Large-Scale Sparse Matrices in Recommender Systems via Alternating Direction Method , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[42]  Shuai Li,et al.  Efficient Extraction of Non-negative Latent Factors from High-Dimensional and Sparse Matrices in Industrial Applications , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[43]  MengChu Zhou,et al.  Generating Highly Accurate Predictions for Missing QoS Data via Aggregating Nonnegative Latent Factor Models , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[44]  MengChu Zhou,et al.  A Novel Approach to Extracting Non-Negative Latent Factors From Non-Negative Big Sparse Matrices , 2016, IEEE Access.

[45]  MengChu Zhou,et al.  Highly Efficient Framework for Predicting Interactions Between Proteins , 2017, IEEE Transactions on Cybernetics.

[46]  George Trigeorgis,et al.  A Deep Matrix Factorization Method for Learning Attribute Representations , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Kenli Li,et al.  An Efficient Parallelization Approach for Large-Scale Sparse Non-Negative Matrix Factorization Using Kullback-Leibler Divergence on Multi-GPU , 2017, 2017 IEEE International Symposium on Parallel and Distributed Processing with Applications and 2017 IEEE International Conference on Ubiquitous Computing and Communications (ISPA/IUCC).

[48]  Changjun Jiang,et al.  Partition-based collaborative tensor factorization for POI recommendation , 2017, IEEE/CAA Journal of Automatica Sinica.

[49]  Li Zhang,et al.  Structured Latent Label Consistent Dictionary Learning for Salient Machine Faults Representation-Based Robust Classification , 2017, IEEE Transactions on Industrial Informatics.

[50]  Sudipto Guha,et al.  Elastic Nonnegative Matrix Factorization , 2018, 2018 IEEE International Conference on Data Mining Workshops (ICDMW).