Relationship between Variants of One-Class Nearest Neighbors and Creating Their Accurate Ensembles

In one-class classification problems, only the data for the target class is available, whereas the data for the non-target class may be completely absent. In this paper, we study one-class nearest neighbor (OCNN) classifiers and their different variants. We present a theoretical analysis to show the relationships among different variants of OCNN that may use different neighbors or thresholds to identify unseen examples of the non-target class. We also present a method based on inter-quartile range for optimizing parameters used in OCNN in the absence of non-target data during training. Then, we propose two ensemble approaches based on random subspace and random projection methods to create accurate OCNN ensembles. We tested the proposed methods on 15 benchmark and real world domain-specific datasets and show that random-projection ensembles of OCNN perform best.

[1]  Victor S. Sheng,et al.  Cost-Sensitive Learning , 2009, Encyclopedia of Data Warehousing and Mining.

[2]  Vera Nadales,et al.  Recognition of Human Motion Related Activities from Sensors , 2010 .

[3]  Martin Steinebach,et al.  Authorship Verification via k-Nearest Neighbor Estimation , 2013 .

[4]  Adrian Burns,et al.  SHIMMER™ – A Wireless Sensor Platform for Noninvasive Biomedical Research , 2010, IEEE Sensors Journal.

[5]  Vito Di Gesù,et al.  A One Class Classifier for Signal Identification: A Biological Case Study , 2008, KES.

[6]  Michael G. Madden,et al.  Multi-Class and Single-Class Classification Approaches to Vehicle Model Recognition from Images , 2005 .

[7]  Shehroz S. Khan,et al.  X-Factor HMMs for Detecting Falls in the Absence of Fall-Specific Training Data , 2014, IWAAL.

[8]  Lior Rokach,et al.  Random Projection Ensemble Classifiers , 2009, ICEIS.

[9]  Victor S. Sheng,et al.  Thresholding for Making Classifiers Cost-sensitive , 2006, AAAI.

[10]  Lior Rokach,et al.  Combining one-class classifiers via meta learning , 2011, CIKM.

[11]  P. Datta.  Characteristic concept representations , 1997 .

[12]  Shehroz S. Khan,et al.  One-class classification: taxonomy of study and review of techniques , 2013, The Knowledge Engineering Review.

[13]  Adriano Lorena Inácio de Oliveira,et al.  A Novel Method for One-Class Classification Based on the Nearest Neighbor Data Description and Structural Risk Minimization , 2007, 2007 International Joint Conference on Neural Networks.

[14]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Aida de Haro-Garćıa,et al.  One-class methods for separating plant/pathogen sequences , 2008 .

[16]  Selim G. Akl,et al.  EFFICIENT CONVEX HULL ALGORITHMS FOR PATTERN RECOGNITION APPLICATIONS. , 1979 .

[17]  Zheng-Zhi Wang,et al.  Center-based nearest neighbor classifier , 2007, Pattern Recognit..

[18]  Robert P. W. Duin,et al.  Support vector domain description , 1999, Pattern Recognit. Lett..

[19]  Dmitriy Fradkin,et al.  Experiments with random projections for machine learning , 2003, KDD '03.

[20]  Robert P. W. Duin,et al.  Data description in subspaces , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[21]  Shehroz S. Khan,et al.  Detecting falls with X-Factor Hidden Markov Models , 2017, Appl. Soft Comput..

[22]  David G. Kirkpatrick,et al.  On the shape of a set of points in the plane , 1983, IEEE Trans. Inf. Theory.

[23]  Petia Radeva,et al.  Approximate polytope ensemble for one-class classification , 2014, Pattern Recognit..

[24]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Ribana Roscher,et al.  Can I Trust My One-Class Classification? , 2014, Remote. Sens..

[26]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[27]  Ian H. Witten,et al.  One-Class Classification by Combining Density and Class Probability Estimation , 2008, ECML/PKDD.

[28]  Caroline Petitjean,et al.  One class random forests , 2013, Pattern Recognit..

[29]  Adriano Lorena Inácio de Oliveira,et al.  Combining nearest neighbor data description and structural risk minimization for one-class classification , 2009, Neural Computing and Applications.

[30]  C. A. Murthy,et al.  DETERMINING THE SHAPE OF A PATTERN CLASS: EXTENSION TO RN , 1997 .

[31]  John A. Quinn,et al.  Factorial Switching Linear Dynamical Systems Applied to Physiological Condition Monitoring , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Borja Calvo,et al.  scmamp: Statistical Comparison of Multiple Algorithms in Multiple Problems , 2016, R J..

[33]  Carlotta Domeniconi,et al.  Nearest neighbor ensemble , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[34]  Dimitris Achlioptas,et al.  Database-friendly random projections , 2001, PODS.

[35]  Tin Kam Ho,et al.  Nearest Neighbors in Random Subspaces , 1998, SSPR/SPR.

[36]  Shehroz S. Khan,et al.  dtFall: decision-theoretic framework to report unseen falls , 2016, PervasiveHealth.

[37]  Shehroz S. Khan,et al.  Towards the detection of unusual temporal events during activities using HMMs , 2012, UbiComp '12.

[38]  C. A. Murthy,et al.  Selection of alpha for alpha-hull in R2 , 1997, Pattern Recognit..

[39]  Francisco Herrera,et al.  A first attempt on evolutionary prototype reduction for nearest neighbor one-class classification , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[40]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[41]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[42]  Vipin Kumar,et al.  Feature bagging for outlier detection , 2005, KDD '05.

[43]  Juan José Rodríguez Diez,et al.  Classifier Ensembles with a Random Linear Oracle , 2007, IEEE Transactions on Knowledge and Data Engineering.

[44]  Shehroz S. Khan.  Classification and Decision-Theoretic Framework for Detecting and Reporting Unseen Falls , 2016 .

[45]  Bernard Chazelle,et al.  An optimal convex hull algorithm in any fixed dimension , 1993, Discret. Comput. Geom..

[46]  Qiang Yang,et al.  Sensor-Based Abnormal Human-Activity Detection , 2008, IEEE Transactions on Knowledge and Data Engineering.

[47]  Elena I. Gaura,et al.  Data set for fall events and daily activities from inertial sensors , 2015, MMSys.

[48]  Yu-Dong Cai,et al.  Prediction of carbamylated lysine sites based on the one-class k-nearest neighbor method. , 2013, Molecular bioSystems.

[49]  Yi-Ning Quan,et al.  Modular ensembles for one-class classification based on density analysis , 2016, Neurocomputing.

[50]  Robert P. W. Duin,et al.  Minimum spanning tree based one-class classifier , 2009, Neurocomputing.

[51]  Vito Di Gesù,et al.  Combining One Class Fuzzy KNN's , 2007, WILF.

[52]  Jordi Vitrià,et al.  Bagged One-Class Classifiers in the Presence of Outliers , 2013, Int. J. Pattern Recognit. Artif. Intell..

[53]  Domenico Cotroneo,et al.  Predicting aging-related bugs using software complexity metrics , 2013, Perform. Evaluation.

[54]  Manolis Tsiknakis,et al.  The MobiFall dataset: An initial evaluation of fall detection algorithms using smartphones , 2013, 13th IEEE International Conference on BioInformatics and BioEngineering.

[55]  Fabrizio Angiulli,et al.  Prototype-Based Domain Description for One-Class Classification , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Loris Nanni,et al.  Experimental comparison of one-class classifiers for online signature verification , 2006, Neurocomputing.

[57]  Stan Matwin,et al.  Addressing the Curse of Imbalanced Training Sets: One-Sided Selection , 1997, ICML.

[58]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[59]  Shehroz S. Khan,et al.  A Survey of Recent Trends in One Class Classification , 2009, AICS.