Using structure-based data transformation method to improve prediction accuracies for small data sets

Small data set problems have been widely considered in many fields, where increasing the prediction ability is the most important goal. This study considers the data structure to identify new data points in a more precise manner, and is thus able to achieve improved prediction capability. The proposed method, named structure-based data transformation, consists of two steps. The first step is using the density-based spatial clustering of applications with noise (DBSCAN) algorithm to separate data sets into clusters, which generates the number of clusters dynamically. The second step is to build up the data transformation function, in which the new attributes are computed using fuzzy membership functions obtained by the corresponding membership grades in each cluster. Three real cases are selected to compare the proposed forecasting model with the linear regression (LR), backpropagation neural network (BPNN), and support vector machine for regression (SVR) methods. The result show that the structure-based data transformation method has better performance than when using the raw data with regard to the error improving rate, mean square error (MSE), and standard deviation (STD).

[1]  Si Wu,et al.  Improving support vector machine classifiers by modifying kernel functions , 1999, Neural Networks.

[2]  Vojislav Kecman,et al.  Adaptive local hyperplane algorithm for learning small medical data sets , 2009, Expert Syst. J. Knowl. Eng..

[3]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[4]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[5]  Kate Smith-Miles,et al.  A meta-learning approach to automatic kernel selection for support vector machines , 2006, Neurocomputing.

[6]  Kumar,et al.  Neural Networks a Classroom Approach , 2004 .

[7]  Theodore B. Trafalis,et al.  A hybrid model for exchange rate prediction , 2006, Decis. Support Syst..

[8]  Morteza Haghir Chehreghani,et al.  Density link-based methods for clustering web pages , 2009, Decis. Support Syst..

[9]  Filiberto Pla,et al.  Non Parametric Local Density-Based Clustering for Multimodal Overlapping Distributions , 2006, IDEAL.

[10]  Der-Chiang Li,et al.  Using mega-trend-diffusion and artificial samples in small data set learning for early flexible manufacturing system scheduling knowledge , 2007, Comput. Oper. Res..

[11]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[12]  Gary J. Koehler,et al.  Using domain-specific knowledge in generalization error bounds for support vector machine learning , 2009, Decis. Support Syst..

[13]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[14]  Chenghu Zhou,et al.  DECODE: a new method for discovering clusters of different densities in spatial data , 2009, Data Mining and Knowledge Discovery.

[15]  I-Cheng Yeh,et al.  Exploring Concrete Slump Model Using Artificial Neural Networks , 2006 .

[16]  Der-Chiang Li,et al.  Utilize bootstrap in small data set learning for pilot run modeling of manufacturing systems , 2008, Expert Syst. Appl..

[17]  Anil K. Jain.  Data clustering: 50 years beyond K-means , 2010, Pattern Recognit. Lett..

[18]  Chin-Chen Chang,et al.  A New Density-Based Scheme for Clustering Based on Genetic Algorithm , 2005, Fundam. Informaticae.

[19]  Der-Chiang Li,et al.  A non-linear quality improvement model using SVR for manufacturing TFT-LCDs , 2012, J. Intell. Manuf..

[20]  Brian Everitt,et al.  Cluster analysis , 1974 .

[21]  Der-Chiang Li,et al.  A non-linearly virtual sample generation technique using group discovery and parametric equations of hypersphere , 2009, Expert Syst. Appl..

[22]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[23]  Emanuela Marrocu,et al.  An Investigation of the Effects of Data Transformation on Nonlinearity , 2006 .

[24]  Chih-Chou Chiu,et al.  Financial time series forecasting using independent component analysis and support vector regression , 2009, Decis. Support Syst..

[25]  Paulo Cortez,et al.  Modeling wine preferences by data mining from physicochemical properties , 2009, Decis. Support Syst..

[26]  Long-Sheng Chen,et al.  Using Functional Virtual Population as assistance to learn scheduling knowledge in dynamic manufacturing environments , 2003 .

[27]  Claudio Moraga,et al.  A diffusion-neural-network for learning from small samples , 2004, Int. J. Approx. Reason..

[28]  Yuzhong Shen,et al.  Noise reduction and edge detection via kernel anisotropic diffusion , 2008, Pattern Recognit. Lett..

[29]  Hans-Peter Kriegel,et al.  OPTICS: ordering points to identify the clustering structure , 1999, SIGMOD '99.

[30]  Peng Liu,et al.  VDBSCAN: Varied Density Based Spatial Clustering of Applications with Noise , 2007, 2007 International Conference on Service Systems and Service Management.

[31]  Tomaso Poggio,et al.  Incorporating prior information in machine learning by creating virtual examples , 1998, Proc. IEEE.

[32]  Swarup Roy,et al.  An Approach to Find Embedded Clusters Using Density Based Techniques , 2005, ICDCIT.

[33]  Der-Chiang Li,et al.  The data complexity index to construct an efficient cross-validation method , 2010, Decis. Support Syst..

[34]  D. Massart,et al.  Looking for natural patterns in data: Part 1. Density-based approach , 2001 .

[35]  Boubakeur Boufama,et al.  A novel SVM+NDA model for classification with an application to face recognition , 2012, Pattern Recognit..