Decision Region Connectivity Analysis: A Method for Analyzing High-Dimensional Classifiers

In this paper we present a method to extract qualitative information from any classification model that uses decision regions to generalize (e.g., feed-forward neural nets, SVMs, etc). The method's complexity is independent of the dimensionality of the input data or model, making it computationally feasible for the analysis of even very high-dimensional models. The qualitative information extracted by the method can be directly used to analyze the classification strategies employed by a model, and also to compare strategies across different model types.

[1]  N. Shor,et al.  New algorithms for constructing optimal circumscribed and inscribed ellipsoids , 1992 .

[2]  Ethem Alpaydin,et al.  Cascading classifiers , 1998, Kybernetika.

[3]  Binhai Zhu,et al.  Approximating Convex Polyhedra with Axis-Parallel Boxes , 1997, Int. J. Comput. Geom. Appl..

[4]  Ashutosh Sabharwal,et al.  Set estimation via ellipsoidal approximations , 1997, IEEE Trans. Signal Process..

[5]  I. Singer.  Abstract Convex Analysis , 1997 .

[6]  D. Titterington.  Estimation of Correlation Coefficients by Ellipsoidal Trimming , 1978 .

[7]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[8]  Jan Paul Siebert,et al.  Vehicle Recognition Using Rule Based Methods , 1987 .

[10]  Jordan B. Pollack,et al.  Theory and scope of exact representation extraction from feed-forward networks , 2002, Cognitive Systems Research.

[11]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[12]  P. Rousseeuw.  5 Introduction to positive-breakdown methods , 1997 .

[13]  Chris Thornton.  Separability is a Learner's Best Friend , 1997, NCPW.

[14]  Panos M. Pardalos,et al.  The maximum clique problem , 1994, J. Glob. Optim..

[15]  Shlomo Moran,et al.  Non Deterministic Polynomial Optimization Problems and their Approximations , 1977, Theor. Comput. Sci..

[16]  László Lovász,et al.  Algorithmic theory of numbers, graphs and convexity , 1986, CBMS-NSF regional conference series in applied mathematics.

[17]  Sergey Bereg,et al.  Covering a set of points by two axis-parallel boxes , 1997, CCCG.

[18]  Arne Frick,et al.  Fast Interactive 3-D Graph Visualization , 1995, GD.

[19]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[20]  Cao Feng,et al.  STATLOG: COMPARISON OF CLASSIFICATION ALGORITHMS ON LARGE REAL-WORLD PROBLEMS , 1995 .