Quantitative Emergence -- A Refined Approach Based on Divergence Measures

The article addresses the phenomenon of emergence from a technical viewpoint. A technical system exhibits emergence when it has certain kinds of properties or qualities that are irreducible in the sense that they are not traceable to the constituent parts of the system. In particular, we show how emergence in technical systems can be detected and measured gradually using techniques from the field of probability theory and information theory. To detect or measure emergence we observe the system and extract characteristic attributes from those observations. As an extension of earlier work in the field, we propose emergence measures that are well-suited for continuous attributes (or hybrid attribute sets) using either non-parametric or model-based probability density estimation techniques. We also replace the known entropy-based emergence measures by divergence measures for probability densities (e.g., the Kullback-Leibler divergence or the Hellinger distance). We discuss advantages and drawbacks of these measures by means of some simulation experiments using artificial data sets and a real-world data set from the field of intrusion detection.

[1]  J. Crutchfield.  The calculi of emergence: computation, dynamics and induction , 1994 .

[2]  C. Shalizi,et al.  Causal architecture, complexity and self-organization in time series and cellular automata , 2001 .

[3]  Alexander Hofmann,et al.  Online Intrusion Alert Aggregation with Generative Data Stream Modeling , 2011, IEEE Transactions on Dependable and Secure Computing.

[4]  Martin Lauer,et al.  A Mixture Approach to Novelty Detection Using Training Data with Outliers , 2001, ECML.

[5]  Sameer Singh,et al.  Novelty detection: a review - part 1: statistical approaches , 2003, Signal Process..

[6]  Marco Vannucci,et al.  Outlier Detection Methods for Industrial Applications , 2008, ICRA 2008.

[7]  Jim Austin,et al.  Neural networks for novelty detection in airframe strain data , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[8]  Bernhard Sick,et al.  Functional Knowledge Exchange Within an Intelligent Distributed System , 2007, ARCS.

[9]  Bernhard Sick,et al.  Controlled Emergence and Self-Organization , 2008, Organic Computing.

[10]  Christian Müller-Schloer,et al.  Emergence in Organic Computing Systems: Discussion of a Controversial Concept , 2006, ATC.

[11]  Hermann de Meer,et al.  On Autonomy and Emergence in Self-Organizing Systems , 2008, IWSOS.

[12]  David A. Clifton,et al.  A Framework for Novelty Detection in Jet Engine Vibration Data , 2007 .

[13]  Tom De Wolf,et al.  Decentralised Autonomic Computing: Analysing Self-Organising Emergent Behaviour using Advanced Numerical Methods , 2005, Second International Conference on Autonomic Computing (ICAC'05).

[14]  Christian Müller-Schloer,et al.  Quantitative Emergence , 2006, 2006 IEEE Mountain Workshop on Adaptive and Learning Systems.

[15]  Alexander Hofmann,et al.  On the versatility of radial basis function neural networks: A case study in the field of intrusion detection , 2010, Inf. Sci..

[16]  Hyoungjoo Lee,et al.  On-line novelty detection using the Kalman filter and extreme value theory , 2008, 2008 19th International Conference on Pattern Recognition.

[17]  Bernhard Sick,et al.  Learning by teaching versus learning by doing: Knowledge exchange in organic agent systems , 2009, 2009 IEEE Symposium on Intelligent Agents.

[18]  S. Roberts.  EXTREME VALUE STATISTICS FOR NOVELTY DETECTION IN BIOMEDICAL DATA PROCESSING , 2000 .

[19]  Sameer Singh,et al.  An approach to novelty detection applied to the classification of image regions , 2004, IEEE Transactions on Knowledge and Data Engineering.

[20]  Bernhard Sick,et al.  Training of radial basis function classifiers with resilient propagation and variational Bayesian inference , 2009, 2009 International Joint Conference on Neural Networks.

[21]  Sameer Singh,et al.  Novelty detection: a review - part 2: : neural network based approaches , 2003, Signal Process..

[22]  Hujun Yin,et al.  Self-Organization, Emergence and Multi-Agent Systems , 2005, 2005 International Conference on Neural Networks and Brain.

[23]  S. Roberts.  Novelty detection using extreme value statistics , 1999 .

[24]  Christopher M. Bishop,et al.  Novelty detection and neural network validation , 1994 .