Conundrum of Combinatorial Complexity

This paper examines fundamental problems underlying difficulties encountered by pattern recognition algorithms, neural networks, and rule systems. These problems are manifested as combinatorial complexity of algorithms, of their computational or training requirements. The paper relates particular types of complexity problems to the roles of a priori knowledge and adaptive learning. Paradigms based on adaptive learning lead to the complexity of training procedures, while nonadaptive rule-based paradigms lead to complexity of rule systems. Model-based approaches to combining adaptivity with a priori knowledge lead to computational complexity. Arguments are presented for the Aristotelian logic being culpable for the difficulty of combining adaptivity and a priority. The potential role of the fuzzy logic in overcoming current difficulties is discussed. Current mathematical difficulties are related to philosophical debates of the past.

[1]  L. N. Kanal,et al.  Uncertainty in Artificial Intelligence 5 , 1990 .

[2]  Moses Maimonides,et al.  A Guide for the Perplexed , 1954 .

[3]  R. Bellman,et al.  V. Adaptive Control Processes , 1964 .

[4]  Earl B. Hunt,et al.  Machine learning: An artificial intelligence approach (vol. 2): R. S. Michalski, J. G. Carbonell, and T. M. Mitchell (Eds.). Los Alton, CA: Morgan Kaufmann, 1986. Pp. x + 738. $39.95 , 1987 .

[5]  M. Mead,et al.  Cybernetics , 1953, The Yale Journal of Biology and Medicine.

[6]  Ee-Peng Lim,et al.  Semantic networks and associative databases: two approaches to knowledge representation and reasoning , 1992, IEEE Expert.

[7]  Noam Chomsky,et al.  Language and Mind , 1973 .

[8]  N. Wiener,et al.  Behavior, Purpose and Teleology , 1943, Philosophy of Science.

[9]  W. Pitts,et al.  How we know universals; the perception of auditory and visual forms. , 1947, The Bulletin of mathematical biophysics.

[10]  Tod S. Levitt,et al.  Uncertainty in artificial intelligence , 1988 .

[11]  Asaf Degani,et al.  Procedures in complex systems: the airline cockpit , 1997, IEEE Trans. Syst. Man Cybern. Part A.

[12]  Ryszard S. Michalski,et al.  Machine learning: an artificial intelligence approach volume III , 1990 .

[13]  Jan Koster,et al.  Levels of syntactic representation , 1981 .

[14]  Marvin Minsky,et al.  A framework for representing knowledge , 1974 .

[15]  Ron Sun,et al.  Computational Architectures Integrating Neural And Symbolic Processes , 1994 .

[16]  W. Eric L. Grimson,et al.  Introduction to the Special Issue on Interpretation of 3-D Scenes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Gert Webelhuth,et al.  Government and binding theory and the minimalist program : principles and parameters in syntactic theory , 1997 .

[18]  Aquinas Saint Thomas,et al.  Summa contra gentiles , 1863 .

[19]  S. Grossberg.  How does a brain build a cognitive code , 1980 .

[20]  Satosi Watanabe,et al.  Pattern Recognition: Human and Mechanical , 1985 .

[21]  I. Kant,et al.  Critique of Pure Reason: Glossary , 1998 .

[22]  Rakesh Mohan,et al.  Multidimensional indexing for recognizing visual shapes , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Stephen Grossberg,et al.  Nonlinear neural networks: Principles, mechanisms, and architectures , 1988, Neural Networks.

[24]  Charles R. Dyer,et al.  Model-based recognition in robot vision , 1986, CSUR.

[25]  Marvin Minsky,et al.  A framework for representing knowledge" in the psychology of computer vision , 1975 .

[26]  B. Skinner.  Is it behaviorism? , 1986, Behavioral and Brain Sciences.

[27]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[28]  Andrew P. Sage,et al.  Uncertainty in Artificial Intelligence , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[29]  Marvin Minsky,et al.  Semantic Information Processing , 1968 .

[30]  W. McCulloch,et al.  Embodiments of Mind , 1966 .

[31]  John R. Anderson,et al.  MACHINE LEARNING An Artificial Intelligence Approach , 2009 .

[32]  W. Grimson,et al.  Model-Based Recognition and Localization from Sparse Range or Tactile Data , 1984 .

[33]  Norbert Hornstein,et al.  Explanation in Linguistics: The Logical Problem of Language Acquisition , 1982 .

[34]  Rodney A. Brooks,et al.  Model-Based Three-Dimensional Interpretations of Two-Dimensional Images , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Yehezkel Lamdan,et al.  Geometric Hashing: A General And Efficient Model-based Recognition Scheme , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[36]  S. Vaisrub,et al.  A guide for the perplexed. , 1966, Manitoba medical review.

[37]  Rudolf P. Botha,et al.  Challenging Chomsky: The Generative Garden Game , 1989 .

[38]  Toshio Odanaka,et al.  ADAPTIVE CONTROL PROCESSES , 1990 .

[39]  H. R. Keshavan,et al.  Introduction to the Special Section on Probabilistic Reasoning , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[41]  Åsa Rudström,et al.  Applications of Machine Learning , 2020, Algorithms for Intelligent Systems.

[42]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[43]  Yunde Jia.  Description and recognition of curved objects , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol. III. Conference C: Image, Speech and Signal Analysis,.

[44]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[45]  Nils J. Nilsson,et al.  Artificial Intelligence , 1974, IFIP Congress.