A Serendipitous Software Framework for Facilitating Collaboration in Computational Intelligence

A major flaw in the academic system, particularly pertaining to computer science, is that it rewards specialisation. The highly competitive quest for new scientific developments, or rather the quest for a better reputation and more funding, forces researchers to specialise in their own fields, leaving them little time to properly explore what others are doing, sometimes even within their own field of interest. Even the peer review process, which should provide the necessary balance, fails to achieve much diversity, since reviews are typically performed by persons who are again specialists in the particular field of the work. Further, software implementations are rarely reviewed, having as a consequence the publishing of untenable results. Unfortunately, these factors contribute to an environment which is not conducive to collaboration, a cornerstone of academia — building on the work of others. This work takes a step back and examines the general landscape of computational intelligence from a broad perspective, drawing on multiple disciplines to formulate a collaborative software platform, which is flexible enough to support the needs of this diverse research community. Interestingly, this project did not set out with these goals in mind, rather it evolved, over time, from something more specialised into the general framework described in this dissertation. Design patterns are studied as a means to manage the complexity of the computational intelligence paradigm in a flexible software implementation. Further, this dissertation demonstrates that releasing research software under an open source license eliminates some of the deficiencies of the academic process, while preserving, and even improving, the ability to build a reputation and pursue funding. Two software packages have been produced as products of this research: i) CILib, an open source library of computational intelligence algorithms; and ii) CiClops, which is a virtual laboratory for performing experiments that scale over multiple workstations. Together, these software packages are intended to improve the quality of research output and facilitate collaboration by sharing a repository of simulation data, statistical analysis tools and a single software implementation.

[1]  T. Micceri.  The unicorn, the normal curve, and other improbable creatures. , 1989 .

[2]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[3]  Randy Shepherd,et al.  Object-Oriented Programming , 1994, Lecture Notes in Computer Science.

[4]  P. Suganthan.  Particle swarm optimiser with neighbourhood operator , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[5]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[6]  Lotfi A. Zadeh,et al.  The concept of a linguistic variable and its application to approximate reasoning - II , 1975, Inf. Sci..

[7]  Ali S. Hadi,et al.  Finding Groups in Data: An Introduction to Chster Analysis , 1991 .

[8]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[9]  Joydeep Ghosh,et al.  Efficient Higher-Order Neural Networks for Classification and Function Approximation , 1992, Int. J. Neural Syst..

[10]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[11]  James Kennedy,et al.  Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[12]  Kirk Reinholtz.  Java will be faster than C++ , 2000, SIGP.

[13]  Nils J. Nilsson,et al.  Artificial Intelligence: A New Synthesis , 1997 .

[14]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[15]  John H. Holland,et al.  Outline for a Logical Theory of Adaptive Systems , 1962, JACM.

[16]  Eric S. Raymond,et al.  The cathedral and the bazaar - musings on Linux and Open Source by an accidental revolutionary , 2001 .

[17]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[18]  Ramnivas Laddad,et al.  Aspectj in Action: Practical Aspect-Oriented Programming , 2003 .

[19]  Lotfi A. Zadeh,et al.  The concept of a linguistic variable and its application to approximate reasoning-III , 1975, Inf. Sci..

[20]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[21]  R. W. Dobbins,et al.  Computational intelligence PC tools , 1996 .

[22]  Jeffrey K. Bassett,et al.  An Analysis of Cooperative Coevolutionary Algorithms A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at George Mason University , 2003 .

[23]  Tzilla Elrad,et al.  Aspect-Oriented Software Development , 2004 .

[24]  Petra Perner,et al.  Data Mining - Concepts and Techniques , 2002, Künstliche Intell..

[25]  David Flanagan,et al.  Java in a Nutshell , 1996 .

[26]  Royston M. Roberts,et al.  Serendipity: Accidental Discoveries in Science , 1989 .

[27]  Chong Ho Yu,et al.  AN OVERVIEW OF REMEDIAL TOOLS FOR THE VIOLATION OF PARAMETRIC TEST ASSUMPTIONS IN THE SAS SYSTEM , 2002 .

[28]  WHM Theunissen,et al.  A Preliminary Investigation of the Impact of Open Source Software on Telecommunication Software Development , 2004 .

[29]  Ingo Rechenberg,et al.  Evolutionsstrategie '94 , 1994, Werkstatt Bionik und Evolutionstechnik.

[30]  J. Snyman.  A new and dynamic method for unconstrained minimization , 1982 .

[31]  A.P. Engelbrecht,et al.  Learning to play games using a PSO-based competitive learning approach , 2004, IEEE Transactions on Evolutionary Computation.

[32]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[33]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[34]  Luis Ibáñez,et al.  Free software , 2006, British Dental Journal.

[35]  Wenjun Zhang,et al.  Dissipative particle swarm optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[36]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[37]  W. Daniel Hillis,et al.  Co-evolving parasites improve simulated evolution as an optimization procedure , 1990 .

[38]  Andries Petrus Engelbrecht,et al.  Comparing PSO structures to learn the game of checkers from zero knowledge , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[39]  Rishab Aiyer Ghosh.  Cooking pot markets: an economic model for the trade in free goods and services on the Internet , 1998, First Monday.

[40]  Roger J.-B. Wets,et al.  Minimization by Random Search Techniques , 1981, Math. Oper. Res..

[41]  José Neves,et al.  Watch thy neighbor or how the swarm can learn from its environment , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[42]  S. Breckler,et al.  Applications of covariance structure modeling in psychology: cause for concern? , 1990, Psychological bulletin.

[43]  H. Ishibuchi.  Genetic fuzzy systems: evolutionary tuning and learning of fuzzy knowledge bases , 2004 .

[45]  E. Mayr.  Animal Species and Evolution , 1964 .

[46]  F. van den Bergh,et al.  CIRG@UP OptiBench: a statistically sound framework for benchmarking optimisation algorithms , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[47]  John C. Mitchell,et al.  Theoretical aspects of object-oriented programming: types, semantics, and language design , 1994, Choice Reviews Online.

[48]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[49]  Marco Dorigo,et al.  Swarm intelligence: from natural to artificial systems , 1999 .

[50]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[51]  J. Kennedy,et al.  Population structure and particle swarm performance , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[52]  Peter M. Williams,et al.  Bayesian Regularization and Pruning Using a Laplace Prior , 1995, Neural Computation.

[53]  E. Thorndike.  On the Organization of Intellect. , 1921 .

[54]  N. Cliff.  Answering Ordinal Questions with Ordinal Data Using Ordinal Statistics. , 1996, Multivariate behavioral research.

[55]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[56]  Guy L. Steele,et al.  Java Language Specification, Second Edition: The Java Series , 2000 .

[57]  David Corne,et al.  Evolutionary Computation In Bioinformatics , 2003 .

[58]  Mu-Chun Su,et al.  An efficient initialization scheme for the self-organizing feature map algorithm , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[59]  David E. Rumelhart,et al.  Product Units: A Computationally Powerful and Biologically Plausible Extension to Backpropagation Networks , 1989, Neural Computation.

[60]  Elliotte Rusty Harold,et al.  XML in a Nutshell , 2001 .

[61]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[62]  Andries Petrus Engelbrecht,et al.  A new particle swarm optimiser for linearly constrained optimisation , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[63]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[64]  Yishai A. Feldman,et al.  Algorithmics: The Spirit of Computing , 1987 .

[65]  Zoran Budimlic,et al.  Compiling java for high performance and the internet , 2001 .

[66]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[67]  Martin Fowler,et al.  Patterns of Enterprise Application Architecture , 2002 .

[68]  Gary Riley,et al.  Expert Systems: Principles and Programming , 2004 .

[69]  Peter Sommerlad,et al.  Pattern-Oriented Software Architecture , 1996 .

[70]  Steven John Metsker.  Design Patterns in C , 2004 .

[71]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[72]  S. Breckler.  Applications of covariance structure modeling in psychology: cause for concern? , 1990, Psychological bulletin.

[73]  Norman J. Walsh,et al.  DocBook: The Definitive Guide , 1999 .

[74]  Roberto Battiti,et al.  First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method , 1992, Neural Computation.

[75]  Peter J. Fleming,et al.  Genetic Algorithms for Multiobjective Optimization: FormulationDiscussion and Generalization , 1993, ICGA.

[76]  Michael R. Anderberg,et al.  Cluster Analysis for Applications , 1973 .

[77]  Amaury Lendasse,et al.  A robust nonlinear projection method , 2000 .

[78]  David G. Green.  The Serendipity Machine: A Voyage of Discovery Through the Unexpected World of Computers , 2004 .

[79]  A. Engelbrecht,et al.  A new locally convergent particle swarm optimiser , 2002, IEEE International Conference on Systems, Man and Cybernetics.

[80]  David B. Fogel,et al.  Evolution-ary Computation 1: Basic Algorithms and Operators , 2000 .

[81]  E. H. Mamdani,et al.  An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller , 1999, Int. J. Man Mach. Stud..

[82]  Frank Buschmann,et al.  A system of patterns , 1995 .

[83]  Thomas Kiel Rasmussen,et al.  Hybrid Particle Swarm Optimiser with breeding and subpopulations , 2001 .

[84]  Clifton Nock.  Data Access Patterns: Database Interactions in Object-Oriented Applications , 2003 .

[85]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[86]  Osvaldo Pinali Doederlein.  The Tale of Java Performance , 2003, J. Object Technol..

[87]  Ramez Elmasri,et al.  Fundamentals of Database Systems , 1989 .

[88]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[89]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[90]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[91]  G. Hardin,et al.  The Tragedy of the Commons , 1968, Green Planet Blues.

[92]  D. Wolpert,et al.  No Free Lunch Theorems for Search , 1995 .

[93]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[94]  John Crupi,et al.  Core J2EE Patterns: Best Practices and Design Strategies , 2001 .

[95]  Abraham Kandel,et al.  Compensatory Genetic Fuzzy Neural Networks and Their Applications , 1998, Series in Machine Perception and Artificial Intelligence.

[96]  Andries Petrus Engelbrecht,et al.  Using neighbourhoods with the guaranteed convergence PSO , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[97]  Kent L. Beck,et al.  Extreme programming explained - embrace change , 1990 .

[98]  Frans van den Bergh,et al.  An analysis of particle swarm optimizers , 2002 .

[99]  Carl J. Huberty,et al.  Statistical Practices of Educational Researchers: An Analysis of their ANOVA, MANOVA, and ANCOVA Analyses , 1998 .

[100]  Andries P. Engelbrecht,et al.  Computational Intelligence: An Introduction , 2002 .

[101]  Lotfi A. Zadeh,et al.  The Concepts of a Linguistic Variable and its Application to Approximate Reasoning , 1975 .

[102]  James Kennedy,et al.  Bare bones particle swarms , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).