Automatically Defined Groups for Knowledge Acquisition from Computer Logs and Its Extension for Adaptive Agent Size

Recently, a large amount of data is stored in databases through the advance of computer and network environments. To acquire knowledge from the databases is important for analyses of the present condition of the systems and for predictions of coming incidents. The log file is one of the databases stored automatically in computer systems. Unexpected incidents such as system troubles as well as the histories of daily service programs’ actions are recorded in the log files. System administrators have to check the messages in the log files in order to analyze the present condition of the systems. However, the descriptions of the messages are written in various formats according to the kinds of service programs and application software. It may be difficult to understand the meaning of the messages without the manuals or specifications. Moreover, the log files become enormous, and important messages are liable to mingle with a lot of insignificant messages. Therefore, checking the log files is a troublesome task for administrators. Log monitoring tools such as SWATCH [1], in which regular expressions for representing problematic phrases are used for pattern matching, are effective for detecting well-known typical error messages. However, various programs running in the systems may be open source software or software companies’ products, and they may have been newly developed or upgraded recently. Therefore, it is impossible to detect all the problematic messages by the predefined rules. In addition, in order to cope with illegal use by hackers, it is important to detect unusual behavior such as the start of the unsupposed service program, even if the message does not correspond to the error message. To realize this system, the error-detection rules depending on the environment of the systems should be acquired adaptively by means of evolution or learning. Genetic programming (GP) [2] is one of the evolutionary computation methods, and it can optimize the tree structural programs. Much research on extracting rules from databases by GP has been done in recent years. In the research [3–5],

[1]  Hitoshi Iba.  Emergent Cooperation for Multiple Agents Using Genetic Programming , 1996, PPSN.

[2]  Akira Hara,et al.  Construction and Analysis of Stock Market Model Using ADG; Automatically Defined Groups , 2002, Int. J. Comput. Intell. Appl..

[3]  Tetsuyuki Takahama,et al.  Extraction of Risk Factors by Multi-agent Voting Model Using Automatically Defined Groups , 2005, KES.

[4]  Sandip Sen,et al.  Strongly Typed Genetic Programming in Evolving Cooperation Strategies , 1995, ICGA.

[5]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[6]  Akira Hara,et al.  Extraction of Error Detection Rules without Supervised Information from Log Files Using Automatically Defined Groups , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[7]  Laurence Hirsch,et al.  Evolving Rules for Document Classification , 2005, EuroGP.

[8]  Akira Hara,et al.  Emergence of the cooperative behavior using ADG; Automatically Defined Groups , 1999, GECCO.

[9]  Alex Alves Freitas,et al.  An Innovative Application of a Constrained-Syntax Genetic Programming System to the Problem of Predicting Survival of Patients , 2003, EuroGP.

[10]  Akira Hara,et al.  Knowledge Discovery and Data Mining in Medicine , 2005 .

[11]  Lee Spector,et al.  Evolving teamwork and coordination with genetic programming , 1996 .

[12]  Celia C. Bojarczuk,et al.  Genetic programming for knowledge discovery in chest-pain diagnosis. , 2000, IEEE engineering in medicine and biology magazine : the quarterly magazine of the Engineering in Medicine & Biology Society.

[13]  Tetsuyuki Takahama,et al.  Extraction of Rules by Heterogeneous Agents Using Automatically Defined Groups , 2003, KES.

[14]  Akira Hara,et al.  Discovering Multiple Diagnostic Rules from Coronary Heart Disease Database using Automatically Defined Groups , 2005, J. Intell. Manuf..