Self-testing software probe system for failure detection and diagnosis

A key problem in today's complex software systems is software failure detection and isolation given that most software failures are only partial, and if efficiently diagnosed, isolated and recovered, could avert a total outage. The probe detects failed software components in a running software system by requesting service, or a certain level of service, from a set of functions, modules and/or subsystems (target) and checking the response to the request. The objective is to localize the failure only up to the level of a target, while, achieving a high degree of efficiency and confidence in the process. Targets can be identified at different levels or layers in the software. The choice is based on the granularity of fault detection that is desired, taken in consideration with the level at which recovery is implemented. The implementation of the probe system is made self testing against any single failure in its operational components, using the idea of a null probe. The probe system has been designed, taking advantage of the latency characteristics of errors, to provide a low-overhead mechanism. The ideas are implementable in either a single or multiple computer system.