Machine Intelligence Mixture of Experts and Bayesian Networks

In Chap. 6, we introduced Bayesian networks and some of the traditionAbstractal methods for designing them from data and pointed out some of the challenges with these approaches. In this chapter, we introduce a much simpler approach that escapes some of these challenges. This approach, we admit, will clearly not be an adequate substitute for all BN applications, but may well be adequate for many classification tasks. This approach uses a simple three-tier network, with the central node (or nodes) for the classification, an input layer for variables that influence the probability of membership in each class, and an output layer for variables whose probability is influenced by the class. We did not invent this approach, but we show how the MI methods in previous chapters, particularly feature subset selection, can greatly simplify the task of specifying the BN topology by reducing the available features to a small and fairly uncorrelated set. This does two things. By reducing the number features, we reduce the data requirements—maybe to manageable levels. By finding uncorrelated features, the assumptions of the simple three-tier design are less likely to be violated.