SparseFIS: Data-Driven Learning of Fuzzy Systems With Sparsity Constraints

In this paper, we deal with a novel data-driven learning method [sparse fuzzy inference systems (SparseFIS)] for Takagi-Sugeno (T-S) fuzzy systems, extended by including rule weights. Our learning method consists of three phases: The first phase conducts a clustering process in the input/output feature space with iterative vector quantization and projects the obtained clusters onto 1-D axes to form the fuzzy sets (centers and widths) in the antecedent parts of the rules. Hereby, the number of clusters = rules is predefined and denotes a kind of upper bound on a reasonable granularity. The second phase optimizes the rule weights in the fuzzy systems with respect to least-squares error measure by applying a sparsity-constrained steepest descent-optimization procedure. Depending on the sparsity threshold, weights of many or a few rules can be forced toward 0, thereby, switching off (eliminating) some rules (rule selection). The third phase estimates the linear consequent parameters by a regularized sparsity-constrained-optimization procedure for each rule separately (local learning approach). Sparsity constraints are applied in order to force linear parameters to be 0, triggering a feature-selection mechanism per rule. Global feature selection is achieved whenever the linear parameters of some features in each rule are (near) 0. The method is evaluated, which is based on high-dimensional data from industrial processes and based on benchmark datasets from the Internet and compared with well-known batch-training methods, in terms of accuracy and complexity of the fuzzy systems.