Practical Considerations for Use of Causation Entropy in Sparsity Identification

The selection of model structure is an important step in system identification for nonlinear systems in cases where the model form is not known a priori. This process, sometimes called covariate selection or sparsity identification, involves the selection of terms in the dynamic model and is performed prior to parameter estimation. Previous work has shown the applicability of an information theory quantity known as causation entropy in performing sparsity identification. While prior work established the overall feasibility of using causation entropy to eliminate extraneous terms in a model, key questions remained regarding practical implementation. This paper builds on previous work to explore key practical considerations of causation entropy sparsity identification. First, the effect of data size is explored through both analysis and simulation, and general guidance is provided on how much data is necessary to produce accurate causation entropy estimates. Second, the effects of measurement noise and model discretization error are investigated, showing that both cause degradation of the causation entropy estimation accuracy but in opposite ways. These practical effects and trends are illustrated on several example nonlinear systems. Overall, results show that the causation entropy approach is a practical technique for sparsity identification particularly in light of the guidelines presented here for data size selection and handling of error sources.