An exploratory investigation on the invasiveness of environmental modeling frameworks

Environmental modeling frameworks provide an array of useful features that model developers can harness when implementing models. Each framework differs in how it provides features to a model developer via its Application Programming Interface (API). Environmental modelers harness framework features by calling and interfacing with the framework API. As modelers write model code, they make framework-specific function calls and use framework specific data types for achieving the functionality of the model. As a result of this development approach, model code becomes coupled with and dependent on a specific modeling framework. Coupling to a specific framework makes migration to other frameworks and reuse of the code outside the original framework more difficult. This complicates collaboration between model developers wishing to share model code that may have been developed in a variety of languages and frameworks. This paper provides initial results of an exploratory investigation on the invasiveness of environmental modeling frameworks. Invasiveness is defined as the coupling between application (i.e., model) and framework code used to implement the model. By comparing the implementation of an environmental model across several modeling frameworks, we aim to better understand the consequences of framework design. How frameworks present functionality to modelers through APIs can lead to consequences with respect to model development, model maintenance, reuse of model code, and ultimately collaboration among model developers. By measuring framework invasiveness, we hope to provide environmental modeling framework developers and environmental modelers with valuable information to assist in future development efforts. Eight implementations (six framework-based) of Thornthwaite, a simple water balance model, were made in a variety of environmental modeling frameworks and languages. A set of software metrics were proposed and applied to measure invasiveness between model implementation code and framework code. The metrics produced a rank ordering of invasiveness for the framework-based implementations of Thornthwaite. We compared model invasiveness results with several popular software metrics including size in lines of code (LOC), cyclomatic complexity, and object oriented coupling. To investigate software quality implications of framework invasiveness we checked for relationships between the Chidamber and Kemerer (1994) object oriented software metrics and our framework invasiveness measures. For the six framework-based implementations of Thornthwaite we found a five-fold variation in code size (LOC). We observed up to a seven-fold variation in total cyclomatic complexity, and a two to three-fold variation in object oriented coupling. For the model implementations we found that total size, total complexity, and total coupling all had a significant positive correlation. The raw count version of our invasiveness measures correlated with application size (LOC), total cyclomatic complexity, total efferent coupling (fan out) and total afferent coupling (fan in). Large size, complexity, and high levels of coupling between units (classes, modules) in a software system are often cited in software engineering as causes of high maintenance costs due to poor understandability and flexibility of the code. This study provides initial results but further investigation is desired to evaluate the utility of our invasiveness measurement approach as well as the software quality implications of framework invasiveness.