A Visual Analysis Concept for the Validation of Geoscientific Simulation Models

Geoscientific modeling and simulation helps to improve our understanding of the complex Earth system. During the modeling process, validation of the geoscientific model is an essential step. In validation, it is determined whether the model output shows sufficient agreement with observation data. Measures for this agreement are called goodness of fit. In the geosciences, analyzing the goodness of fit is challenging due to its manifold dependencies: 1) The goodness of fit depends on the model parameterization, whose precise values are not known. 2) The goodness of fit varies in space and time due to the spatio-temporal dimension of geoscientific models. 3) The significance of the goodness of fit is affected by resolution and preciseness of available observational data. 4) The correlation between goodness of fit and underlying modeled and observed values is ambiguous. In this paper, we introduce a visual analysis concept that targets these challenges in the validation of geoscientific models - specifically focusing on applications where observation data is sparse, unevenly distributed in space and time, and imprecise, which hinders a rigorous analytical approach. Our concept, developed in close cooperation with Earth system modelers, addresses the four challenges by four tailored visualization components. The tight linking of these components supports a twofold interactive drill-down in model parameter space and in the set of data samples, which facilitates the exploration of the numerous dependencies of the goodness of fit. We exemplify our visualization concept for geoscientific modeling of glacial isostatic adjustments in the last 100,000 years, validated against sea levels indicators - a prominent example for sparse and imprecise observation data. An initial use case and feedback from Earth system modelers indicate that our visualization concept is a valuable complement to the range of validation methods.

[1]  Martin Mladenov,et al.  Identifying Place Histories from Activity Traces with an Eye to Parameter Impact , 2012, IEEE Transactions on Visualization and Computer Graphics.

[2]  Hans-Christian Hege,et al.  Tuner: Principled Parameter Finding for Image Segmentation Algorithms Using Visual Response Surface Exploration , 2011, IEEE Transactions on Visualization and Computer Graphics.

[3]  E. Gröller,et al.  Nodes on Ropes: A Comprehensive Data and Control Flow for Steering Ensemble Simulations , 2011, IEEE Transactions on Visualization and Computer Graphics.

[4]  Patrick Wu,et al.  Glacial isostatic adjustment in Fennoscandia-A review of data and modeling , 2011 .

[5]  Heidrun Schumann,et al.  Visual analytics for stochastic simulation in cell biology , 2011, i-KNOW '11.

[6]  Thomas Nocke,et al.  Information Visualization in Climate Research , 2011, 2011 15th International Conference on Information Visualisation.

[7]  Peter Filzmoser,et al.  Uncertainty‐Aware Exploration of Continuous Parameter Spaces Using Multivariate Prediction , 2011, Comput. Graph. Forum.

[8]  W. Aigner,et al.  Visualization of Time-Oriented Data , 2011, Human-Computer Interaction Series.

[9]  Alan M. MacEachren,et al.  Geo-historical context support for information foraging and sensemaking: Conceptual model, implementation, and assessment , 2010, 2010 IEEE Symposium on Visual Analytics Science and Technology.

[10]  Jian Huang,et al.  Scalable Multi-variate Analytics of Seismic and Satellite-based Observational Data , 2010, IEEE Transactions on Visualization and Computer Graphics.

[11]  James P. Ahrens,et al.  Verifying Scientific Simulations via Comparative and Quantitative Visualization , 2010, IEEE Computer Graphics and Applications.

[12]  Eduard Gröller,et al.  World Lines , 2010, IEEE Transactions on Visualization and Computer Graphics.

[13]  Denis Gracanin,et al.  Interactive Visual Analysis of Multiple Simulation Runs Using the Simulation Model View: Understanding and Tuning of an Electronic Unit Injector , 2010, IEEE Transactions on Visualization and Computer Graphics.

[14]  Stefan Bruckner,et al.  Result-Driven Exploration of Simulation Parameter Spaces for Visual Effects Design , 2010, IEEE Transactions on Visualization and Computer Graphics.

[15]  Andrew Mercer,et al.  Noodles: A Tool for Visualization of Numerical Weather Model Ensemble Uncertainty , 2010, IEEE Transactions on Visualization and Computer Graphics.

[16]  Doris Dransch,et al.  Assessing the quality of geoscientific simulation models with visual analytics methods – a design study , 2010, Int. J. Geogr. Inf. Sci..

[17]  Heidrun Schumann,et al.  Space, time and visual analytics , 2010, Int. J. Geogr. Inf. Sci..

[18]  Helwig Hauser,et al.  Exploration of Climate Data Using Interactive Visualization , 2010 .

[19]  Valerio Pascucci,et al.  Ensemble-Vis: A Framework for the Statistical Visualization of Ensemble Data , 2009, 2009 IEEE International Conference on Data Mining Workshops.

[20]  Frank Scherbaum,et al.  Unsupervised feature selection and general pattern discovery using Self-Organizing Maps for gaining insights into the nature of seismic wavefields , 2009, Comput. Geosci..

[21]  Nelis Franken,et al.  Visual exploration of algorithm parameter space , 2009, 2009 IEEE Congress on Evolutionary Computation.

[22]  Heidrun Schumann,et al.  Visual support for the understanding of simulation processes , 2009, 2009 IEEE Pacific Visualization Symposium.

[23]  H. Hauser,et al.  Interactive Visual Steering - Rapid Visual Prototyping of a Common Rail Injection System , 2008, IEEE Transactions on Visualization and Computer Graphics.

[24]  Helwig Hauser,et al.  Hypothesis Generation in Climate Research with Interactive Visual Data Exploration , 2008, IEEE Transactions on Visualization and Computer Graphics.

[25]  Thomas Nocke,et al.  Visual exploration and evaluation of climate-related simulation data , 2007, 2007 Winter Simulation Conference.

[26]  Laura Painton Swiler,et al.  Calibration, validation, and sensitivity analysis: What's what , 2006, Reliab. Eng. Syst. Saf..

[27]  Gennady L. Andrienko,et al.  Exploratory analysis of spatial and temporal data - a systematic approach , 2005 .

[28]  M. Watkins,et al.  The gravity recovery and climate experiment: Mission overview and early results , 2004 .

[29]  Louis Moresi,et al.  Effective exploration and visualization of geological parameter space , 2003 .

[30]  A. B. WATTS,et al.  Isostasy and Flexure of the Lithosphere , 2001 .

[31]  Paul Johnston,et al.  Reply to comment by W. Fjeldskaar ‘What about the asthenosphere viscosity? Sea‐level change, glacial rebound and mantle viscosity for northern Europe’ , 2000 .

[32]  D. Legates,et al.  Evaluating the use of “goodness‐of‐fit” Measures in hydrologic and hydroclimatic model validation , 1999 .

[33]  Paul Johnston,et al.  Sea‐level change, glacial rebound and mantle viscosity fornorthern Europe , 1998 .

[34]  Alan M. MacEachren,et al.  How Maps Work - Representation, Visualization, and Design , 1995 .

[35]  M. Debbabi,et al.  Verification, Validation, and Accreditation , 2010 .

[36]  Hsinchun Chen,et al.  Evaluating event visualization: a usability study of COPLINK spatio-temporal visualizer , 2005, Int. J. Hum. Comput. Stud..

[37]  V. Liere,et al.  Hyperslice visualization of scalar functions of many variables , 1994 .

[38]  Donald R. Lehmann,et al.  Validity and Goodness of Fit in Data Analysis , 1975 .

[39]  Helwig Hauser,et al.  Accepted for Publication in Ieee Transactions on Visualization and Computer Graphics, Authors' Personal Copy 1 Interactive Visual Analysis of Heterogeneous Scientific Data across an Interface , 2022 .

[40]  Wolfgang Berger,et al.  Eurographics/ Ieee-vgtc Symposium on Visualization 2010 Hypermoval: Interactive Visual Validation of Regression Models for Real-time Simulation , 2022 .