Evaluation of Data Reduction Algorithms for Real-Time Analysis

Abstract Data reduction tools are developed and evaluated using a data analysis framework. Simple (nonadaptive) and intelligent (adaptive) thinning algorithms are applied to both synthetic and real data and the thinned datasets are ingested into an analysis system. The approach is motivated by the desire to better represent high-impact weather features (e.g., fronts, jets, cyclones, etc.) that are often poorly resolved in coarse-resolution forecast models and to efficiently generate a set of initial conditions that best describes the current state of the atmosphere. As a precursor to real-data applications, the algorithms are applied to one- and two-dimensional synthetic datasets. Information gleaned from the synthetic experiments is used to create a thinning algorithm that combines the best aspects of the intelligent methods (i.e., their ability to detect regions of interest) while reducing the impacts of spatial irregularities in the data. Both simple and intelligent thinning algorithms are then applied...

[1]  Istvan Szunyogh,et al.  The North Pacific Experiment (NORPEX-98): Targeted Observations for Improved North American Weather Forecasts , 1999 .

[2]  F. Rabier,et al.  The potential of high‐density observations for numerical weather prediction: A study with simulated observations , 2003 .

[3]  Robert J. Renard,et al.  EXPERIMENTS IN NUMERICAL OBJECTIVE FRONTAL ANALYSIS1 , 1965 .

[4]  Bengt Hallmans,et al.  Introduction to BOOT , 1999 .

[5]  A. Lorenc.  A Global Three-Dimensional Multivariate Statistical Interpolation Scheme , 1981 .

[6]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[7]  G. Peckham.  The information content of remote measurements of atmospheric temperature by satellite infra‐red radiometry and optimum radiometer configurations , 1974 .

[8]  Timothy J. Hoar,et al.  Assimilation of Surface Pressure Observations Using an Ensemble Filter in an Idealized Global Atmospheric Prediction System , 2005 .

[9]  Jordan G. Powers,et al.  A Description of the Advanced Research WRF Version 2 , 2005 .

[10]  Charles A. Doswell,et al.  Radar Data Objective Analysis , 2000 .

[11]  J. Horel,et al.  Near-Real-Time Applications of a Mesoscale Analysis System to Complex Terrain , 2002 .

[12]  J. R. Eyre,et al.  The information content of data from satellite sounding systems: A simulation study , 1990 .

[13]  Florence Rabier,et al.  The interaction between model resolution, observation resolution and observation density in data assimilation: A one‐dimensional study , 2002 .

[14]  C. Rodgers,et al.  Retrieval of atmospheric temperature and composition from remote measurements of thermal radiation , 1976 .

[15]  Qin Xu,et al.  Measuring information content from observations for data assimilation: relative entropy versus shannon entropy difference , 2007 .

[16]  N. Dyn,et al.  Adaptive thinning for bivariate scattered data , 2002 .

[17]  G. Evensen.  Sequential data assimilation with a nonlinear quasi‐geostrophic model using Monte Carlo methods to forecast error statistics , 1994 .

[18]  David D. Turner,et al.  ARM Site Atmospheric State Best Estimates for AIRS Validation , 2003 .

[19]  Xiang-Yu Huang,et al.  Variational Analysis Using Spatial Filters , 2000 .

[20]  Steven E. Koch,et al.  An interactive Barnes objective map analysis scheme for use with satellite and conventional data , 1983 .

[21]  Chris Snyder,et al.  Summary of an Informal Workshop on Adaptive Observations and FASTEX , 1996 .

[22]  Kenneth H. Bergman,et al.  Analysis error as a function of observation density for satellite temperature soundings with spatially correlated errors , 1976 .

[23]  Dietmar Saupe,et al.  ON THINNING METHODS FOR DATA ASSIMILATION OF SATELLITE OBSERVATIONS , 2007 .

[24]  Florence Rabier,et al.  An update on THORPEX-related research in data assimilation and observing strategies , 2008 .

[25]  Christopher D. Barnet,et al.  Accuracy of geophysical parameters derived from Atmospheric Infrared Sounder/Advanced Microwave Sounding Unit as a function of fractional cloud cover , 2006 .

[26]  D. Saupe,et al.  THE INTERACTION BETWEEN MODEL RESOLUTION, OBSERVATION RESOLUTION AND OBSERVATION DENSITY IN DATA ASSIMILATION: A TWO-DIMENSIONAL STUDY , 2006 .

[27]  Andrew C. Lorenc,et al.  Analysis methods for numerical weather prediction , 1986 .

[28]  G. Dimego,et al.  The National Meteorological Center Regional Analysis System , 1988 .

[29]  N. Roberts,et al.  Numerical Aspects of the Application of Recursive Filters to Variational Statistical Analysis. Part II: Spatially Inhomogeneous and Anisotropic General Covariances , 2003 .

[30]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[31]  L. Larrabee Strow,et al.  Atmospheric Radiation Measurement site atmospheric state best estimates for Atmospheric Infrared Sounder temperature and water vapor retrieval validation , 2006 .

[32]  William L. Smith,et al.  AIRS/AMSU/HSB on the Aqua mission: design, science objectives, data products, and processing systems , 2003, IEEE Trans. Geosci. Remote. Sens..

[33]  R. James Purser,et al.  Recursive Filter Objective Analysis of Meteorological Fields: Applications to NESDIS Operational Processing , 1995 .

[34]  Jidong Gao,et al.  A Three-Dimensional Variational Data Analysis Method with Recursive Filter for Doppler Radars , 2004 .

[35]  Tilo Ochotta,et al.  Adaptive thinning of atmospheric observations in data assimilation with vector quantization and filtering methods , 2005 .

[36]  K. Novak,et al.  Collaborative effort , 2002, Nature Reviews Cancer.

[37]  Li Wei,et al.  Measuring information content from observations for data assimilations: connection between different measures and application to radar scan design , 2009 .