Wavelet based density estimation for multidimensional streaming data

In the present information economy, the colossal amount of data generated daily has spawned the need for realtime data driven algorithms that extract actionable intelligence with minimal user inuence. Density estimation is one path to extract this intelligence from the data and the incorporation of wavelets serves to boost the accuracy of the density estimation framework. The goal of this project was to develop a multidimensional computational implementation of this wavelet density estimation framework and showcase its utility on a relevant application. The report contains background information on wavelets, density estimation and all the details about the Matlab and Java implementations of the multidimensional wavelet density estimator. Finally, we demonstrate the utility of the approach by applying it to nancial market analysis.

[1]  George Tauchen,et al.  THE PRICE VARIABILITY-VOLUME RELATIONSHIP ON SPECULATIVE MARKETS , 1983 .

[2]  Haibo He,et al.  SOMKE: Kernel Density Estimation Over Data Streams by Sequences of Self-Organizing Maps , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Arnold P. Boedihardjo,et al.  Fast adaptive kernel density estimator for data streams , 2013, Knowledge and Information Systems.

[5]  C. Burrus,et al.  Introduction to Wavelets and Wavelet Transforms: A Primer , 1997 .

[6]  Adhemar Bultheel,et al.  Generalized cross validation for wavelet thresholding , 1997, Signal Process..

[7]  Jennifer Widom,et al.  Models and issues in data stream systems , 2002, PODS.

[8]  H. Cramér.  A contribution to the theory of statistical estimation , 1946 .

[9]  Shai Ben-David,et al.  Detecting Change in Data Streams , 2004, VLDB.

[10]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[11]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[12]  T. W. Epps,et al.  The Stochastic Dependence of Security Price Changes and Transaction Volumes: Implications for the Mixture-of-Distributions Hypothesis , 1976 .

[13]  Dimitrios Gunopulos,et al.  Online outlier detection in sensor data using non-parametric models , 2006, VLDB.

[14]  Lesław Gajek,et al.  On Improving Density Estimators which are not Bona Fide Functions , 1986 .

[15]  Helwig Hauser,et al.  Interactive visualization of streaming data with Kernel Density Estimation , 2011, 2011 IEEE Pacific Visualization Symposium.

[16]  Li Wei,et al.  M-kernel merging: towards density estimation over data streams , 2003, Eighth International Conference on Database Systems for Advanced Applications, 2003. (DASFAA 2003). Proceedings..

[17]  P. Clark.  A Subordinated Stochastic Process Model with Finite Variance for Speculative Prices , 1973 .

[18]  Liam Paninski,et al.  Estimation of Entropy and Mutual Information , 2003, Neural Computation.

[19]  Martin Vetterli,et al.  Adaptive wavelet thresholding for image denoising and compression , 2000, IEEE Trans. Image Process..

[20]  Bernhard Seeger,et al.  Maintaining Nonparametric Estimators over Data Streams , 2005, BTW.

[21]  Teruko Takada.  Bivariate Nonparametric Density Estimation of Stock Prices and Volume , 2001 .

[22]  Sanjay Ranka,et al.  Statistical change detection for multi-dimensional data , 2007, KDD '07.

[23]  Sudipto Guha,et al.  Dynamic multidimensional histograms , 2002, SIGMOD '02.

[24]  I. Johnstone,et al.  Density estimation by wavelet thresholding , 1996 .

[25]  Steven K. Donoho,et al.  Early detection of insider trading in option markets , 2004, KDD.

[26]  Wim Sweldens,et al.  An Overview of Wavelet Based Multiresolution Analyses , 1994, SIAM Rev..

[27]  C. Gargour,et al.  A short introduction to wavelets and their applications , 2009, IEEE Circuits and Systems Magazine.

[28]  Christopher Heil,et al.  Continuous and Discrete Wavelet Transforms , 1989, SIAM Rev..