Risk Bounds for Levy Processes in the PAC-Learning Framework

Levy processes play an important role in the stochastic process theory. However, since samples are non-i.i.d., statistical learning re- sults based on the i.i.d. scenarios cannot be utilized to study the risk bounds for Levy processes. In this paper, we present risk bounds for non-i.i.d. samples drawn from Levy processes in the PAC-learning frame- work. In particular, by using a concentra- tion inequality for infinitely divisible distri- butions, we first prove that the function of risk error is Lipschitz continuous with a high probability, and then by using a specific con- centration inequality for Levy processes, we obtain the risk bounds for non-i.i.d. samples drawn from Levy processes without Gaus- sian components. Based on the resulted risk bounds, we analyze the factors that affect the convergence of the risk bounds and then prove the convergence.

[1]  O. Barndorff-Nielsen,et al.  Lévy processes : theory and applications , 2001 .

[2]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[3]  Bin Yu RATES OF CONVERGENCE FOR EMPIRICAL PROCESSES OF STATIONARY MIXING SEQUENCES , 1994 .

[4]  Peter L. Bartlett,et al.  Localized Rademacher Complexities , 2002, COLT.

[5]  Michael B. Marcus,et al.  $L^1$-Norm of Infinitely Divisible Random Vectors and Certain Stochastic Integrals , 2001 .

[6]  V. V. Petrov Limit Theorems of Probability Theory: Sequences of Independent Random Variables , 1995 .

[7]  D. Applebaum Lévy Processes and Stochastic Calculus , 2004 .

[8]  Robert H. Sloan,et al.  Proceedings of the 15th Annual Conference on Computational Learning Theory , 2002 .

[9]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.

[10]  O. Kallenberg On Symmetrically Distributed Random Measures , 1975 .

[11]  O. Kallenberg Canonical representations and convergence criteria for processes with interchangeable increments , 1973 .

[12]  C. Houdr'e,et al.  Median, Concentration and Fluctuation for Lévy Processes , 2006, math/0607022.

[13]  Mehryar Mohri,et al.  Rademacher Complexity Bounds for Non-I.I.D. Processes , 2008, NIPS.

[14]  C. Houdré Remarks on deviation inequalities for functions of infinitely divisible random vectors , 2002 .

[15]  David Applebaum Levy processes - from probability theory to finance and quantum groups , 2004 .

[16]  O. Bousquet Concentration Inequalities and Empirical Processes Theory Applied to the Analysis of Learning Algorithms , 2002 .

[17]  A. Dembo,et al.  A note on uniform laws of averages for dependent processes , 1993 .

[18]  C. Houdr'e,et al.  Risk bounds for the non-parametric estimation of Lévy processes , 2006, math/0612697.

[19]  A. Tsybakov,et al.  Optimal aggregation of classifiers in statistical learning , 2003 .

[20]  Nathan Srebro,et al.  Fast Rates for Regularized Objectives , 2008, NIPS.