Spectral methods for nonlinear functionals and functional differential equations

We present a rigorous convergence analysis for cylindrical approximations of nonlinear functionals, functional derivatives, and functional differential equations (FDEs). The purpose of this analysis is twofold: first, we prove that continuous nonlinear functionals, functional derivatives and FDEs can be approximated uniformly on any compact subset of a real separable Hilbert space by high-dimensional multivariate functions and high-dimensional partial differential equations (PDEs), respectively. Second, we show that the convergence rate of such functional approximations can be exponential, depending on the regularity of the functional (in particular its Frechet differentiability), and its domain. We also provide necessary and sufficient conditions for consistency, stability and convergence of functional approximations to linear FDEs. These results open the possibility to utilize numerical techniques for high-dimensional model representation such as deep neural networks and numerical tensor methods to approximate nonlinear functionals in terms of high-dimensional functions, and compute approximate solutions to FDEs by solving high-dimensional PDEs. Numerical demonstrations are presented and discussed for prototype nonlinear functionals in the space of square-integrable functions, and for an initial value problem involving a linear FDE.