BESSEL FUNCTIONS OF MATRIX ARGUMENT

Our principal results fall into three main classes. First, a large number of formulae from the classical theory of special functions are given appropriate generalizations. Some of these turn out to have applications to lattice-point problems and to the theory of non-central Wishart distributions in statistics. Secondly, the L2-theory of the Hankel transform is established with the generalized Bessel functions furnishing the kernel, i.e. the transformation g(A) = JM>oA y(AM)f(M) (det M)7 dM is a self-reciprocal unitary correspondence of the Hilbert space of functions for which f A>O I f(A) 12 (det A)7 dA < c* onto itself. Here Py is a real number greater than -1 and A and M are positive definite matrices. In this connection there are two results we wish to emphasize. (1) A complete set of eigenfunctions for the Hankel transform is given in the form' etr(- A)L(') (2A), t running over a certain index class, with the L() (A) as polynomials in the entries of the matrix A. These polynomials enjoy generalized versions of nearly all the properties of the Laguerre polynomials to which they reduce in the scalar case. (2) The ordinary multi-dimensional Fourier transform of a function of mk variables satisfying a certain generalized radiality condition reduces to a Hankel transform. More precisely, arrange the mk variables in a k X m matrix T; then if the function depends only on R = T'T, T' being the transposed matrix, the Fourier transform may be computed in terms of the Hankel transform of order -y = 2 (k - m - 1) defined for functions of positive semidefinite m X m matrices R. The third class of results concerns the properties of harmonic polynomials in several variables having a certain matrix homogeneity. We call a polynomial, P(T), in the entries of the k X m matrix T, an H-polynomial if (1) P(T) is a harmonic function of mk variables and (2) P(TZ) = (det Z)VP(T) for some integer v and all m X m symmetric matrices Z. These H-polynomials behave like "Stieffel-manifold" (in contrast to "spherical") harmonics. They are related in a natural way to generalized Gegenbauer polynomials which are in turn defined as hypergeometric functions.