Estimation of cumulants and large deviations for certain nonlinear transformations of a stationary Gaussian process