Spectral form factor in non-Gaussian random matrix theories