Testing nonstandard neutrino matter interactions in atmospheric neutrino propagation