Long-Term Gravitational Quenching by Clumpy Accretion in Galaxies and Clusters