The non-classical nucleation of crystals: microscopic mechanisms and applications to molecular crystals, ice and calcium carbonate